[NextGen] Carmack... un uomo un mito

[.:|Physalis|:.]

Probabilmente dato che risale alla QuakeCon........ ma comuqnue interessante...



John Carmack's unscripted presentations each year at QuakeCon have always been a highlight, even if the last couple years have been erratic (a cold shut him down in 2003, and last year he videotaped his presentation in advance because his son was born during the show). His talk, aimed at Quake fans, aspiring developers, and graphics nuts, is never really a scripted affair. Carmack himself describes it as a "stream of consciousness," his own personal ramblings on what's happening in the industry.

But still, as one of the premiere real-time graphics coders on the planet and creator of the game engines that have powered some of the most influential PC games in history (Wolfenstein, DOOM, Quake, etc.) his unedited thoughts on where the industry is going offer some pretty valuable insights. This year Carmack stuck to a couple of themes: The varying architecture differences of the next-gen consoles and how to nurture creativity in an overly-conservative industry.

There were also a couple surprises in store, including how he plans to release the Quake 3 source code under the GPL license within the next week or so! Parts of the talk, as always, were pretty technical so once again (with apologies to the Slashdot crowd) we'll do our best to boil it down to layman's terms.


The Breakneck Pace of Graphics

There used to be a saying back in the early days of 3D rendering where someone quipped, "Reality is 80 million polygons per second." It was a great catchphrase at the time because during those days, it was an obscene number -- ridiculously beyond the technology of the day. But nowadays, Carmack points out, even cheap console systems can crank out that kind of performance. More importantly, today's technology can render that many polygons in realtime with tons more features than was ever available before, such as programmability or multiple passes. And the high-end keeps getting higher.

At this level, are graphics all that important? Carmack says that sometimes he gets static from other game developers, who say (only half-jokingly) that id Software has raised the graphical bar so high that other companies have to scramble and dedicate tons of resources to catch up. Once consumers see that level of graphics, they expect it from every title. Carmack says he's sympathetic to that, he understands where they're coming from. But at the same time, he made it clear that he really sees huge value in what advanced graphics bring to the table. People can talk about how gameplay is king, but he asserts that technology allows you to make a really memorable experience. "Presentation really is important," he explains. He admits it would be nice not to have to do tons of motion-capture and cinematic stuff with every new game released, but argues that that extra layer of fidelity makes games better.

People may have fond memories of simpler gaming eras, but as much fun as Quake I was for its time, he asserts that it just doesn't have the presence or impact of modern games. "The golden age is right now!" he argues.


Development on Next-Gen Consoles

id Software has begun to step up its efforts in the console development space. It's not that the company is a stranger to console development (DOOM 3 was released on the Xbox shortly after the PC, and in days past id had worked on Nintendo projects), but now id Software is getting serious and hopes to have a near-simultaneous release for its next game.

Carmack's platform of choice for console development is the Xbox 360, and he explained why. But first, a bit of history:

Back in the DOOM era of game development, high-speed graphics were coded at the register level (basically in machine language). Then, as development moved from DOS to Windows and code started being done through APIs, game development became more abstract and higher-level. Carmack remembers a frustrating time when there were literally 20 different graphics chips that game makers had to code for, but nowadays there's only two to worry about. While coding through these layers is sometimes easier, it can also be frustrating. Carmack often finds himself wondering if a mistake was "my fault, the drivers fault, or the hardware's fault?"


But the Xbox 360 was designed to have a very thin API layer. In Carmack's words, he can "basically talk directly to the hardware ... doing exactly what I want."Here Carmack heaped praise on the decisions that Microsoft has made with the Xbox 360. "It's the best development environment I've seen on a console," he says. Microsoft has taken a very developer-centric approach, creating a system that's both powerful but easy to code for. This is in contrast to Nintendo, Sony, and (formerly) Sega, who generally focused on the hardware.

Carmack ruminated on how throughout history consoles have swung back and forth between providing high-end hardware or development tools. Until the PS1 came out, nearly everything was done at the register level, but Sony's first console shipped with tools to help speed the development process. This was in opposition to the Sega Saturn, which was very powerful but nearly impossible to efficiently code for. Then, with the release of the PS2, Sony flip-flopped: the PS2 had much more complicated hardware and you basically had to program it at the low level again. Then along came the Xbox, which didn't have low-level access but was way easier to program.

Carmack looks forward to what's coming up. "It'll be real interesting to see how this next generation pans out," he said. This time around, the Xbox 360 is coming out sooner and is easier to program; will it be enough to supplant Sony's market lead?


Single vs. Multithread Processing

Carmack points out that there could be diminishing returns with the next generation of consoles, thanks to the architecture of the hardware. Although the PS3's Cell processor is powerful, it's still a single-thread processor (unlike high-end PC processors). There's not much more technology can do to crank more speed out of a single-threaded processor. To make up for this deficiency, multiple processors are used: multithreading and parallelism is the way to go. But this isn't easy to do in games! "The returns will initially be disappointing," he explains.

It will take a long time before game developers will figure out how to get the most out of parallel processors, and in the meantime, it's going to make high-end game development more difficult -- "Not a good thing," Carmack says. There's no easy solution to this problem, no special compiler that'll take away the grunt work. "There's no silver bullet for parallel programming."


Sony's position seems to be similar to the company's stance with the PS2: Sure, it'll be hard, but the really good developers will suck it up and figure it out. But Carmack wonders aloud: wouldn't it have been better to use multi-threaded processors to begin with?

From here, he brought the talk around to discuss alternate ways to divide up game code to take advantage of multiple processors working in parallel, and why it's difficult...

Multiple Processors for AI or Physics

Proponents of faster and faster processors sometimes argue that now that graphics are reaching their 'peak,' extra processing power can be dedicated to calculation-intensive physics or Artificial Intelligence. (Carmack relates how an Engineer at IBM told him that graphics were basically "done.") Carmack disagrees, seeing that graphics still have a long way to go. "We'd like to be doing Lord-of-the-Rings type rendering in real-time," he states. That's still an order of magnitude more than what's possible with current machines, and Carmack is looking forward to it.

That aside, Carmack spent a few minutes talking about Artificial Intelligence as something that can be offloaded to another processor for a cutting-edge game. Carmack is skeptical. AI is a very bleeding-edge science, and it can often be processor intensive, but when applied to games AI is usually a matter of scripting. What game designers want is a way to act as the 'director,' telling enemy and friendly characters where to stand and what to do. This doesn't take a ton of processing power.

Moreover, even if you did throw tons of resources toward the AI, it might not be the best thing for gameplay. For instance, writing tons and tons of code to enable monsters to hide in the shadows and sneak around behind the player would be interesting, but often these types of things could be scripted for a fraction of the effort and - for most players - the experience would be just as cool if not cooler. Carmack recounts how players of the original DOOM would think that the monsters were doing all sorts of scheming and plotting and ambushing when, in truth, they were just using the equivalent of one page of C code and running the most basic of scripts.


So, if we're not going to need a parallel processor to run reams of AI calculations, what about physics simulation? Carmack thinks we're a long way off from seeing any major movement in this area. If you created a huge world where objects all around you were simulated in real physics, it would definitely be processor-intensive. But there are a number of problems.

"The problem with physics is that it's not scalable in orders of magnitude like graphics," he explains. There's no equivalent of 'level of detail' in physics - objects far away have to be simulated just the same as objects that are close. Furthermore, you can't decide not to simulate something just because the player doesn't see it on the screen; if you knock over a pile of crates, they have to fall over even if your back is turned. And of course you don't want the physics to work differently based on whether a player is looking at the object being simulated or not.
   

For this reason Carmack thinks we're still a ways away from seeing deep physics simulations being built into a game in such a way that they have a major impact on the gameplay. We'll see a lot of 'non-interactive' physics in the near future - such as realistic smoke pouring across the environment, or cooler simulations of liquid - but nothing major in the way of gameplay.

Carmack notes with some irony that in the same way id Software raises the graphical bar for games, other companies are raising the standards for physics, so that id has to integrate simulations into their games or they look dated.

He goes on to say that simulating a complex world is a hard problem. "Your game will either be fragile or slow," he says, and it had better do "something amazing" with regards to gameplay in order to make up for that. Carmack says that, in the next generation, he'd rather see processing power go towards making console games run at 60 frames per second rather than doing in-depth physics.

[.:|Physalis|:.]

Open vs. Closed Console Platforms

Carmack's talk changed gears at this point, starting with an aside about Sony. Although he raves about the Xbox 360 development tools, Carmack noted that Sony is making noise about making the PS3 a more open platform. As a big proponent of open source and - well, open anything - the programmer is excited to see if this goes anywhere. His biggest pet peeve with the console market is how closed off it is: you have to apply to be a developer and get a special development kit and get product approval from the hardware manufacturer, etc. (As opposed to the PC market, where anyone can develop and game creators can release content updates, point releases, and so on.) That's just the nature of the market. If the PS3 opens up, it'll be more like the old Commodore Amiga, a platform anyone can use for a variety of applications. Certainly Microsoft will never do this with the Xbox product line, but Carmack is holding out hope that Sony could experiment.

Following that train of thought, Carmack asked how many people in the room had HDTVs at home. (I was surprised at how few people raised their hands - from where I sat it looked like less than a quarter of the audience. And these guys are hardcore!) Carmack pointed out that any sort of plan for using a console or set-top box as a computing device really requires the high resolution of an HDTV or computer monitor to be effective. Bringing the topic around full circle, he pointed out that Microsoft may enact a policy of requiring all Xbox 360 games to be rendered at HDTV resolutions, regardless of whether or not that decision helps with the design of the game. He says that some marketing person probably made that decision, another thing that bothers him about the console industry.


Gaming Innovation in Small Packages

Now Carmack's talk changed gears completely. He recalled that several months ago he tinkered around with developing a game on a cell phone. (One nice thing about cell phone development is that it's still a very open platform - anyone who wants to learn can download Java and start programming.) Carmack says his experimental cell phone game was done in four months using just a few people. "There are a lot of really neat aspects to that," he says.

Here he drew a comparison between the computer games industry and his other favorite field, aerospace. Aerospace development is very, very slow because so much money is on the line with every mission. Every project is so mind-bogglingly expensive that there's no point where an engineer can walk up and say, "Hey guys, I've got this crazy idea..."




The Quake 3 source code will shortly be available!

Game development isn't as costly as aerospace but, as game development budgets spiral toward the hundred million dollar per title mark, a similar culture is developing. "There is a strong incentive not to do something nuts," Carmack explains. Which is why we see so many sequels and follow-ups on game shelves. Not that there's anything inherently wrong with that: Carmack asserts that Doom 3 or Quake 4 are fun titles that'll appeal to a lot of people. But it's hard for companies to bring fresh new titles or experimental genres to life.

Things are different in the cell phone space. Carmack put together a simple turn-based combat game for cell phones, small enough to fit into a 300K zip file. He described it as "almost a pure design exercise," and said that it got his creative juices flowing until he had "a backlog of a half dozen ideas" for other simple games. He theorized that small developers can create games on these platforms as test concepts, possibly channeling them into larger titles down the road.

Learning to code for the platform was easy and the results were fast. "I had the feel I had when I first started programming," he claims, pausing to talk about how hard it is for new programmers to come into the industry and figure out today's complicated AAA games. Which followed naturally into his next point...

Open Source, Modding, and Innovation

Carmack is aware of the difficulties for new programmers who want to get into the business: it's a long way from staring at a blank page in a compiler to having Doom 3. What can be done? Over the past several years Carmack has done his part by releasing the source codes to his previous game engines. Which led up to a big announcement:

Sometime over the next week or so, the Quake 3 source engine will be released under the GPL license. Since Q3 is still a viable development platform, the community should learn a lot from tearing into it. Carmack notes that the Punkbuster code has been removed so he hopes that cheating won't be a huge problem.


Carmack thinks that innovation in the game industry, since it won't often come from large professional development houses, will likely come from the modification or open source communities. There, people can try random ideas to see what works. Example? Counter-Strike, one of the most popular games played online today, which started out as a handful of mod-makers trying to make something fun.

The GPL license will allow people to take the Quake 3 engine and even go so far as to release a commercial product with it - provided that the source code is published alongside. Nobody has done this with any of the Quake engine games yet, but he hopes to see it happen someday.

As an aside, he noted that most companies are too secretive with their source code. "It's not about the magic in the source code," he says, it's about the tons of little decisions coders make along the way to producing a finished game. It's all in the execution.


   

Along with source code releases, and a commitment to mod tools for Quake 4, Carmack hopes that the current boom in cell phone gaming will help spur creativity in the industry. On the other hand, he notes that technology is moving so fast that even cell phone hardware development is growing exponentially. "Given a year or two we'll have PSP-level technology on cell phones," he says, which require large teams and budgets to put together. So that's one window that might be closing.

A Short Rant About Textures

Carmack wrapped things up with a short talk about his one pet peeve about the current state of graphics hardware. Firstly, he doesn't believe that any sort of procedural texture generation could ever look as good as something artists can put together with the right tools. Secondly, current hardware is based on the 'tiled textures' model, which he described as a "limited form of data compression," but that's not the only way to skin the texture cat.

Unique texturing, where artists can basically paint the textures of a whole scene using a set of tools and decals, can make a scene look incredible. Enemy Territory: Quake Wars is the first of id's attempts to play with this idea, visible in its huge but unique landscapes. "There's clearly a correct way to do this in hardware," Carmack claims, briefly explaining how to do so in technical terms. But the graphics hardware companies aren't taking the bait, and it's not being included in next-gen cards. He hopes to see that change.

Questions & Answers

A short Q&A followed the speech, but we'll only cover the highlights.

When asked about the differences in platforms, Carmack noted that there's less of a distinction between developing for the next-gen consoles as there is for this generation. The difference between the Xbox and the PS2 was massive, but going back and forth between the Xbox 360 and the PS3 looks to be less of a problem. (Although still difficult.)

PSP or DS? Carmack hasn't really looked at the two handheld systems in depth, but he says he really likes the PSP. Although the development tools aren't as good as what Microsoft would do, he likes the relatively simple architecture of the system. He estimates that the PSP could run a Quake 3 level of technology, but has no immediate plans to develop for it.


What about standalone physics accelerator cards? Carmack doesn't see these taking off in the near future, as they'd be hard to integrate with games. Right now they'd be useful for "fancy smoke and water and waving grass," but nothing integral to gameplay. He fears that the first generations of these cards would allow games to do more, but at the cost of slowing the framerate down. "There will be some neat stuff, I guarantee," he admits, speaking mostly about the technology demos that would ship with the hardware, but "don't expect it to set the world on fire."

Does he see major improvements in the area of networking? Carmack noted that if we could continue to improve broadband, it may someday be possible to get rid of cheating entirely. All PCs playing the game could just be dumb terminals, echoing information from the server, with no chance of messing with the data or using a hacked client. Obviously that would require tremendous bandwidth, and isn't something we'll see in the near future. As an aside, Carmack noted how happy he is to see how far massively multiplayer games have come. The original team at id Software really wanted to play around in that area and never got around to it.

Finally, someone asked about facial expressiveness in games. Carmack said that the technology used to create really amazing faces, such as the ones that the digital movie studios put together, is no mystery to game developers. However, it's tough. Creating expressive faces is slow and expensive, requiring tons of man hours to tweak all the control points so that a face looks just right. He says work like that would lead to the $100 Million dollar game budgets, and right now the gameplay impact just isn't worth it.

All in all, it was a two-hour brain dump from someone at the center of state-of-the-art game development, and a real treat for all attendees. Look forward to the release of the Quake 3 source code as early as next week, and stick around for the release of Quake 4 before the end of the year.

[Turrican3]

He goes on to say that simulating a complex world is a hard problem. "Your game will either be fragile or slow," he says, and it had better do "something amazing" with regards to gameplay in order to make up for that. Carmack says that, in the next generation, he'd rather see processing power go towards making console games run at 60 frames per second rather than doing in-depth physics.



If the PS3 opens up, it'll be more like the old Commodore Amiga, a platform anyone can use for a variety of applications.



Carmack thinks that innovation in the game industry, since it won't often come from large professional development houses, will likely come from the modification or open source communities. There, people can try random ideas to see what works. Example? Counter-Strike, one of the most popular games played online today, which started out as a handful of mod-makers trying to make something fun.