Friday, March 31, 2006

Win some, lose some

Last night I decided to bring up a minor feature I had been contemplating for a long time: the addition of audio feedback to Swat. This is nothing more than little beeps and boops that sound in response to your button clicks. I've long felt that these sounds are crucial to a good user interface, so I set to work. But gad, what a battle I faced! First there were problems getting a microphone to work with my old Mac, then recording the sounds, then storing them in a file format that my sound editor would recognize, then editing them, then getting them into a file format that Java would recognize (and, along the way, a great deal of time wasted trying to find beeps and boops on the Internet in a usable format.) After getting all my sound effects together, I added code to Swat that would play the sounds, and I was set.

Or so I thought. It turns out that the sound player I used was synchronous: it plays the sound and nothing else happens while it is playing the sound. This flabbergasted me. The old Mac 9 operating system supported asynchronous sounds (you start the sound playing, then proceed to other computations so that the sound plays while the interface is drawing the screen or waiting for input.) But Java, a threaded language that supports multiple threads, doesn't seem to be aware of asynchronous sounds. Oh, I'm sure that there are asynchronous sounds somewhere in the libraries. I'm just using the wrong sound library. "Oh!" some expert will say, "you're using the javax.sound.blech library. If you want to use asynchronous sounds, you have to use the javazzz.adgwed.yorch library. Didn't you know that?" Yeah, right.

So I reverted my code to what I had running Thursday and left it at that. To hell with sound, for now. We'll put it in when we can find somebody willing to wade knee-deep through the sewers of the Java libraries.

Saturday, March 25, 2006

Trip Report

On Monday, I spoke at the Serious Games Summit, part of the Game Developers Conference. I was on a panel discussing the incorporation of human characters into serious games. My thesis (presented in 8 minutes and 22 seconds), was that you can't have human interaction without language, and natural language required the inclusion of reality, and reality is too big to fit into a computer. Fortunately, it was possible to build toy languages to match the toy universes we create in software, and I had done just this with Deikto. I invited anybody with further interest to take a brochure; about a hundred people did just that.

Dave and I walked the Expo floor where we waded knee-deep through sewers of techie-schlock. We did, however, examine three technologies for face generation, which I shall refer to by the nationality of the vendor:

1. British: GeneHead, at http://www.genemation.com. Primarily for animation purposes. They have primitive controls for setting emotional expressions. The results look good.

2. Japanese: Motion Portrait. Their website has nothing to show, the product is not yet released. This was the most impressive of the three. They had a camera set up facing a blank background. They sit an attendee down in front of the camera, take a picture, feed it to the computer. The computer crunches for about a minute, and then they begin an animation in which the subject's face is animated with a variety of facial expressions. They do feature recognition automatically! Very impressive.

3. Russian. http://www.lifemi.com. I was not impressive by this group. Their facial models are rather doll-like in appearance.

Now, here is how we evaluate these options. Our objective is to deliver any specified face with any specified emotional expression to our player. If we have a hundred faces and a hundred emotional expressions, that's ten thousand different images -- too many to build by hand. So there are three basic ways to do this:

A. Software that allows us to create a face, and then manually set up an emotional expression on that face. This would require ten thousand separate editing actions on Paulo's part -- rather tedious. Right now, that's the best we've got.

B. Software that allows us to create a face, and separately create an emotional expression, and then apply that emotional expression to that face, and save the result. This would still require ten thousand actions on Paulo's part, but it's a MUCH simpler task, something we can build up to steadily. This is probably the best we can hope for at present.

C. Software that renders the face directly. We simply send it a message saying, "Draw Face #27 with Emotional Expression #83" and it renders the face automatically. This would be fabulous, but at present is not available. However, it is conceivable that one of these products might evolve into something capable of doing this.

Another important consideration is platform independence. All of these products are built for Windows machines and are written in C++. That means we can only use them inside our own studio. It's not a killer problem, but a definite inconvenience.

Thursday was an action-packed day. I drove to Santa Cruz and spoke to a class of 150 undergraduates in a course on games. It's always fun to mix it up with students.

In the afternoon I drove back to San Jose for the rant session on which I was speaking. While walking through the floor, I ran into a professor at a Portuguese university who wanted to invite me to speak at a conference he's setting up for September 26th-30th.

The rant session was illuminating. There were six of us. The first rant was wonderful -- a lady making fun of the sexist imagery that is so common in games. The second rant was disturbing. The ranter, a business guy, ranted about game designers who bring unrealistic financial expectations to the table, failing to understand that a game must sell enough copies to earn its costs. It's this kind of thinking that keeps the games industry mired in its rut. Any entertainment medium that insists that each and every product must be financially successful will surely stagnate. The wisdom that this big shot fails to understand is that a company must be strategically conservative and tactically liberal. You take chances on a wide range of products, and the one hit pays back all the misses.

Another speaker ranted against the people who rant about the lack of creativity in the industry. I am caricaturing his position when I say that he argued that innovation wasn't as important as execution -- but that was the basic direction of his comments. He said something very close to "it's more important to learn how to copy and adapt other people's ideas well than to create new ideas." Are you beginning to see why the games industry is so stagnant?

My rant, that there's nothing to rant about because the games industry is already brain-dead, went across OK. I hit pretty hard, saying that the panel was like a bunch of doctors gathered around the bedside of a brain-dead patient, arguing how to restore her to life and vivacity. I said that I preferred to walk down the corridor to the maternity ward, where there was a newborn infant called "interactive storytelling" that had a much brighter future. I ruffled some feathers and triggered some hostile questions. One guy in particular was pretty exercised, insisting loudly that there's really no difference between games and storytelling. There were also some people who fervently thanked me afterwards for having the courage to say what nobody else would say.

Friday, March 17, 2006

To market, to market...

I'm off to the Game Developers Conference. I have a busy schedule during my four days in the Bay Area. I shall be speaking on two panels: one on Monday at 11:30 and the other on Thursday at 5:30. Irene is printing up a bunch of brochures, which I shall hand out at the conference. I'll also be speaking on Thursday afternoon at UC Santa Cruz. I have a meeting with Dr. Peter Jenniskens at the SETI Institute about the videotapes we took of the Leonid meteor storm of 1999. And Dave Walker and I have to resolve some very complicated technical issues about Swat and Storytron.

Meanwhile, the software is falling into shape. There are still a lot of gaps, but we have a basically functional Swat right now and Storytron does its core Deikto function correctly. I have not yet brought up the Engine (the third major component of the entire system), but the core routines are already separately operational and the task now is one of integration.

I had wanted to release an alpha Swat by this time, but it looks as if that won't happen for another couple of weeks. I want to get the dictionary in better shape before I release Swat.

To market, to market...

I'm off to the Game Developers Conference. I have a busy schedule during my four days in the Bay Area. I shall be speaking on two panels: one on Monday at 11:30 and the other on Thursday at 5:30. Irene is printing up a bunch of brochures, which I shall hand out at the conference. I'll also be speaking on Thursday afternoon at UC Santa Cruz. I have a meeting with Dr. Peter Jenniskens at the SETI Institute about the videotapes we took of the Leonid meteor storm of 1999. And Dave Walker and I have to resolve some very complicated technical issues about Swat and Storytron.

Meanwhile, the software is falling into shape. There are still a lot of gaps, but we have a basically functional Swat right now and Storytron does its core Deikto function correctly. I have not yet brought up the Engine (the third major component of the entire system), but the core routines are already separately operational and the task now is one of integration.

I had wanted to release an alpha Swat by this time, but it looks as if that won't happen for another couple of weeks. I want to get the dictionary in better shape before I release Swat.

Friday, March 10, 2006

On To Storytron

Yesterday I wrapped up a workable version of Swat and shipped it out to the other members of the group; now it's their turn to poke around with it and give me feedback. This is not a major milestone; the really big milestone will come when I release Swat to the general public for alpha testing. Figure two to four weeks for that to happen.

In the meantime, I have begun work on Storytron, the program that acts as the interface between the player and the storyworld. Storytron's most significant feature is the presentation of the Deikto language. I have managed to get it running, but it doesn't do anything yet. Isn't that impressive?

Wednesday, March 01, 2006

Screen Layout

We have now an image showing how a storytron screen layout could possibly look like. You can find it inside the storytron page in the overview section, or just by clicking here.

Additions to the Overview

I've finished writing two new articles for the overview. The first deals with Verb-based dramatic interaction, and the other gives an example of the storybuilding process. The latter was plagiarized from Chris, but I'm not afraid to get sued, mainly because no judge will fail to declare me not guilty by reason of mental defect.

I had two reasons for writing these articles - first of all, these are topics that weren't very clear to me when I was a rookie, and I think that made my learning curve steeper. Second, I presume many of you are intrigued by Storytronics, but right now we don't have much material to offer you. These articles might help satiate your curiosity.

I hope the articles will be up within a day or two. You'll find them in the Storyworld portion of the overview. Remember, they're here for you. Please let me know if something is not helpful, informative, interesting, clear or what have you. Also feel free to tell me if there's anything you'd like to know about that isn't covered.