If It's Just a Virtual Actor, Then Why Am I Feeling Real Emotions?

Quotes:


    "The actors in Greek tragedy, whose costumes we know mainly from
     Greek vase-paintings of theatrical scenes, wore elaborate robes,
     often brightly coloured and heavily embroidered, and high boots
     in imitation of their god, Dionysus... The most important feature
     of their costume was the mask, said to have been introduced by
     Thespis.  In light wood, cork or linen, this enabled the three
     actors in a tragedy to play several parts each, and also, in a
     theater staffed entirely by men, to impersonate women.  Each mask
     -- more than thirty types are known to have existed -- indicated
     not only the age, standing and sex of the character, but the
     dominant emotion -- fear, rage, hate, despair.  The actor, denied
     the use of facial expression and limited by his costume to broad,
     sweeping gestures, was forced to rely on the range and
     expressiveness of his voice for most of his effects."

                              -- Phyllis Hartnoll, 1964 ?
                                 A Concise History of the Theater

    "You have to work the suit."

                              -- Michael Keaton, 1993 ?
                                 discussing his role as Batman

    "Even though we may regret the lack of great dramatists ... Italy
     deserves her reputation as the cradle of the modern theater.
     ... There was another ... important aspect of the Italian theater
     ... whose vitality and exuberance may have drained away from other
     areas the energies of the theatrically minded, and that is the
     commedia dell'arte.

    "This form of theater, which appears parallel with the growth of the
     serious academic theater, depended primarily upon the actor and not
     the playwright.  Its dialog, from a simple exchange between two
     comedians to a full-scale play involving a main and sub-plot and
     a number of actors, was entirely improvised, though a skeleton plot
     or scenario was provided to keep the actors within bounds.  There
     were also long stock speeches which, once formulated, written down
     and learned by heart, could be adapted to almost any circumstance.
     But these were used mainly by the more serious characters, the
     young lovers, and also perhaps by the old fathers, the pedant
     lawyers, and the braggart soldiers.  The zanni, or comic
     servants, whose antics made up the greater and most popular part
     of the entertainment, had less to say, since much of their humor
     visual, consisting of variations on standard jokes, the
     lazzi, or slighter comic touches, and the berle,
     longer acts involving a practical joke.  Both the lazzi and
     the berle afforded the actor great scope for improvisation.
     By tradition the zanni could take the basic situation of the
     play in performance as far from the prescribed path as they
     pleased, provided they brought it back to a point where the
     scenario could be picked up again.  That called for a high degree
     of skill and a quick wit.  But all that we know of the commedia
     dell'arte shows that its practitioners were unequalled in their
     profession, combining the attributes of dancer, singer, acrobat,
     low comedian, mime and pantomimist, together with incredible
     agility of mind and body.  A subtle command of gesture was also
     essential, as the comic actors wore masks and so were denied the
     use of their facial expression."

                              -- Phyllis Hartnoll, 1964 ?
                                 A Concise History of the Theater

    "In terms of scholarship, animation is the least theorized area of
     film.  In neglecting animation, film theorists -- when they have
     thought of it at all -- have regarded animation either as the
     'step-child' of cinema or as not belonging to cinema at all,
     belonging rather to the graphic arts.  In the former case film
     theory still sees animation as a form of film, albeit its most
     inferior form, as child to live action's adult form.  In the latter
     case, it would no longer be possible to speak of animation as the
     most neglected form of film nor to attribute any responsibility for
     that neglect to the to the discipline of Film Studies.

    "If one may think of animation as a form of film, its neglect would
     be both extraordinary and predictable.  It would be extraordinary
     insofar as a claim can be made that animation film not only
     preceded the advent of cinema but engendered it; that the
     development of all those nineteenth century technologies -- optical
     toys, studies in the persistence of vision, the projector, the
     celluloid strip, etc. -- but for photography was to result
     in their combination/synthesizing in the animatic apparatus of
     Emile Reynaud's The'a^tre Optique of 1892; that, inverting the
     conventional wisdom, cinema might then be thought of as animation's
     'step-child.'"

                              -- Alan Cholodenko, 1991
                                 Introduction to
                                 The Illusion of Life
                                 Essays on Animation

Fragments

Lets begin with the technology. Some of you may already be familiar with these concepts, but this is Wired; everyone likes to read about technology here.

Here are the pieces of the puzzle. We have television, including the Cathode ray Tube (CRT) which can display arbitrary color images in a sequence that can produce the illusion of motion at 30 frames per second. We have 3-dimensional computer graphics, which can take a mathematical description of a set of objects, made from polygons or curved surfaces and possibly draped with textures (2-dimensional images), and combine these with lighting and viewing information to produce a synthesized view of the objects. We have computers that can create these synthetic views in rapid succession, fast enough to drive a CRT in "real time" (i.e., at 30 frames per second). And we have various input devices to control the placement and orientation of the mathematical objects. Put them together and you have Performance Animation: a real time cartoon controlled by a human operator, drawn by the computer just in time for you to see it, and fast enough for you to interact with it.

The input devices include the Data Glove, the Data Suit (full body), and the ??? (facial Waldo). The first motion trackers used magnetic fields to determine a point's position in space as X, Y and Z coordinates. They were too jiggley. Then mechanical linkages of various kinds were experimented with. Most recently optical reflectors have been used with multiple video cameras pointed at them, and sophisticated software to compute the XYZ values from the video images.

At the SIGGRAPH conference in 1991, held in Las Vegas, the team from SimGraphics demonstrated the first Performance Animation, which at that time they billed as a Performance Cartoon. Called the Silver Surfer, it was a rather voluptuous female body with a featureless spheroid head and shiny reflective metal skin, perched on a surf board. The body animation was controlled by a Data Suit. It was wise of them to avoid animating a human face at that time, since no input device yet existed for driving facial animation.


I grew up in southern California in the 1960s watching Walt Disney every Sunday night as he unveiled the latest magical illusions developed by his "Imagineers." Like many other Americans in my approximate age bracket, I remember when Walt introduced us to "audio-animatronics(TM)," that elegant blend of engineering and art that results in the life-like robots of animals and people at the Disney theme parks. His first demo was "Pedro" the mechanical parrot, a preview of the Tiki Room attraction. Like any normal nine-year-old boy I was fascinated. By moving his hand in a mechanical motion-detector (a "Waldo") Walt could make Pedro dip and nod and move his beak, all powered by hidden tubes of compressed air and controlled by electronic signals. What he was showing us was how the bird could be operated live; this is how a performer, working like a puppeteer, would initially create the bird's motions. The resulting electronic control signals would be recorded on a magnetic computer tape, and the computer could then play back the bird's motions the same way every time. Disney's Imagineers would add recorded sound to complete the illusion, and a room full of these birds would entertain guests at Disneyland. But one thing bothered me: Walt and his Imagineers got to operate the birds and fiddle around with them, while all we -- the paying customers -- got to do was sit and be the audience for a pre-recorded show. This idea of controlling the animatrons interactively (what we might today call real-time telerobotics or the teleoperation of electronic puppets) seemed much more appealing to me.

Because of the close proximity of Disneyland we visited about once a year, and so it was never long before I could see firsthand the new illusions previewed by Walt on television. Sure enough, as we crossed the bamboo bridge into the jungles of Adventureland, there was Pedro the parrot on a perch overhead, entreating the crowd below to come into the Tiki Room and see the show. Of course, he only was able to play back a pre-recorded pitch, but at that time the technology was completely novel, and Pedro served well as a "barker" for the show. (In fact, he attracted such crowds that the entrance to Adventureland routinely jammed up, and eventually poor Pedro was removed to restore smooth pedestrian flow.) Once inside the turnstile, we next had to wait in a garden decorated with tikis and shaded by a huge artificial tree. A movie promoting Dole pineapples was shown to keep us amused. And then, something wonderful happened. One at a time, the tikis in the garden began to talk. Each in turn told a little story about who he or she was and what magical powers they possessed. Their mouths did not actually move, but as each spoke there were some associated special effects that grew more elaborate. The god of earthquakes shook back and forth. The god of rain squirted a mist of water over the crowd. The goddess of fire erupted in flames. Lastly, the huge tree turned out to be a god of fertility, and his blossoms sprouted babies hanging on short vines. As it turned out, I was far more fascinated by this garden of tikis outside than by the singing birds inside the Tiki Room. For these tikis had at first seemed to be inanimate objects, and the families waiting for the show would clump around them heedlessly. Then as the tikis began to speak people reacted with surprise and sometimes mild panic, much to the amusement of those around them. It was a kind of pseudo-interactivity, in that it tended to trigger an entertaining group response. (These days I visit Disneyland every New Year's Day; this year, there was woman talking loudly to her companion who blithely ignored the talking tikis -- until the one over her sprayed the mist. She suddenly jumped up in mid-word and bounded across the courtyard with alarm. It got the predictable laugh. I thought it was nice to see that the gag still works.)

Of course, what Walt was really aiming at was the animation of life-like human forms, and just a few years later he achieved that goal, with the "Great Moments With Mr. Lincoln" attraction housed in the state of Illinois pavilion at the 1964 New York World's Fair. (The attraction was moved to Disneyland after the fair ended.) This in turn was followed by the elaborate Pirates of the Caribbean ride featuring perhaps hundreds of animated pirates and their victims, and ushered in a whole new era of "dark rides" at the Disney parks.

Still, as a boy I felt that the magic of interactivity (now a '90s buzzword) would add so much more to the animatrons. My parents had decorated our back yard in a Polynesian style, including a bamboo bar, and I daydreamed of having my own talking tiki mounted beside the bar, remotely controlled by a human operator (well, me, actually) who would sit on the other side of a one-way mirror. The trick would be to have a cocktail party, and then wait until a guest was sitting alone at the bar, three sheets to the wind. Then the tiki would begin to wink and whisper. "Pssst, hey buddy," it would say, rolling its eyes, and of course the soused guest would rush right off to gather others to come and see this amazing talking tiki. But when the others came, the tiki would be silent and still, staring straight ahead. (This plot is in some ways lifted from a classic Warner Brothers cartoon about a man who finds a singing frog.) I never did realize the idea, but I talked about it now and again with friends.

Meanwhile, the years rolled by and I grew up. Eventually I became a computer graphics expert, working on such projects as modeling the earth as a 3D globe, simulating jets landing on carriers, creating graphics for heads-up cockpit displays, and animating astronauts building a space station in zero gravity while tethered to a space shuttle. Over twenty years my career took me on several hops between the vendors who build and sell computer graphics systems and the customers who buy them and use them. And nearly every year I attended the big computer graphics conference and trade show, SIGGRAPH, the Special Interest Group on Graphics It is the largest "SIG" of the Association for Computer Machinery (ACM), a prominent professional society for the computer industry. The SIGGRAPH conference moves around from year to year, rarely visiting the same city twice.

In 1986 SIGGRAPH came to the Anaheim. I was working on the space station project at Rockwell International in the Los Angeles area at the time, and a bunch of us from the computer graphics lab drove the eighteen miles down to the Anaheim Convention Center (across the street from Disneyland) to attend, including my boss, Steve Tice, who had originally hired me in to the Space Division. The big new thing that year was from a company called Pixar, a spin-off of Lucasfilms (the production company George Lucas founded with the profits from Star Wars). They were working on computer graphics for the movie industry, among other things. This was quite a novelty in '86, since mostly the field was dominated by aerospace companies doing things like flight simulation for pilot training, remote sensing for intelligence gathering, and situation room displays for command and control. This was also shortly before the terms "visualization" and "virtual reality" were coined, so most of us thought that all we were doing was "real-time interactive color 3D graphics." ??? where was I going with this ???

By five years later I was helping to sell Application Visualization System (AVS) software to scientists and engineers at places like Rockwell, and Steve Tice had started his own company, SIMGRAPHICS Engineering Corp., to sell consulting services to about the same market. The 1991 SIGGRAPH conference was held in Las Vegas (a first), and Steve invited me to help him with a project there called "Tomorrow's realities Gallery," which was a showcase for multimedia and virtual reality technology. I showed up three days early and served as technical coordinator for the dozens of exhibits brought in by various organizations. One of the most interesting, to me, was the demo put on by Steve's own company. They called it a "performance cartoon," and it consisted of a human operator with a joystick (???) controlling a computer-generated video image of a silver surfer woman on a surfboard. The image was projected on a huge screen behind the human operator. I stood there and watched this demonstration, and saw the cartoon move in synch with the operator's joystick (???), and I knew I was seeing something revolutionary. I saw two things in this demo: money -- since traditional cartoon animation is time-consuming and therefore expensive, this new technology could save animators a lot; and interactivity -- here for the first time was the potential for humans and cartoons to converse in the same time scale. (Since that first demo SIMGRAPHICS has trademarked the name "VActor," short for Virtual Actor, to describe this technology.)

To follow a parallel thread of this story we have to jump back to two years earlier, when my wife Dixie and I vacationed in Orlando, Florida with another couple, Bob and Trish. Actually, we started out by attending SIGGRAPH '89 in Boston, Massachusetts, and then drove a rented van all the way down the Atlantic coast to Florida, ending up at Walt Disney World. We spent several days visiting the theme parks and sampling the rides, until Bob said, "I don't think I can stand to ride on any more boats to see dolls." Dixie and I had left early for a family reunion, and Bob and Trish stayed on and visited the new Disney night club complex called Pleasure Island. Later Bob told me that I had missed something that was right up my street. You see, I had mentioned my idea for the talking tiki to him at some point, and he told me that a similar gag was being performed every night at a bar on Pleasure Island called the Adventurers Club.

Well, a year later (1990) I was back in Orlando, this time for a mathematics conference, and alone, alas. I made sure that I saw this club. In fact, I ended up going back nearly every night. My friends had been right -- it was right up my street. I was enchanted from the start. You approach the Adventurers Club as you top a small hill, and it emerges from thick foliage. It looks like a typical 1930s vintage private London club, except that there is the wreckage of a small airplane in front, still smoking from the crash. Inside, you overlook a circular main salon from a gallery a story above. The walls on both levels are covered with unusual artifacts brought back by adventuring club members from all corners of the globe, usually with an explanatory card on the wall nearby. Stairs take you down into salon, and as you enter you see the statue of Zeuss, arm extended -- only he has a tackle bag over his shoulder and is casting with a rod and reel wedged into his outstretched hand. To the left is the bar, with its three bartenders all named "Nash," (or so their name tags say), and where patrons sit on elephant foot stools that sometimes move up and down very slowly. To the right doors lead into the library, where a pipe organ has apparently fallen through the ceiling and cratered the small stage. At one o'clock high is a box seat halfway up the wall, where an ancient Boer War Colonel seems to nap -- unless that is just one of those shriveled apple dolls. At eleven o'clock is the door to the necessary facilities, and above this door a huge stone mask of a goddess gazes down on the drinkers below.

Last update: 06-Feb-1998 by ABS