Inkwell: Authors and Artists

    
We are delighted to have Jon Peddie with us to discuss his new book
The History of Visual Magic in Computers. Dr. Jon Peddie is one of the
pioneers of the graphics industry, and formed Jon Peddie Research (JPR)
to provide customer intimate consulting and market forecasting
services. Peddie lectures at numerous conferences on topics pertaining
to graphics technology and the emerging trends in digital media
technology. Recently named one of the most influential analysts, he is
frequently quoted in trade and business publications, and e is also the
author of several books including Graphics User Interfaces and
Graphics Standards, High Resolution Graphics Display Systems,
Multimedia and Graphics Controllers, a contributor to Advances in
Modeling, Animation, and Rendering, a contributing editor of The
Handbook of Visual Display Technology, and a contributor to State of
the Art in Computer Graphics, Visualization and Visual Analytics. His
most recent book is, The History of Visual Magic in Computers.
  

    
Our moderator, David Duberman, has been involved with creative
computing and multimedia as a user, journalist, and writer since the
early '80s. He hasedited various publications including Antic, Video
Toaster User,and Morph's Outpost on the Digital Frontier. He currently
works documenting 3D software for Autodesk.
  

    
This should be great guys, thanks for doing this. Before you both get
started I would like to ask the "dummy" question and get it out of the
way, before you get technical.

Jon, exactly how do 1's and 0's become colors? 
  

    
All digital computers (which include smartphones, PCs, tablets, and
game consoles) use the binary system to calculate, communicate, read
sensors (including camera sensors), and display the results. The value
or weight of each bit (which is set at either a one or a zero)
increases by the power of two, so a series of bits such as 111001 would
represent the decimal value 39 (1+2+4+0+0+32). The screen (of your TV,
phone, PC, or tablet) has three primary colors: red, blue, and green
(RGB). Each one of those colors can be set at an intensity level or a
brightness. So red 11111111 would be very bright, full on red, and
00000000 red would be totally off, or black. 
The three primary colors can be mixed, added together, at various
levels of brightness and create all the colors in the rainbow. The
screens in our devices are LCDs (which stands for liquid-crystal
display) and they are like a valve, that can be wide open or shut and a
zillion levels in between. The number of levels is their bit-size, and
LCDs have 8-bit primary color capabilities. That means there are 2 to
the 8 levels of brightness for each primary color—or 256 levels of
brightness. And here’s where it gets interesting. When you combine
those 256 levels, you can generate 16 million combinations of colors
and brightness. 
So you have 8-bits, per primary, and three primaries. And if you set
each primary at max (all 8-bits on – 11111111) you generate white, and
all off would be black, and if the  red and green were off and blue on,
the screen would be blue, and you could vary the blue to get 256
shades of it. 
The next thing to know about are the pixels. An HD TV has about 2
million (1920 x 1080) of them. Each pixel, so small you can’t easily
see an individual one, has a RGB value—24-bits, 8 for red, 8 for green,
and 8 for blue. So each pixel can be set to 16 million different
colors and intensities. That means there are 2 million times 16 million
possible color and intensity levels on your HDTV screen—that’s a
staggering 34.7 trillion possible combinations. Not all unique, but
some combinations of 16 million. 
So the three dimensions of your screen are the x-pixel, times the
y-pixels, times the intensity or z-depth. 
But wait—it gets even more mind shocking – our displays are actually 4
dimensions—they change with time, very short amounts of time. The TV
screen changes (in the US) 60 times a second. So now you have 35
trillion pixel combinations 60 times a second. 
And all that astonishing activity is what makes the images that you
and I look at on our PCs, tablets, and TVs every day, and just take for
granted. I think it is amazing, simply amazing—it’s magic. 
  

    
Wow, I don't remember which film director said it, but it was
something to the effect of "now that we have CGI we can do anything".
Easy to see why with your explanation above.
  

    
Hi Jon,

Thanks for taking part in this conversation. I've been aware of your
work and valuable contributions to the field for quite some time, and
am quite impressed with your credentials.

I'm going to start by asking a bunch of random questions; feel free to
answer whichever tickle your fancy and ignore the rest.

What is your favorite thing about CGI, and conversely, what's your
least favorite aspect of the field? What do you feel most needs
changing, if anything?

Which are your favorite CGI applications, and why?

What will it take to overcome the uncanny valley, in terms of
software, hardware, and creative ability?

What are the ethical ramifications of being able to simulate reality
absolutely convincingly? What could go wrong? What could go right?
What's your take on stereoscopy? It seems as though the push by TV
manufacturers to popularize stereo TVs has failed miserably. Why did
that happen, and what will it take to make it a major trend? Is it
inevitable?

It's in the interest of those of us who are excited about the creative
potential of 3D to get the tools into more hands. What in your view
might be some more-effective ways to get more of today's youth into
producing with 3D?

For young people who are already interested in working in the CGI
field, what is your advice for getting started?

Are you familiar with the Leap Motion Controller, and if so, how do
you think something like that (highly accurate and responsive motion
tracker) can affect the future of 3d content creation? 

Thanks!
  

    
Hello David
Thank you for the kind words, and the challenging questions. I’ll
answer them in-line and report the question for clarity.

What is your favorite thing about CGI, and conversely, what's your
least favorite aspect of the field? What do you feel most needs
changing, if anything?

JON – my favorite thing about CG is what I like to call Peddie’s first
law: In Computer Graphics too much is not enough.

What that means is in CG we are trying to create a world that is
enchanting and believable. It’s a double-edged sword. One side we want
to create an absolutely perfect image that can’t be distinguished from
a photograph, and we’ve done that using what is called physically
accurate ray-tracing. Today 99% of the images you see of an automobile
are done in the computer using ray-tracing, and they are, literally,
perfect. For biological entities, like a human face, we use multiple
cameras (and lights) and create a light-field map or image. These
calculations even take into consideration the properties of light as it
penetrates your skin, scatters in your subdural layers and re-emerges
in a colored diffuse manner. It’s too bad I can’t insert a picture
here, this is such a visual topic it’s a bit constraining to have to do
it all in words. In any case the goal is get the viewer to suspend
disbelief, to allow themselves be or get immersed in the scene and the
story. 

The flip side of photorealism is animation. Here you don’t want
photorealism (think about the latest Pixar/Disney  or Dreamworks movie
you’ve seen). But, the imagery has to be pleasing. It can’t be too
disproportional or disturbing. In animation you not only have to create
interesting characters, scenery, and story, you also have to create
physical rules—do things bounce, break, reflect, echo, etc.  

So the answer to the question (what is my favorite thing about CG) is
that we are never done, there is no end in sight. A small anecdote—Jim
Blinn, one of the fathers of CG has “Blinn’s Law”— which states that as
technology advances, rendering time remains constant because rather
than using improvements in hardware to save time, artists tend to
employ it to render more complex graphics. So therefore, we’re never
going to be done, we’ll just get better and better images.

Which are your favorite CGI applications, and why?

JON – Games, which are entertaining examples of a simulation.
Simulations like flight simulators, animations and cartoons  (when done
correctly), and ray-traced images of houses and buildings, cars,
boats, and bridges. I also admire what are known as “paint”
programs—programs that allow an artist to create images just as if she
or he were using a pencil or brush or touch-up device.  Why? Because I
think what attracts people to CG is the art, the pretty pictures, and
that leads to a curiosity of how was it done, and then you’re hooked.


What will it take to overcome the uncanny valley, in terms of
software, hardware, and creative ability?

JON – More processing power.  But first we should explain what the
uncanny valley is. As you make an image of a person more realistic you
leave the area of total falsification (such as a cartoon) and approach
photorealism there is a point in between where the character is neither
cute and cartoonish, or believable. And when that point is reached the
character looks creepy, scary, or disgusting. There was a dancing baby
animation and the baby didn’t move realistically and his face was too
old for a baby’s body—it was totally creepy. So the way to avoid it is
to have a perfect model (the 3D wireframe construction under the skin
that gives the object/person their form) and the best lighting (which
includes shadows and secondary shadows) and physical proportions (we
like eyes a certain distance apart, and above the nose). 


What are the ethical ramifications of being able to simulate reality
absolutely convincingly? What could go wrong? What could go right?
What's your take on stereoscopy? It seems as though the push by TV
manufacturers to popularize stereo TVs has failed miserably. Why did
that happen, and what will it take to make it a major trend? Is it
inevitable?

JON – That’s a lot of question, I’ll parse it. The down side of
simulating reality is that you can’t trust a photograph any more. When
you see a photograph of say a two headed chicken, you can’t be sure its
real or a CG image (commonly called “Photoshopped” after Adobe's award
winning Photoshop software). 

Stereo is a challenge and a technology I am really attracted to—if
only it worked. Here again it’s an issue of computer power. We know
what the issues are in Stereo (which I like to abbreviate as “S3D”).
The subject and technology has been studied for over a century. The
problem is, to do it right is very expensive. The cost begin at the
content creation side. Avatar the most successful S3D movie ever made,
actually had very light S3D in it. Cameron used S3D as one of his
tools, not the end result. The problem is too many, maybe most, content
creators can’t get the novelty idea out of their heads, and lack the
creativity to use S3D to enhance, instead of shock. That is the biggest
barrier we face—lack of creativity in the use of S3D. The second part
is the mechanics of manipulating the eyes so as to trick the brain into
seeing – AND FEELING depth. The mathematics of S3D is not a simple
geometric solution. Just as lighting is very complex when trying to
create a photorealistic image, depth is just as complex and has on top
of it all the lighting issues. The clues we get subconsciously when
viewing a scene are so subtle, and so insidious they are almost
impossible to replicate. For example, how is it possible for a person
with just one eye to perceive depth? And yet they do. I overly
enthusiastically predicted that by 2015 all smartphones would be S3D
capable. I may have to eat those words. I think S3D is the next thing
after color with regard to viewing a screen. That’s probably true, but
the path to that world is not clearly understood yet.


It's in the interest of those of us who are excited about the creative
potential of 3D to get the tools into more hands. What in your view
might be some more-effective ways to get more of today's youth into
producing with 3D?

JON – Ask them, or yourself, to explain how and why you see depth.
Look at something(s)  that is no more than three feet away. Understand
its depth. Pretend you’re describing it to a blind person. What makes
the depth visible. The tool is you, not the computer, not the screen,
they are just mechanical aids. You have to understand it before you can
replicate it.


For young people who are already interested in working in the CGI
field, what is your advice for getting started?

JON – Computer graphics is multifaceted. You can enter the field via
engineering and design computers, and/or software. You can build
displays, simulators, and CAVEs. You can be an artist. All of those
skills and talents are used in CG, and more. There’s a home for almost
any creative person in CG, and there are opportunities for all. But you
can’t just do what everyone else has done or is doing. You have to
(try and) bring something new, fresh, exciting to the field. 

Are you familiar with the Leap Motion Controller, and if so, how do
you think something like that (highly accurate and responsive motion
tracker) can affect the future of 3d content creation? 

JON – Imagine you were modeling with clay. Now imagine a program that
will create a trail of where your move your fingers and hands. Think of
software as clay, and you can move your hands to mold the image, the
shape you want, and easily erase or change it with no consequence other
than your lost time. Leap is a tool to connect your imagination with
your hands. (maybe I should try and sell that to them as a marketing
slogan   ).
  

    
Where are we evolving with all this technology - WRT augmented
realities, virtual realites, and what is being termed fractal or hybrid
realities? 
  

    
Short external link to this conversation is
http://tinyurl.com/mvol365

If you're not a member of the WELL, but you'd like to participate,
send comments or questions to inkwell@well.com and we'll post them
here.
  

    
Great answers, Jon; thanks!

Couple more questions:

Can you take us back to the early days of CG and tell us how it all
got started?

And at the other end of the time spectrum, coming up next week is
SIGGRAPH, the annual biggest computer-graphics conference in the world.
I understand you'll be attending: What sorts of things are you most
looking forward to seeing at the show?
  

    
The "realities" - one of the driving forces behind CG is to create
different realities. We do it in the movies and take you away from the
real world for two to three hours. We do it in games and you can spend
hundreds of hours exploring strange and magical worlds.
The environment we do it in has different names which are used to
describe the hardware. In a virtual reality environment you are
immersed and can only see, and hear, and sometimes feel the virtual
world. Your vision is blocked and forced into either a headset (like
the new Oculus) or a room like a CAVE. Virtual reality is interactive,
you can move things, poke at them, get a reaction from them. A good
flight simulator puts you in a cockpit mockup and you literally fly the
airplane—and if you don’t do it right you crash. The magic is if you
have suspended disbelief (which is the definition of “a good
simulator”) when you crash that airplane you will have a major
emotional experience. I’ve had those experiences and they are
terrifying at the time and exhilarating afterwards. Going to the movies
is a for of virtual reality, you are immersed, you suspend disbelief
and get involved emotional with the story, but you do not have any
interactivity, you are a voyeur. However, you may scream and root for
the hero, or weep for the victim.
In between passive the passive VR of the movies, and the totally
immersive VR of a CAVE or encompassing glasses, you have augmented
reality. This is where you wear a set of glasses which allow you to
see, touch, feel, smell, and hear the real world, and there is a
computer-generated image or text superimposed on the glasses—an
overlay. I like to say “AR lets you see things that aren’t there.” For
example, wearing AR glasses you can look at a building and see the
rooms inside it, where are the doors, and the equipment. You can look
at an engine, see the parts inside, or at a person, and see the parts
inside. You can also have real-time translations allowing you to look
at sign in a foreign country and see on your glasses the sign
translated into your native language. 
Computer graphics is all about altering your reality.