Bright New Worlds Unlock Learning

In virtual reality, a kid becomes T-Rex (or a triangle)

By Jaron Lanier

Jaron Lanier, a pioneer in the development of virtual reality and the man who coined the term, lives in New York City. This article is adapted from his remarks at ITTE's 1994 Technology + Learning Conference in Dallas.

We contradict human nature by placing children in schools, then asking them to learn and remember many, many different things, all in exactly the same environment. That's why virtual reality can become such a profound enhancement of education.

One of the main positive potentials of virtual reality has to do with the most natural modes of learning for people, who in fact are mammals that evolved in a certain environment. Like any other animal, we have a certain set of specializations, including mental abilities evolved for some pretty specific purposes. We're very good at remembering environments and finding our way around them. We're good at certain hand manipulations, too, as well as certain types of problem solving.

Now, consider the human ability to remember an environment. If you live in a large city like New York or Dallas, and if you go away for a few years and come back to it, your whole memory of the city quickly will be revived. That's a remarkable feat, if you think about what a complicated object a city is--remembering all the different streets, all the traffic rules, and everything.

Next, consider that virtual reality is a medium that naturally creates artificial places. This means that within the capabilities of virtual reality you have a marvelous way to present information so the brain can remember it, so a child, or any person, can re-enter that information, with understanding, at a later date.

Human memories can be keyed to many things--to a person you're with when you first encounter a piece of information, to a certain smell. But memories are keyed especially to environments. That's why field trips are so important, of course, and why bringing new media into the classroom is so important, too.

Essentially, virtual reality can place kids in an entirely new place every time they do a new thing. When kids use virtual reality, they really do think of it as another place.

If you talk to a kid who's used virtual reality, he doesn't say, "I went to school and used a virtual reality computer." He says, "I went to school, and then I went into virtual reality, and then I went back to school." So it has that strong sense of place, and children remember it keenly.

In addition, virtual reality has the potential to let kids become the thing they're studying. Think of all the ways you can teach a lesson about the dinosaurs: You can have pictures of dinosaurs, movies of dinosaurs, you can visit a natural history museum and see big models of dinosaurs. But only in virtual reality can a kid become a dinosaur.

Children naturally are very interested in themselves, so if they get to become a Tyrannosaurus Rex and try to step on their friends down on the forest floor, they will remember that experience. In fact, we've already done some preliminary work of that kind, having kids becoming triangles to learn about trigonometry, kids becoming strands of dna, and so forth. I think this is a very viable means for teaching, with strong precedents in certain world traditions of education. In China, for instance, children become Chinese characters to learn to write.

Another thing about virtual reality: Kids instinctively want to share what they imagine; they want to be creators. The problem kids face is that they're born with an incredible instinct for imagination that in most cases they don't have a chance to express fully.

A working definition

Virtual reality is a new means of expression. But what exactly is virtual reality? The term is used for just about everything these days. I've decided sometimes it just means whatever somebody needs it to mean for marketing purposes.

But now I'm going to tell you what virtual reality really is. I made up the phrase, so I claim the right to define it. (Obviously, the world doesn't always agree with me.)

As a starting point, consider physical reality--where you and I are right now. You know you're in physical reality thanks to your sense organs: eyes, ears, skin, and so forth. With virtual reality, we want to create an imaginary world that feels real. We do that by creating computerized clothing that provides sensations to your sense organs, which conveniently enough are on the body's surface.

We equip the eyes, the visual system, with virtual reality goggles--generically called head-mounted displays. The goggles, which were first invented in the 1960s by a brilliant guy named Ivan Sutherland, provide each eye with the images it would see if an alternative world existed.

Inside the goggles are two screens--similar to televisions--one in front of each eye. Some clever optics make the image appear to wrap around the whole person, so you feel like you're inside the image rather than looking at a device from the outside. The goggles are also equipped with a head tracker, a sensor that measures where your head is and the direction toward which you're looking. This special sensor talks to the computer that is painting the images in front of your eyes and tells it what point of view you have on the imaginary world.

Because of the head tracker, if you turn your head to the left, an imaginary object moves to the right to compensate. So a fixed object always seems stationary as you move around it--just like a real object. That's why virtual reality feels so real.

But because the images of objects must be shifted around constantly to adjust to your head motion--so they appear to be standing still--they must be created by a computer. You can't create the virtual-reality experience by just shooting a movie and playing it back in virtual reality; you must be able to recreate whatever you see from any angle, as quickly as the human head can move.

The result is a visual medium that will always have a "computery" quality and images that always will be a little less good than, say, television images. But objects in virtual reality nonetheless have an incredible physical presence--much more so than television--though they don't necessarily look as real as a camera image.

The next sense organ engaged in virtual reality is hearing. The goggles used for virtual reality have conventional earphones--like those for a home stereo--except for one difference: Special processors move sounds around in three dimensions. It's the same head-tracking technique that is used for visual objects. Move your head to the left, and the apparent sound source moves to the right to compensate, and thus feels as if it's standing still.

Even with current technology, the three-dimensional quality of virtual-reality sound has led to one exciting offshoot for blind people. Virtual reality is spatial and can be composed of sound as well as images. So people who are blind can move inside of virtual space and find objects by using sound cues--just as they do in the physical world.

Returning to the body, we move to the hands. What better for a hand than a glove? The most common kind of virtual-reality glove senses what your hand does and tells a computer hand in the virtual world to do the same thing. It's a remarkable experience to wear such a glove for the first time and see a computer-generated hand that moves exactly as your hand is moving. A psychological transformation takes place: You suddenly start treating that hand as your hand. That's when your mind starts believing in virtual reality.

A trade secret

You might think that developers of virtual reality must simulate an alternative world perfectly and then provide every bit of stimulus to your sense organs that would exist in such a world. But we don't have to do it perfectly. We're aided by your brain's tremendous ability to create a seamless feeling of continuous reality, which it is always doing.

As a small example of how hard your brain is working to create this feeling, consider that in the middle of each of your eyes is a blind spot, the place where the optic nerve comes in. You are completely blind in that spot, which is rather big and in the center of your field of view. But it's very hard to become aware of it, because your brain fills in the blind spot. The brain actually makes up what you should see there--by guessing, based on what the other eye sees and what the brain knows about the environment.

Fortunately, the developers of virtual reality are able to get a free ride on the brain. We just have to make objects well enough so the brain treats our alternative world as the outside reality that it's enhancing toward. As the brain starts to believe in the virtual reality world, it starts working in its usual conscientious way to make that outside world look good.

This phenomenon is another reason objects in virtual reality feel more real than on TV or film--and why virtual reality worked right at the start, at least somewhat. Other experimental areas of computer science--such as artificial intelligence, which doesn't get the same free ride--have never quite worked.

Virtual baseball

Let's look again at the hand in a virtual-reality glove: You not only can wiggle your fingers, but you can reach out and pick up an imaginary thing as if it really existed, like a virtual baseball, and throw it. You're interacting with the imaginary world in the same way as you would with the physical world.

That's a profound difference from the abstract way you usually communicate with a computer--by, say, pointing at an icon or typing in a command. In virtual reality, you're using your most primal abilities and communicating in the way your body has evolved to communicate. That factor is important when you talk about letting children use virtual reality.

Fancier gloves have also been developed that actually push back at you. If you pick up an object, you cannot close your fingers through it. You might wonder whether these "forced-feedback" gloves might be essential for virtual reality to be convincing. Otherwise, how could you possibly stand being able to put your hand through a baseball or, for that matter, through somebody else's head?

But once again, a psychological phenomenon bails out the developer of virtual reality. This time, it's synesthesia, the confusion between senses. Suppose you put someone in virtual reality for the first time without saying anything in advance about what to expect. If you tell the person what to expect, your volunteer will be too conscious of what's happening. Now, you place your unsuspecting volunteer in front of a virtual desk. You ask that person to slam his hand down on the desk, and as he does, he sees a shadow converging with his hand, and when the two hit, he hears a nice satisfying slap.

And guess what: His arm won't go through the desk, because his brain believes a desk must be there, and his brain makes him believe his hand has hit something solid. Once again, virtual reality is nothing but a tool for enabling brains to do what they do best.

And sometimes these brains are doing things you'd never dreamed of. During the early development of virtual reality, it was no surprise that there were bugs in the system. Bugs are inevitable with software. The surprising thing was that the bugs often were much more entertaining than with a conventional computer. For instance, I once went into virtual reality, and my hand seemed 2 miles long. Now, if your hand is that large and you bend your finger, you end up inside your finger. Having such a strange body is fun in itself. But what is interesting is that if I gave people strange bodies in virtual reality, they could still control them.

I would have thought that changing someone's body image even a little bit would disorient him or her. But people learn their new bodies quickly. We used to have people turning into lobsters and gazelles and all kinds of crazy things. This raises many of the best possibilities for the use of virtual reality in education.

The complete package

In addition to computerized clothing, a complete virtual reality system needs three more elements: computer systems, networks, and software. I'll mention some aspects of each of these.

First, the computer systems. The hard fact is that virtual reality is computationally intensive, and to do it well is enormously expensive right now. The virtual-reality applications I work with usually involve something like $500,000 worth of equipment per user, and sometimes there are multiple users interacting.

That high cost will postpone what I regard as one of the most exciting things going on culturally on the planet right now--having children involved in virtual reality. Although there are new efforts to make virtual reality more accessible economically and to allow hobbyists to create something like virtual reality in their garages using PCs, none of those efforts is really quite satisfying. They're fun to work with--and children often use them to great effect--but they're missing the sensual element.

Fortunately, one great feature of computers is that the cost drops so quickly. In five or 10 years, what is very expensive will be merely expensive and later will become downright cheap. The timetable depends on the usual conglomeration of forces and marketing strategies; it's hard to predict where, but we know it will happen.

Next is computer networking--which, when combined with virtual reality, creates the possibility of having shared virtual worlds. In other words, you can have more than one person at a time in a single virtual world.

Being in a virtual world with someone else is rather like being in the physical world together. Each of you has a body and can look at the other from your own independent point of view, but in a shared space. That is where virtual reality becomes utter magic--and most fascinating to children.

The final element of a virtual-reality system is software. Virtual reality requires many types of software--operating system software, networking software, and so forth--but the software I'm most concerned with consists of authoring tools. These wonderful virtual worlds have to come from somewhere, and it's pretty hard to make a good virtual world.

I dream of the time when virtual-reality authoring tools will become so good that they'll become second nature to children, so children can make up these worlds and essentially share their dreams, perhaps create their own lessons, because I think that's what they really want to do.

Already I've seen children under 10 years old using technology much more actively and creatively than ever before. Little children are producing impressive multimedia works on a Macintosh, for example, that far exceed their parents' understanding of the machine. But to harness children's incredible creative energy, the tools have to be available for them to go forward.

I believe--and dearly hope--that the use of this technology might open up society to children's creativity as never before and encourage every child to express his or her inner world fully with others. That could lead to a remarkably creative society. And the more we have to do to occupy ourselves creatively, the less time we have to be violent and nasty to each other. In other words, we might discover a program to make the world a better place.