A Vintage Virtual Reality Interview


This interview captures some of the wild bliss I exuded in my twenties as I told the world about Virtual Reality for the first time. It was first published in 1988 or so in the Whole Earth Review, but was conducted a few years earlier. It was reprinted many times in many languages.

Adam Heilbrun: "The word 'virtual' is computer jargon. Could you clarify it for those unfamiliar with the concept."

Jaron Lanier: Maybe we should go over what Virtual Reality is. We are speaking about a technology that uses computerized clothing to synthesize shared reality. It recreates our relationship with the physical world in a new plane, no more, no less. It doesn't affect the subjective world; it doesn't have anything to do directly with what's going on inside your brain. It only has to do with what your sense organs perceive. The physical world, the thing on the other side of your sense organs, is received through these five holes, the eyes, and the ears, and the nose, and the mouth, and the skin. They're not holes, actually, and there are many more senses than five but that's the old model, so we'll just stick with it for now.

Before you enter the Virtual Reality you'll see a pile of clothing that you have to put on in order to perceive a different world than the physical world. The clothing consists of mostly a pair of glasses and a pair of gloves. Exactly what clothing there will be it's too early to say because there are a lot of different variations that are possible and it's really too early to predict which will be the most popular ones. A minimal kind of Virtual Reality outfit would have a pair of glasses and a glove that you put on.

The glasses allow you to perceive the visual world of Virtual Reality.
Instead of having transparent lenses, they have visual displays that are rather like small three-dimensional televisions. They're much more sophisticated than small televisions, of course, because they have to present a three-dimensional world to you that's convincing, and there's some technology involved in accomplishing that, but that's a good metaphor. When you put them on you suddenly see a world that surrounds you - you see the virtual world. It's fully three-dimensional and it surrounds you, and as you move your head to look around, the images that you see inside the eye glasses are shifted in such a way that an illusion is created that while you're moving around the virtual world is standing still.

The images come from a very powerful special computer which I like to call the Home Reality Engine. It will be sitting there in your room and will plug into the phone outlet. I'll say some more words about the Home Reality Engine in a second, but let's stay with the glasses for now.

There's another thing that the glasses do. At the end of the stems they have little headphone speakers very much like a Walkman, which allow you to hear the sounds of the virtual world. There's nothing too unusual there; they're just exactly like your everyday Walkman speakers. The sounds you hear on them are a little bit unusual in that they're processed to have three-dimensional quality; they come from certain directions.

The glasses do one other thing too; they have sensors in them that can sense your facial expression. This is very important because you are a part of the Virtual Reality and the clothing that you wear has to sense as much as it can about your body. It uses that information to control the virtual version of your body, which both you and other people perceive as being you in the Virtual Reality.

So, for instance, you might choose to become a cat in Virtual Reality, or really anything. If you're a cat you might very well be wired, so to speak, so that when you smile in the real world the cat that you are in Virtual Reality smiles. As your eyes dart around looking, the eyes of the cat dart around as well. And so the eye glasses also have a function in sensing your face.

The headset, the eyeglasses - they're sometimes called eyephones -you have to remember that we're witnessing the birth of a culture here, so a lot of terms aren't really settled down into being a particular way just yet. I think we have to give the community of people working in Virtual Reality a chance to jostle about these different possibilities before we decide definitely what things are called and exactly what they'll do. But this is a very plausible setup here that I'm describing.

You wear gloves on your hands. These allow you to reach out and feel things that aren't really there. The inside of the surface of the glove has tactile stimulators so that when the Home Reality Engine can tell that your hand is touching a virtual object (even though there's no object there) you'll actually feel the object.

The second function of the gloves is that they actually allow you to interact with objects. You can pick up an object and do things with it, just like you would with a real object. You can pick up a virtual baseball and throw it. So it allows you to do things to the world.

It does more than that; the glove also measures how your hand is moving. This is very important so that in the virtual world you can see a version of your hand to see your movements. It's important that you wear clothing that not only transfers sensations to you but measures what your body is doing. The computer that's running the Virtual Reality will use your body's movements to control whatever body you choose to have in Virtual Reality, which might be human or might be something quite different. You might very well be a mountain range or a galaxy or a pebble on the floor. A piano... I've considered being a piano. I'm interested in being musical instruments quite a lot. Also, you can have musical instruments that play reality in all kinds of ways aside from making sound in Virtual Reality. That's another way of describing arbitrary physics. With a saxophone you'll be able to play cities and dancing lights, and you'll be able to play the herding of buffalo's plains made of crystal, and you'll be able to play your own body and change yourself as you play the saxophone. You could become a comet in the sky one moment and then gradually unfold into a spider that's bigger than the planet that looks down at all your friends from high above.

Then, of course, there's the Home Reality Engine. The Home Reality Engine is a computer that by1989 standards is a very powerful computer but in the future will just be a regular computer. It has a lot of jobs to do. It has to be repainting the graphics that your eyes see, and calculating the sounds that your ears hear, and calculating the textures that your skin feels, all the time quickly enough so that the world is realistic. That's a very big task. It has to communicate with other Home Reality Engines in other people's houses so that you can share realities with other people, and that's a very big task. It's quite a special computer and it makes a Macintosh look like a little speck.

AH: "When you first put on your clothing and become aware of the Home Reality Engine, are you presented with something analogous to the Macintosh desk-top, that is to say a work space with tools in it?"

JL: Once again this is a cultural question. The point is that in Virtual Reality there's no need for a single metaphor, whereas there is a need for a single design metaphor in a computer. We are used to switching contexts in real life. It's normal to be in your living room in which you behave one way and in which you do certain things, and then go to work, say, and you do something totally different, you go to the beach and you're in an utterly different state of mind, and you go into a temple and you're in a still different state of mind. All those places are really different streams of life that we associate with an overall environment.

There's simply no need for one unified paradigm for experiencing the physical world, and there's no need for one in Virtual Reality either. Virtual Reality is not like the next way computers will be; it's much much broader than the idea of a computer. A computer is a specific tool. Virtual Reality is an alternate reality and you shouldn't carry over into Virtual Reality the limitations that are necessary for computers to make sense. It's an absurd limitation. Because what we're synthesizing here is reality itself and not just a particular isolated machine, there are lot more possibilities than with the Macintosh.

What will probably happen is that the Home Reality Engine will have a capability to scan the room that it's in and so will your glasses. The very first thing that you'll see when you put on Virtual Reality clothing for the first time will simply be an alternate version of the physical room that you started out in. So, for instance, if you are in your living room and you put on Virtual Reality clothing - let's suppose that your living room has a couch, and it has a set of shelves, and it has a window, it has two doorways, it has a chair; it has all these things and it has certain dimensions (walls and ceiling). When you put on your glasses the first thing you'll see is an alternate version of your living room with the same dimensions. Wherever there is a thing in the living room there will be something in the Virtual world. Where there's a chair in the living room there will be a something in the Virtual world. It probably won't be a chair- it very well might be a chair, though. The Home Reality Engine will just do a substitution. The reason for this is that it will prevent you from bumping into anything.

It will have some early tools that you can use. For instance, it will have the equivalent of directories in computers, but they won't look like directories, of course. There may be giant trellises, trellises a million miles across, that are perfectly lightweight, that you can pull yourself through, that carry with them all sorts of different objects, a veritable museum of different objects that you might explore. You might have one of those that shows up in your room. You might very well have a whole bunch of little buckets, and whenever you put one of those buckets on your head you find yourself inside another world, another universe. There will be things like that.

"AH: Will these buckets be things that you've created yourself or will they come as a software package?"

JL: There will be some starter ones. They would have been created over time communally by the community of users. They would have been started by some of us. You'll certainly make your own after a while.

But the thing that you have to remember is that Virtual Reality is a much broader idea than, say, the Macintosh. Its purpose will be general communication with other people, not so much getting sorts of work done. The Macintosh was conceived as a way of automating desk-type of work, so they used the desk-top metaphor. It was quite appropriate and obviously it's been very successful; it was a cultural match.

Virtual Reality is conceived of as an expansion of reality, the provision of alternate realities for people in mass in which to share experiences, and so the types of metaphors that come up are things like cars, travel, different countries, different cultures. For instance, you might very well have a virtual car that you ride around even though physically you're in one place. It would go through different territories in Virtual Reality so that you could get around them - or transporter booths, perhaps. So you could have geographical metaphors. There might very well evolve a new geography, let's say - a fictitious planet with new continents that you can dive into to find new realities.

In the early Virtual Realities you'll only be able to see the Virtual Reality when you're in it. Later, there will be more sophisticated ones where you can blend Virtual objects and physical objects so that you can live in a mixed reality for a while and be able to see your physical environment as if you were wearing sunglasses but also have nonphysical objects mixed in it. That will be a later stage. We're already developing technology to do that but it's an order of magnitude more complex to pull off.

In Virtual Reality any tool is possible and there will be some wonderful tools. In Virtual Reality your memory can be externalized. Because your experience is computer-generated, you can simply save it, and so you can play back your old experience anytime from your own perspective. Given that, you can organize your experience and use your experience, use your externalized memory in itself, as the basis for what you would call The Finder in the Macintosh. That will be quite a different thing. You can keep whole universes in your pocket or behind your ear and pull them out and look through them any time.

AH: "Mechanically, how do you go about playing back your memory?"

JL: What would you actually do? See, that's a very personal decision. You have to understand that in the Virtual Reality, each person might have very personal idiosyncratic tools that might even be invisible to others, but it's the shared reality that you affect by using your tools that counts. That's what's the most important thing. And it's nice to see each other's tools too; it's kind of intimate but it will be fun.

My way of having my memories might be ... I think I'll keep them behind my ear. I imagine reaching behind my ear and pulling them out in front of my eyes and then I'll suddenly find myself wearing bifocals where I wasn't before. In the lower half of the bifocals I see the virtual world as it is shared and in the upper half I'm looking into my memories of the past. These aren't real bifocals, of course. From now on whenever I refer to anything, I'm talking about virtual things, not physical things.

There will be a machine that looks like that machine at the optometrist's where you can flick little lenses into place; there will be this machine that's floating out in front of me, and each of the lenses I can flick into place filters out different aspects of my history. One will say, "Well, filter out everything that wasn't in this room." Another will say, "Filter out anything that wasn't with this other person." And another will say, "Filter out anything that didn't involve music," and so forth. When I flick all these filters into place, I have a narrower and narrower view of my history, so I'm looking at less and less of it. Another filter I might flick into place will order it in different ways. I might want to order it chronologically as I experienced it, or I might want to have it play back sorted according to its geographical distance in the virtual geographic space.

Then I have a little device, a knob that I can turn to go forwards and backwards through my memories and flick filters at the same time. The filters might also change the ways the filters look. For instance, one object might make only certain kinds of things bigger and brighter. Like if I only want to find musical instruments from the past, I might go forwards and backwards through my history and the instruments will be particularly easy to see because they'll be bigger and brighter but they'll still be in their context, so I can still rely on my internal memories, which remember things in context.

Of course, I'm somewhat simplifying things because I'm only using the visual metaphor now. I'll have the same thing for tactile and sonic memories.

Then, if I see something I want to bring into the present reality, or if I see an old memory that I want to relive in a different way with the people that I'm currently with, we can either pull something out of it (simply reach into that memory and pull it out into the current circumstance) or we can all climb into the memory - either way. It doesn't matter.

AH: How did all of these memories get from your mind into the Virtual Reality?"

JL: They never were in my mind. You see, they're memories of external reality. Let's say you're experiencing a few moments in Virtual Reality and perhaps you're sitting on the rings of Saturn -whatever turns you on. Now what's happened is that in order for you to perceive everything you perceived, in order to perceive that you were looking out into the vastness of space and that if you looked behind you there was this huge planet Saturn and so forth, in order to perceive that, the Home Reality Engine was generating those sensations. It was generating the images you saw in your glasses. It was generating the sounds you heard in your earphones. It was generating the textures you felt inside the glove. It can simply store these like any other computer information. There they are. You can play back exactly what you experienced. Experience becomes something you can store in a computer file.

I know that might sound rather cold. I'm the first one to criticize this horrible substitution of information for human experience. I think information in itself is a dreadful concept. It robs us of the richness of life. It robs us of the act of the joy of each moment and the mystery of the next. But it is simply true that the external experience, not the internal experience, but the external experience of Virtual Reality, is a computer file. And it's that simple.

Now the reason that the whole thing works is that your brain spends a great deal of its efforts on making you believe that you're in a consistent reality in the first place. What you are able to perceive of the physical world is actually very fragmentary and a lot of what your nervous system accomplishes is covering up the gaps in your perception. In Virtual Reality this natural tendency of the brain works in our favor and as soon as there's a threshold, the brain will tend to think of either the physical world or the virtual world as being the reality you're inside of. But as soon as the brain thinks the virtual world is the reality you are inside of, all of a sudden it's as if all the technology works better. All variety of perceptual illusions come into play to cover up the flaws in the technology. All of a sudden the world becomes much more vivid than it should be. You perceive things that aren't there. You perceive the resistance of objects that actually have no mass as you try to push on them, and things of that kind.

Absolute Physics

AH: "For interface shouldn't you be able to talk within your environment?
Current voice recognition technology isn't very impressive."

JL: Well you should be able to and it would be a nice thing but it's not central at all. In fact, it's pretty superfluous, at least the way I think about Virtual Reality. I'm pretty sure that it will turn out to be a not very important aspect of it. It would take a while to explain why, but I suppose I should!

There are a few special things about Virtual Reality to keep in mind, the things that make it important. One is that it's a reality in which anything can be possible, provided it's part of the external world. It's a world without limitation, a world as unlimited as dreams. It's also a world that's shared, like the physical world. It's as shared and as objectively real as the physical world is, no more, no less. Exactly how shared or real that is, is open to question, but whatever the physical world has Virtual Reality has as well. The thing that's remarkably beautiful to me about Virtual Reality is that you can make up reality in Virtual Reality and share it with other people. It's like having a collaborative lucid dream. It's like having shared hallucinations, except that you can compose them like works of art; you can compose the external world in any way at all as an act of communication.

pull quote:
Language comes as a stream of little discrete symbols
... the world is made of continuity and gesture.

The question is: well, given that you have a world where you can change it, how do you change it? Do you just talk to it and does it become the way you say it should be? Or do you do something else? Now, there are real limits to how you can change the world by talking. For instance, imagine that you were trying to teach a robot to fix a car engine and you tell the robot, "Okay now, connect this piece to that piece, turn this bolt and so forth." Well, you can do that to a degree but you can't really do that with a person. You have to show them. You can't run the world with language. Language is very limited. Language is a very very narrow stream through the plain of reality. It leaves out a great deal. It's not so much it leaves things out as that language comes as a stream of little discrete symbols and the world is made of continuity and gesture. Language can suggest things about the world, but no painting could ever be fully described by words, nor can reality.

The way that you can probe the reality is with a special kind of physics that can only exist in Virtual Reality. It's what I call Absolute Physics. For some time now I've been working on software that will be able to make Absolute Physics work in Virtual Reality.

Coming back to the physical world for a second, there are only a very few things in the physical world that you can change fast enough to use as forms of communication. Mostly it's your tongue, and to a lesser degree the rest of your body. Your body is basically the extent of the physical world that you can communicate with in real time, but you can communicate with it as fast as you think. That's the way the body is. But then, beyond that you can change the physical world but you need tools. You can suddenly change a room from being dark to light by turning the switch because the switch is there. Technology in the physical world mostly functions to extend the human body one way or another so that it can be used as a medium for human action. The problem is that the kinds of tools that you can have are very limited. You can't have a light switch that turns day to night or a knob that makes the room suddenly grow or shrink in size. You can have tools that can color your face, but you can't have tools that can change you from one species to another. Basically, all that absolute physics is, is a physics that has any kind of causality at all, so you can have all these tools. Once you have all these tools, you can start, using whatever body you choose to have in Virtual Reality, to use the tools to change the world very quickly in all kinds of ways. Then, you have this idea of being able to improvise reality. That's the thing that excites me the most about it.

"AH: What does the interface look like? If I wanted to turn this cup green, what would I do to make it green?"

JL: Okay, here's the deal. There's no one way. There would be a million ways to change the cup green. You could make up new ones and you could change that one over there. See, the tools that you use to change reality are somewhat private. It's the result of the change in reality that's the more social thing. People will be somewhat idiosyncratic about that and that will be an aspect of someone's personality.

You have to understand what these tools are. In Virtual Reality .... well, things are vastly different. You would be able to have all kinds of tools with you all the time. In fact, memory in Virtual Reality is external. You have a movie of your life that's there all the time that you can pull out. It has in it all the tools you've ever used. You can find them quickly. You'll have all kinds of tools.

Now the way you turn the cup green would probably be with some kind of little coloring device. The kind of coloring device that I'm going to have is a little wand thing, a little prism that I pick up. I turn it and it reflects the rainbow of my eyes. Whenever the color looks right I'll squeeze it and whatever it's pointed at will turn into that color. That will be my personal one; you might want something completely different.

Broadcast Media vs. Social Media

AH: ""We are witnessing a breakup of concensus reality in the external world right now and the political repercussions seem rather frightening as large segments of society have no common ground, no shared assumptions about reality. Will Virtual Reality not further undermine consensus reality."

JL: It's a complicated question with many many angles to it. Let me just cover a few. One is that it's important to understand that notions of consensus reality are of a different order than what Virtual Reality is. Consensus reality involves a series of subjective realities and Virtual Reality only addresses objective reality, that is, the shared reality that is external to the senses. But there is interaction between the two on many levels.

Another angle is that idealistically, I might hope that Virtual Reality will provide an experience of comfort with multiple realities for a lot of people in western civilization, an experience which is otherwise rejected. Most societies on earth have some method by which people experience life through radically different realities at different times, through ritual, through different things. Western civilization has tended to reject them but, because it's a gadget, I do not think that Virtual Reality will be rejected . It's the ultimate gadget. It's the culmination of gadgetry in many ways. I think that it will bring back into western experience something that has been lost. Why that is so is a big topic.

It will bring back a sense of the shared mystical altered sense of reality that is so important in basically every other civilization and culture prior to big patriarchal power. I hope that that might lead to some sense of tolerance and understanding. But there's more to it than that. I often worry about whether it's a good technology or a bad technology. I have a little benchmark I use for that. I believe that if a technology increases human power or even human intelligence and that's it's sole effect, then it's simply an evil technology at birth. We're already both powerful enough and smart enough to accomplish a great deal. All of our problems are self-brought at this point. If the technology, on the other hand, has a tendency to increase human communication, human sharing, then I think it's a good one overall, even though there might be many ways it could be used badly. My chronic examples of these are that the television is bad but that the telephone is good. I could go on about that forever.

I do hope that Virtual Reality will provide more meeting between people. It has a tendency to bring up empathy and reduce violence, although there's certainly no panacea ultimately. People simply have to grow up and that could take a long time, too long .

There are some other levels of interaction, too. You see, Virtual Reality starts out as a medium just like television or computers or written language, but once it gets to be used to a certain degree, it ceases to be a medium and simply becomes another reality that we can inhabit. When it crosses over that boundary it becomes another reality. I think of it as acting like a sponge where it absorbs human activity from the physical reality plane into the Virtual reality planes. To the degree that that happens there is a very beneficial asymmetry that comes into play. When Virtual Reality sponges up good energy from the physical plane, then what you get in Virtual Reality is beautiful art, beautiful dance, beautiful creativity, beautiful dreams to share, beautiful adventures. When Virtual Reality soaks up bad energy from the physical plane, what we get is some decrease, however small, in violence and hurt on the physical plane in exchange for events on the virtual plane which, while they might be uglier, are of no consequence whatsoever because they're virtual.

Barbara Stack: "Unless they're syndicated, in which case they are educational propaganda. And don't they have consequence in that they further brutalize the participants?"

JL: Well, physical reality is tragic in that it's mandatory. Virtual Reality is multiple channel. People can choose and switch which Virtual Reality plane they're on. They can also simply take off their clothing if they want to get out of it. It's easy to take the physical world for granted and forget that you're inside it. (Well, that's a hard comment to explain.) It's harder to forget that you're inside of Virtual Reality and therefore it's harder to suffer it. You can simply take the clothing off.

pull quote:
physical reality is tragic in that it's mandatory

AH: "One of the images that haunts me is growing up watching Tom and Jerry cartoons, where in an alternative reality you can see somebody squashed by a steamroller and then pop up and be whole again. I think having absorbed so much of that king of imagery has numbed us ;we have become a generation that is unaware of the pain of others."

JL: Virtual Reality is a very different situation than movies or television. I'm going to say something roundabout but it comes back to exactly the point you're bringing up. Movies and television are, first of all, broadcast media, so one facility has to generate the material that you see. Furthermore, it's very expensive to generate this material so very few people are in a position to do it. Therefore, the material becomes supernaturally remote and universal. It has a numbing effect on people and it reduces empathy. Television ultimately reduces empathy because people live in a world in which they can't act or have responsibility or meet each other. The shocking statistics about the number of hours that people in the United States spend watching television explain, I think, all too much about our actions in the world and our lack of empathy. When a person chooses to spend that much time watching television, it's equivalent to death as far as society is concerned. They cease to function as a responsible or social persons during the time that they're simply perceiving media.

Now Virtual Reality is just the opposite. First of all, it's a network like the telephone where there's no central point of origin of information. But, much more importantly, since nothing is made of physical matter, since it's all just made of computer information, no one has any advantage over anyone else in their ability to create any particular thing within it. So there's no need for a studio. There might be occasional needs for one, if somebody has a bigger computer to generate a certain kind of effect, or certainly if somebody's assembled people of a certain talent or reputation. But in general there's no built-in difference from one person to another in terms of ability to create.

This means that there's going to be such a profusion of different forms. There will be movie studios that get involved in making Virtual Realities, but I think more there will be little entrepreneurs who will be like Reality Troubadours who will travel about spinning realities, if anything. What'll happen is that there will be such an enormous variety of form that "things" will become cheap. Basically, in a Virtual Reality everything is in infinite supply, except for one mysterious thing called creativity. And time, certainly, and health, and other things that are really still inside your body. But in terms of external things, they're infinite and wonderful and abundant and ever-varied and all equally valuable because they all can be made as easily.

So what really is of value, what really will stand out as a foreground against a background in Virtual Reality is quite different than what stands out in the physical world. In the physical world mere excess or novelty will often make something stand out. A thousand dollar bill will stand out in the physical world. In the Virtual world there is absolutely no difference between a thousand dollar bill and a one dollar bill; they are simply two different graphic designs and they are both as plentiful as you can make them.

Other people are the life of the party in Virtual Reality. Other people are the unique things that will animate Virtual Reality and make it astonishingly unpredictable and amazing. Personality will be accentuated since form will be so cheap; since form will be so non-precious, personality will be quite accentuated.

A good experiment to do is to observe somebody watching television. They look like a zombie. Then watch somebody using the telephone and they look animated. The difference is that one is a social medium and the other is a broadcast medium. In the social medium they're interacting with people. Virtual Reality is like that, more so than any medium ever has been, including, I believe, things like spoken language. And so you'll see people activated and animated. When people get social and see each other, especially in a context that will be so, let's say, 'illuminating' in the sense .... Virtual Reality is the ultimate lack of class or race distinctions or any other form of pretense since all form is variable. When people's personalities meet, freed of all pretense of that kind in the virtual plane, I think that will be an extraordinary tool for increasing communication and empathy. In that sense it might have a good effect on politics.

You can't really ask what the purpose of Virtual Reality is because it's just too big. You can ask what the purpose of a chair is because it's a small enough thing to have a purpose. Some things are just so big that they become the context, or they become the problem.

AH: "That is what we mean by a paradigm shift."

JL: I think Virtual Reality will have an effect of enhancing and, in a sense, completing the culture. My view is that our culture has been abnormally distorted by being incredibly molded by technology, but when technology was astonishingly young. I mean, television is a weird anomaly that will be remembered as a bizarre technology in the 20th century, and Ronald Reagan could only exist in television. We have to remember that we're living in a very peculiar bubble. Virtual Reality, by creating a technology that's general enough to be rather like reality was before there was technology, sort of completes a cycle. I think the reasons for having Virtual Reality are everything. There's recreation, there's education, there's expression, there's just pure work, there's therapy - all of those things. All of the same things that you'd find in language or physical reality or any other very large human pursuit.

AH: There's been a lot of loose talk over the past few years about Gaia, about our planet being an organism. What kind of vision can we have of Virtual Reality becoming an externalized consciousness of that organism?"

JL: It's an interesting question. Virtual Reality represents a new mystery on the order of the mystery that nature presents us. It's a mystery that's entirely human-made in that it's the intersection of people in Virtual Reality that creates the mystery, that creates the chaos that will make it into a full-scale reality that's worthy of being experienced.

I don't view machines as becoming conscious myself. It's not that I'm opposed to the notion; it's simply that I think that it's the wrong question to ask. But I do think that there will be a new emergent social consciousness that can only exist through the medium of Virtual Reality. Virtual Reality is the first medium that's large enough not to limit human nature. It's the first medium that's broad enough to express us as natural beings. It's the first medium within which we can express our nature and the whole of nature to each other. Actually, that's all rather vague, so let me just say that when we can make nature ourselves, we can empathize with the nature that there is and appreciate it fully.

pull quote:
The flow anywhere is the same flow

We've split ourselves off in this culture of ours from nature. Our egos are very important to us and we really separate ourselves off from environment and from the overall flow of life. What'll happen is that in Virtual Reality we'll recreate the flow. The flow anywhere is the same flow, so the flow that we create in Virtual Reality will be a new flow but it's also a part of the same eternal flow and we'll become all of a sudden....

See, now there's an opposition. Our power in the world, our power of action, has always involved doing things to physical matter. It's very slow to do things to physical matter, so in order to be powerful we have to limit ourselves. Now in Virtual Reality, all of a sudden we become powerful in terms of - we can have action without that limitation. It allows us to not wish we could behave like gods but actually to behave like gods, albeit in a simulation. But that really doesn't matter because the simulation recreates exactly the same role that the physical world has to us (it's an external shared reality).

It will reunite us with the flow of nature. Because ultimately, a new flow we create is just the same old flow of nature popping up in a new place. We'll just be paying attention to it since we'll be able to feel powerful in relation to it, which we can't with the physical one. We can do it a little bit with the atomic bomb every once in a while, but that's about it. It's actually very limited and I think it frustrates us terribly.

I think we all feel like we did when we were just born; we want so much but we can do so little. All we can do is scream and then eventually we learn to talk, and maybe we learn some technology and can do a little more to the world, but we never overcome that horrible frustration at not being able to make the world around us that we share with other people as fluid as our imagination. It's so frustrating! We are of the world, we act in the world, and yet we're limited in it.

Now of course, Virtual Reality only gives us a temporary limitlessness. We still live in our physical bodies and we're still mortal. It might highlight our mortality to a degree that will make it harder to ignore than it is currently. People imagine Virtual Reality as being an escapist thing where people will be ever more removed from the real world and ever more insensitive. I think it's exactly the opposite; it will make us intensely aware of what it is to be human in the physical world, which we take for granted now because we're so immersed in it.

The Hardware

AH: "Are these going to be linked up by phone lines ?"

JL: Absolutely. Obviously we're not talking about the current phone lines but about future phone lines, so this whole project is not something that's going to happen next year. It's going to wait until the placement of fiber optic phone lines into American homes. But that's already beginning; there's quite a few in place already.

I should point out, just for the technologically-oriented readers, that the bandwidth required for Virtual Reality is actually rather low because you're only communicating the changes in a database; you don't actually have to send images or sounds over the phone lines. So it's actually quite a low bandwidth communication. It's almost conceivable to do it with current phone lines. In fact, it probably is, if you just have a few of them instead of one. You can probably do crude ones. So it's not a major bottleneck in getting the technology working.

AH: Could you give us some idea of the state of current prototypes and how far down the road until I'll have a virtual reality device in my own home?"

JL: Well, it's very early right now. We're in the same stage with Virtual Reality that computer science was in the very earliest days. We're about in the same place computer science was back in 1958 or 1960 perhaps. The systems being built were rather large and special purpose. Only institutions could afford them. But that will be changing, and it will be changing much faster than it did with computers. The first headset, the eyeglasses, were invented in1969 by Ivan Sutherland, who was also the founder of Computer Graphics. Actually, Marvin Minsky, the founder of Artificial Intelligence, did make a pair in 1965, but the person who really got the whole thing going was Ivan Sutherland. The glove was originally invented by Tom Zimmerman. The current glove was designed by Young Harvel. Both of those people are from VPL.

Right now, all of the basic components I've described exist, although in rather crude forms. The overall system works, although in a rather crude form. The best ones are behind locked doors in defense contracting companies and probably have no bearing to any real conversation about the subject. The most fun one that's working as a complete system is the one at NASA Ames, which is called the View Lab. It was put together by Mike McGreevy and Scott Fisher. VPL has some wonderful surprises in store, but part of the fun is not telling just yet.

You'll start to be able to experience Virtual Reality within a few years. There will be Virtual Reality rooms at universities that students can do projects in. I think there will be rather spectacular amusement park rides that will be tacky and not really worth bothering with. I've toyed with the idea of opening a Virtual Reality Parlour that would be a little bit more civilized. It would be sort of like a salon scene where people could have Virtual Reality conversations and have wild experiences, but they would be collaborative. It wouldn't be like an amusement park, some dumb experience designed to get you to drink a certain soft drink and see a certain movie and buy certain clothes, but rather would be a Virtual Salon. I think that would be very nice. Perhaps we'll see something like that in a few years. I hope so. I think so.

A few years is a little bit vague, but I have to be because there are so many unknowns. But in 3-5 years, let's say, these things will be around. They'll be too expensive to have in your home, but a lot of people will be able to experience them through those institutions and businesses. On the other hand, Mattel has licensed the data glove from VPL and is marketing an inexpensive version as a Nintendo controller.

In terms of actually having them in your home, I see it as being roughly around the turn of the century when that will start to happen. It won't be so much that you buy a set of reality clothing as that it will be through the phone company. They'll own the clothing itself or they'll own parts of it and you'll own other parts.

Right now it's rather expensive, but at the turn of the century I don't think it will be. You'll pay for the time that you use it in very much the same way the telephone was introduced. I see the telephone, from a business point of view, as being an extremely analogous kind of technology. Now telephones are so cheap that you go ahead and buy them. Originally, the telephone company continued to own the equipment and made the money off of your phone bill.

In a few years we will see medical Virtual Realities, where handicapped people can experience full motion interaction with others, where people with movement disabilities or paralysis will be able to experience a complete body.

Another medical use is having surgery simulators so surgeons can enjoy the same benefits that pilots do and learn without putting lives at risk. Of course, surgeons can do that with cadavers, but it's not the same thing. A cadaver isn't the same thing as a body that really reacts, that can really bleed. You can't really make mistakes on a cadaver. There are people that are actively pursuing this. There's Dr. Joe Rosen at Stanford and Dr. Robert Chase, who are both looking at the problem from different angles. Joe Rosen might also be familiar to some people as the inventor of the nerve chip, but that's another story.

Another area is having miniature robots that could enter the human body. They would have microcameras and tiny hands. You would transfer your actions to the robot and the robot would transfer its perceptions to you so that you'd have the sensation of being inside the patient's body performing microsurgery. There are actually people now working on this technology. I'm sure none of the current projects will be the ones that work, but it is already something that people are attempting to do, and I'm sure that we'll see that sometime. I think that it will be working by the end of the century.

BS::"When I think about what kind of old age I want as well as what old age is going to be possible or feasible in a society that's going in the way it is going ... if I'm going to be locked into a very small room, I want to be locked into that room with a lot of machines that I love. So in a way it will liven up our old age and in the process to be connected not with people who happen to be in that neighborhood rest home but, in fact, the people who we want to communicate with who are spread all over the world . But on the other hand, it will be a good justification for their locking us up because after all, we've got our machine. It will be a cheap way to deal with us...."

Yeah, that's certainly a horrible thought. I tell you the most vivid experience of Virtual Reality is the experience of leaving it. Because after having been in the reality that is man-made, with all the limitations and relative lack of mystery inherent in that, to behold nature is directly beholding Aphrodite; it's directly beholding a beauty that's intense in a way that just could never have been perceived before we had something to compare physical reality to. And that's one of the biggest gifts that Virtual Reality gives us, a renewed appreciation of physical reality.

And so, I'm not sure what to say. I'm sure bad things will happen with Virtual Reality; there will be some pain that it plays a part in because it will be a big thing and the world can be cruel. But I think overall it will actually have a tendency to enhance people's sensitivity towards nature, towards preserving the earth, because they'll have a point of comparison.

Post-symbolic Communication

AH: Will Virtual Reality have interface with a Xanadu-like database of the world's knowledge"?

JL: Well, Virtual Reality raises the question: "What is knowledge and what is world?" As soon as the world itself becomes changeable, it becomes immediate, and in a sense description becomes obsolete. But this goes into another realm that might take a long time to describe. Very briefly, there's an idea that I'm very interested in called post-symbolic communication. This means that when you're able to improvise reality as you can in Virtual Reality and then that's shared with other people, you don't really need to describe the world anymore because you can simply make any contingency. You don't really need to describe an action because you can create any action.

Yes, there will be Xanadu-like knowledge bases, but then, once again I think that the Xanadu conception still separates knowledge and the world. Xanadu is still a way of connecting together descriptions in a network; it is still very descriptionary. Virtual Reality really opens up a territory beyond description, that transcends the idea of description.

AH: "It seems to me that having a vast knowledge base, the great ideas and images from history, would be a wealth of raw materials out of which to compose our Virtual Realities."

JL: Absolutely, absolutely. That will be wonderful.

AH: "Only it's the stage setting. We have this legacy of sets and props and costumes."

JL: Yes. Virtual Reality is a very general thing and it can do a lot of things. You can make up a Virtual Macintosh that would act like a real Macintosh within it, or a book, or a library, or a Sanskrit tablet, or whatever. It does all those things and all of those structures within Virtual Reality that act like things in the physical world are very important because they serve as the bridge. I think they'll be essential. In fact, I'm going to be talking to the Xanadu people about interfacing, about what we're doing and what they're doing, so that we'll have the bridge going from the start. I don't know how they'll feel about this but I think that from the Virtual Reality point of view, Xanadu might end up as being a standard interface to the physical world because it will have the best descriptive universe in the physical world. Computers live by description. We shouldn't, though.

Let's suppose that you could have a time machine go back to the earliest creatures who developed language, our ancestors at some point, and gave them Virtual Reality clothing. Would they have ever developed language? I suspect not because as soon as you can change the world in any way, that is a mode of expression of utter power and eloquence; it makes description seem a little bit limited. I don't fully know what this means. I don't know what direct reality communication would be like, reality improvisation without symbols. I doubt if we'll ever leave symbols behind because our brains have grown to adapt to symbols; you know, there's a language cortex now and all that.

All a symbol is, is something that we perceive that refers to something else. And so every symbol has at least a double nature, which is it in itself if you don't interpret it as a symbol and then, whatever it stands for. What happens is that in, say, a poem, there are both the thing referred to by the words as a collection and then there's also the inherent rhythm and typography and all of the other things that are the nonsymbolic aspects of it as an artifact. Even those things might also have symbolic aspects. For instance, a font might symbolize something but it also is a font in itself. It gets kind of complicated; it gives philosophers jobs. There's really no nonsymbolic way to communicate a lot of things and our lives are just built around symbols. By symbols I'm speaking pretty broadly, including gestures and pictures as well as words. Virtual Reality will inaugurate a whole new parallel stream of communication.

I've been working on a whole description of what it might be like to communicate without symbols. It has a different rhythm. For instance, in symbolic communication, you have the notion of question and answer and this kind of repartée which defines the flow of communication. In Virtual Reality, since people are collaboratively changing a shared reality as a means of communication, what you'll have is nodes of relative static quality vs. periods of very dynamic quality. There will be this rhythm between when the world is being changed quickly and when it sort of settles down. That rhythm is something like what a sentence is in language. In spoken language you have the phenomenon of finding the next world and going, "Um...um..." The same thing will happen in Virtual Reality where people will go through an interval of spacing out from the reality, preparing their next change to the shared world.

I can point to the direction of what it might be like in the general sense, but it's almost by definition impossible to make completely compelling examples. I'll give you a few, though.

If we think of an experience where you're describing something to someone else - let's suppose you're describing what it's like to live in the East Coast in these grungy violent cities and how you have a completely different set of expectations vs. living in what seems to be the rather safe and lovely but rather bland and aimless cities of California - now to describe those things ... I just did that. I just came up with some brief symbolic descriptions of what cities in New York and California are like. In Virtual Reality there's the possibility of simply playing back one's memory with the other person inside it from the other city. When you directly have at your beck and call external reality to be played back, created, improvised at will, description is simply narrow.

Now description is interesting because, in its narrowness it does bring in possibilities for poetry that probably don't exist in the fullness of post-symbolic communication where you can just create experience as a whole all the time. On the other hand, in creating experience as a whole all the time, you have this possibility of a kind of collaboration that you really can't have with symbols, where people can be simultaneously molding a shared reality.

I realize these things are hard to describe and that's appropriate. What I'm trying to describe is communication that is itself beyond description. The idea might turn out to be wrong; it might turn out that communication without symbols and description is just a silly idea and a path wrongly taken. So it's really a grand experiment, and I think it will be a lot of fun.

Of course communication without symbols already happens constantly. First of all, the clearest example of receiving communication that is nonsymbolic is to commune with nature. The direct perception you have when nature communicates to you as you walk in the forest is simply prior to/beyond symbols. There's no need to prove that. Any linguist who would argue otherwise is beneath contempt.

An example of communicating outwards without symbols is when one moves one's own body. You don't send a symbol to your arm or to your hand; you communicate prior to symbols to your own body. The most beautiful example that now exists of an intense sort of communicating outward without symbols is in lucid dreaming. When you dream lucidly you are aware that you're dreaming and you control the dream. It's rather like Virtual Reality except that it's not shared. The means by which you communicate to your dream is without symbols. There you are spinning the world, spinning anything in the world without symbols, simply making it be.

Now, of course, those are the purified examples, some purified examples of nonsymbolic communication that already exist. But, of course, all of life is deeply imbued with nonsymbolic communication. A book has its nonsymbolic aspects; I mean, a book is a book as an object prior to being a book that can be decoded as a bearer of symbols.

Everything has symbolic and nonsymbolic aspects to it. A thing isn't a symbol; it's just that you can use anything as a symbol. The idea of symbol is a use for a thing, but everything is also a thing in and of itself; everything has a primary thingness. (Twisty sentences like that are part of what led me to the search for post-symbolic communication!)

AH: "How does Virtual Reality relate to the images of cyberspace that we've had in so much recent SF"?

JL: It's better. I mean, cyberspace is just another set on which to have adolescent fantasies unfold. In these novels, like True Names and Neuromancer and so forth, people don't do anything interesting with the artificial reality.

AH: "It's like CB radio."

JL: That's right. Cyberspace is the CB radio of Virtual Reality. That's an excellent metaphor. It's a trivial use of it.

AH: Considering the freedom that people have to fantasize in D & D games, it's amazing how restricted the imagination is that's manifested."

JL: I agree completely. And I'm sure there will be mundanity in Virtual Reality as well, because mundanity is a part of humanity. But I'm not too worried about it. The whole 'economy' of Virtual Reality is set up to accentuate creativity since it is, as I said, the only thing in short supply. In a sense it's the only thing that really exists. Personality and creativity are perceivable and forms will be less and less noticed because they'll be so plentiful.

"AH: Where does that leave those of us who collect "stuff"? What are the prospects of house cleaning now?"

JL: Well, the stuff will seem more precious and the house more dirty because the physical world will appear amplified in comparison.
I'm not anti-physical world and I'm not anti-symbolic communication at all. I mean, I love those things.


AH: "Are there any historic images that come to mind that set the stage?"

JL: Oh many, many. God, that's a huge question too. So many, so many. There are the lost memory arts, the memory palaces. Most of the western cultures relied on imagined Virtual Realities, these imagined palaces that people hung their memories in as artworks. People would memorize their palaces in order to have a way of remembering things, and before Gutenberg that was a very important thing. It was absolutely as primary as music or the arts of war to a particular culture. The memory arts sort of vanished because they were rendered obsolete. But they were remarkably like Virtual Reality.

So many things come to mind about that. It's really too broad a question. Our attempts to change the physical world. We have raped the physical world because we don't have Virtual Reality. I mean, technology is just our attempt to use the physical world as a means of action. The physical world resists it and therefore we have the ugliness that we live with all the time. But Virtual Reality is the ideal medium for that type of action. Just architecture, technology in general, is really the strongest precedent, our attempts to modify the physical world as a means of human action. That's the strongest precedent.

Oh, so much more. The very first time that somebody put on glasses that had imagery in them actually wasn't - I mean the glasses I mentioned that were done by Marvin Minsky and Ivan Sutherland were the first ones that had computer imagery. But some people had attached a stereo camera to a set of eyeglasses with stereo television sets as early as 1955. Some engineers from Philco had it rigged up in a periscope-like setup. There was a stereo camera on the top floor of a building. You could look out through it from inside the building. It had a limited degree of tracking, so you could have the feeling of looking over the side of the building. That was a big thrill back then. It probably still would be now.

AH: I can imagine what a thrill it was for the first people who looked through their stereopticons back at the turn of the century."

JL: Absolutely. There are so many precedents. I think Virtual Reality is a major node of culmination of culture. I think there are just enormous things that are fulfilled by it and an enormous number of things that can be seen as precedents.

Go back to Jaron's home page.