An Abbreviate History of Augmented Reality

A colleague is working on a book chapter in which he wants to include a section on AR.  He asked me for a brief history (not to be include in the book) to give him an idea of how AR came to be what it is today.  I sent him the following.  There is A LOT missing in here, but it was only intended to be an informal “one page history”.  I thought I share it here.

Augmented reality has a history much longer than most people think.  To understand its history you must first understand what “augmented reality” means at its most fundamental.  At its most basic, augmented reality is quite literally augmenting one’s experience of the physical world by modifying or adding sensory information that would not otherwise be available.  In one way or another we have been doing this since torch light was first used to allow us to see clearly in the dark.  However, augmented reality in its modern form (and potentially future forms) was first formally conceptualized by Ivan Sutherland in his historic article “The Ultimate Display”.  Not long thereafter, Sutherland developed the first head-mounted augmented reality display.  This display was dubbed “The Sword of Damocles” since the device and its associated equipment seemed to dangle precariously above the head of the user.  For better or worse, the basic design of most augmented reality systems, and later virtual reality systems, remained essentially unchanged for decades to come.  (An important side note is that of a common misconception.  Many people assume that virtual reality was the predecessor to augmented reality, mostly due to VR’s early popularity in the 1990s.  However, Sutherland’s 1968 AR display served as the template for essentially all future AR and VR displays.)

For quite some time afterward advances in augmented reality centered around non-head-worn displays, like the head-up displays (HUDs) often used in modern aircraft.  These displays enabled pilots to view critical control information while retaining a direct view of their surroundings.  As is the case with many new technologies, the ramifications of its adoption were not fully understood at first.  Unfortunately, misconceptions and misapplications of head-up displays led to multiple dozens of preventable aircraft accidents in the early 1980s.  However, through considerable research and hard earned experience HUD designs improved to the point that they are common place in many commercial and military aircraft.

As computer and display technologies became faster and smaller, virtual reality saw a surge in popularity in the 1990s.  However, the majority of the systems developed during this time could only offer limited experiences and usually only in tightly controlled lab settings.  Excellent examples of these were the virtual reality systems employed by NASA to visualize and virtually explore the terrain of our neighboring planets.  (One of the VR displays used by Michael McGreevy, Steve Ellis, and others from NASA Ames is currently on display at the Smithsonian Air and Space Museum.)  A few commercial systems were available to the public though.  Unfortunately, most of these were expensive, cumbersome, and provided lackluster experiences.  Perhaps a too often cited example is the Nintendo Virtual Boy game system which debuted in 1995.  The Virtual Boy, though hyped among the youth at the time, offered a monochromatic, low resolution gaming experience that was often reported to quite efficiently make players very uncomfortable, a condition dubbed simulator sickness.

In the late 1990s, augmented reality saw a bit of a resurgence with the release of commercially available see-through head-mounted displays such as the Sony Glasstron.  The Glasstron was mostly geared toward businessmen who wanted to discretely watch videos while traveling, but it became one of the most commonly used displays in augmented reality research due to its small size and relatively affordable price.  It was around this time that AR for training became a point of interest and led to the development of experimental AR frameworks for military training.  Though a myriad of augmented reality display designs and manufacturers have come about since the early 2000s, the same basic head-worn archetype pioneered by Sutherland persisted as the dominant display type.  This, however, rapidly changed as smartphones became ubiquitous.  Smartphones contained almost all of the components used in AR and VR systems: cameras, motion sensors, and a display.  This gave rise to a “magic window” form of augmented reality that allowed users to view computer generated imagery superimposed on the video feed from the phone’s camera.  This method became quite popular and has even been adopted in the realm of handheld game systems.  Though smartphone and tablet based AR applications are becoming quite common, there are still significant efforts to bring head-worn AR into the mainstream.  Probably the most notable of these is the Google Glass project which is pushing to bring lightweight, see-through head-up displays to the masses.  However, substantial research is still being devoted to the use of traditional head-worn stereoscopic 3D AR and VR displays.  This will likely continue as cheap VR displays, like the Oculus Rift, are becoming more commonplace.

I have an interesting story about the Oculus Rift that I might post later.  We’ll see if time allows.