Hasta La Gadget!

As you shove your way through the crowd in a baseball stadium, the lenses of your digital glasses display the names, hometowns and favorite hobbies of the strangers surrounding you. Then you claim a seat and fix your attention on the batter, and his player statistics pop up in a transparent box in the corner of your field of vision.

It’s not possible today, but the emergence of more powerful, media-centric cellphones is accelerating humanity toward this vision of “augmented reality,” where data from the network overlays your view of the real world. Already, developers are creating augmented reality applications and games for a variety of smartphones, so your phone’s screen shows the real world overlaid with additional information such as the location of subway entrances, the price of houses, or Twitter messages that have been posted nearby. And publishers, moviemakers and toymakers have embraced a version of the technology to enhance their products and advertising campaigns.

“Augmented reality is the ultimate interface to a computer because our lives are becoming more mobile,” said Tobias Höllerer, an associate professor of computer science at UC Santa Barbara, who is leading the university’s augmented reality program. “We’re getting more and more away from a desktop, but the information the computer possesses is applicable in the physical world.”

Tom Caudell, a researcher at aircraft manufacturer Boeing, coined the term “augmented reality” in 1990. He applied the term to a head-mounted digital display that guided workers through assembling electrical wires in aircrafts. The early definition of augmented reality, then, was an intersection between virtual and physical reality, where digital visuals are blended in to the real world to enhance our perceptions.

Futurists and computer scientists continue to raise their standards for a perfectly augmented world. Höllerer’s dream for augmented reality is for it to reach a state in which it does not rely on a pre-downloaded model to generate information. That is, he wants to be able to point a phone at a city it’s completely unfamiliar with, download the surroundings and output information on the fly. He and his peers at UCSB call this idea “Anywhere Augmentation.”

But we have a long way to go — perhaps several years — before achieving Anywhere Augmentation, Höllerer said. Augmented reality is stifled by limitations in software and hardware, he explained. Cellphones require superb battery life, computational power, cameras and tracking sensors. For software, augmented reality requires a much more sophisticated artificial intelligence and 3-D modeling applications. And above all, this technology must become affordable to consumers. The best possible technology that is available today would nearly cost $100,000 for a solid augmented-reality device, Höllerer said.

Given the cost of creating decent augmented-reality technology, early attempts have focused on two areas. One, augmented reality for your computer is prominently appearing in attention-grabbing, big-budget advertisements. And a few consumer applications of the technology are just beginning to surface in smartphones.


A recent example of augmented reality appeared in the marketing campaign for the sci-fi blockbuster District 9. On the movie’s official website was a “training simulator” game, which asked computer users to print a postcard containing the District 9 logo and hold it in front of a webcam. The postcard contains a marker; when the game detects that marker in the webcam video, it overlays a 3-D hologram of a District 9 character on the computer screen. From there, players can click buttons to fire a gun, jump up and down or throw a human against a wall in the game. (See video above.)

Mattel is using the same type of 3-D imaging augmented reality in “i-Tag” action figures for James Cameron’s new movie Avatar. The toy includes a card containing a marker, which is projected as a 3-D action figure on a computer. This way, children can battle each other’s virtual characters on a computer screen.

But augmented reality isn’t truly useful in a static desktop environment, Höllerer said, because people’s day-to-day realities involve more than sitting around all day (outside of work, at least). And that’s why smartphones, which include GPS hardware and cameras, are crucial to driving the evolution of augmented reality.

Brian Selzer, co-founder of Ogmento, a company that creates augmented reality products for games and marketing, recognizes the need for augmented reality to go mobile. He said his company is working on several projects coming in the near future to help market mainstream movies with augmented reality smartphone apps. For example, movie posters will trigger interactive experiences on an iPhone, such as a trailer or even a virtual treasure hunt to promote the film.

“The smartphone is bringing AR into the masses right now,” Selzer said. “In 2010 every blockbuster movie is going to have a mobile AR campaign tied to it.”

On the consumer end of the spectrum, developers have recently released augmented reality apps for the Google Android-powered HTC G1 handset. Layar, a company based in Amsterdam, released an augmented reality browser for Android smartphones in June. The Layar browser (video above) looks at an environment through the phone’s camera, and the app displays houses for sale, popular restaurants and shops, and tourist attractions. The software relies on downloading “layers” of data provided by developers coding for the platform. Thus, while the information appears to display in real time, it’s not truly real-time: The app can’t analyze data it hasn’t downloaded ahead of time.

“This is the first time media, internet and digital information is being combined with reality,” said Martin Lens-FitzGerald, co-founder of Layar. “You know more, you find more, or you see something you haven’t seen before. Some people are even saying that it might be even bigger than the web.”

Cellphone giant Nokia is currently testing an AR app called Point & Find, which involves pointing your camera phone at real-world objects and planting virtual information tags on them (above). Users of the app can view each other’s tags on the phone screen, essentially crowdsourcing an augmented reality.

“This year we’re feeling a real urgency to work on augmented reality because the hardware is finally catching up to our needs,” said Rebecca Allen, director of Nokia’s research center in Hollywood.

Georgia Tech is also busy tinkering with augmented reality. The video demo above demonstrates an augmented-reality zombie shooter called ARhrrrr. The smartphone in use is a prototype containing an Nvidia Tegra, a powerful chip specializing in high-end graphics for mobile devices. How do you play? Point the phone camera at a map containing markers, and a 3D hologram of a town overrun by zombies appears on the phone’s screen. Using the phone, you can shoot the zombies from the perspective of a helicopter pilot. And you can even place (real) Skittles on the physical map and shoot them to set off (virtual) bombs.

As for the iPhone, officially there are no augmented reality apps in the App Store yet — because Apple doesn’t provide an open API to access live video from the phone’s camera. This barrier prompted augmented reality enthusiasts and professionals to write an Open Letter to Apple pleading for access to this API to make augmented reality apps possible in the App Store.

Brad Foxhoven, Selzer’s partner at Ogmento, said Apple has told him the next version of the iPhone OS (3.1) “would make [AR developers] happy,” implying the live-video API will become open, and AR apps will become available very soon.

Meanwhile, some augmented reality developers have already hacked away at the iPhone’s software development kit to code proof-of-concept augmented reality apps. The video above demonstrates an app called Twittaround, an augmented reality Twitter viewer on the iPhone. The app shows live tweets of mobile Twitter users around your location.

“We’re doing as much as we can with the current technology,” Selzer said regarding the overall augmented-reality developer community. “This industry is just getting started, and as processing speeds speed up, and as more creative individuals get involved, our belief is this is going to become a platform that becomes massively adopted and immersed in the next few years.”

See Also:

• At the Dawn of the Augmented Reality Industry
• Augmented Reality and atemporality
• Augmented Reality, coming soon to a government near you
• Blink-182 Rocks ‘Augmented Reality’ Show in Doritos Bag …
• African Augmented Reality
• Augmented Reality Business Cards: WoW For Suits?
• 3D Avatar in Baseball Cards is the New Bubble Gum

Photo: Layar