Augmented reality technology is getting a lot of attention these days — particularly the use of AR with smartphones. The idea is that by using certain software, you can turn your iPhone, Droid or other smartphone into a virtual heads-up display.
Aim your phone’s camera at a shop, restaurant or landmark, and information about the place, such as hours of operation, reviews or directions, appears on the device’s screen as graphics floating over the image of the place.
Dozens of developers of mobile augmented reality apps are banking on AR becoming the Next Big Thing in the mobile market. Indeed, a recent Juniper Research report predicted that annual revenues from mobile AR apps will reach $732 million by 2014, up from less than $1 million in 2009.
Reality through the eye of the iPhone holder?
Related Story: John Mayer’s augmented reality video offers fans an immersive experience
During this early stage in the use of mobile AR applications, users should be advised not to set their expectations too high, because current models of smartphones have limited capabilities. But the biggest question is whether augmented reality will turn out to be nothing more than hype.
Remember a few years ago when corporations rushed to establish virtual offices and storefronts in Second Life and other virtual worlds, only to see them wither on the vine? While AR appears to be more useful than virtual worlds (and therefore more likely to succeed), it remains to be seen how the technology will be developed and adopted in real-world use. In particular, those in the business world would like to know if, and when, their operations could somehow benefit from using AR.
With those thoughts in mind, here’s a short primer on AR for mobile devices, along with some need-to-know points about the technology.
Augmented reality 101
The term augmented reality can actually be applied to two types of technologies. One version of AR involves systems that use a webcam or a video camera to capture an image of a user (his head, hands or body, etc.) or of a real-world object in real time and put that image on a computer screen. Software then tracks the user’s or the object’s movements in real space so it appears that that user or the object is interacting with a virtual object (like a 3D graphic model) on the screen.
This type of AR technology has been used in video games (like the EyeToy for the PlayStation 2), in promotional tools (like this system in Lego stores), and as an online shopping aide — for example, a system could allow a shopper to “try on” clothes virtually before making a purchase. In the business world, this version of AR might be used to test products or marketing ideas. Total Immersion SA is one company that sells this type of technology to other businesses.
Wikitude World Browser at Seattle’s Space Needle.
The second kind of augmented reality systems use webcams or the cameras of smartphones or other devices to capture real-world images and then lay text, links or other objects (again, like a 3D graphic model) over the images on-screen. With this type of app, you can point the camera of an Internet-enabled device at a building or landmark and receive helpful information about it right on your screen.
That’s how many of today’s AR apps work. They include GeoVector’s World Surfer (for iPhone OS and Android) and Mobilizy’s Wikitude World Browser (for iPhone OS, Android and Symbian OS). This form of augmented reality has been getting the most attention because of the novel way it allows the user to interact with the world. But how does it work?
First, the AR app reads your phone’s GPS data to find your location on the planet. Then it determines the phone’s orientation from its electronic compass, and in some cases its accelerometer, to determine which direction you’re pointing. (The compass indicates the direction in which the device is being pointed; the accelerometer determines its tilt.)
The app then searches its database for objects (text, hyperlinks, pictures, etc.) that have been location-tagged (categorized by latitude, longitude and altitude coordinates) in the indicated compass direction from your geolocation. If it finds any such objects (which could be provided by other users, a third-party provider such as a mapping service or Web site, or the app’s developer), it lays them over the image of the building or landmark on your screen.
A GeoVector illustration shows how mobile AR apps do their stuff.
Say you aim your phone’s camera at a restaurant. The AR app should be able to use the information in its database to identify the establishment and pull up specific pieces of information about it — operating hours, a menu, reviews, directions and so on — and superimpose links to that data over the image of the restaurant as it appears, in real time, on the screen of your phone.
Mobile AR apps are still in their infancy
World Surfer, Wikitude World Browser and Yelp’s iPhone 3GS app are typical of today’s mobile AR apps. They let people use smartphones to search for information about nearby restaurants, businesses and landmarks. Users can contribute comments and reviews about such places, and those reviews will be available to others, who you can access them by clicking on links that appear over the images captured on their smartphones.
Acrossair’s Nearest Subway iPhone app.
Such apps aren’t limited to providing information about objects that are directly in front of you, nor do they always focus on businesses and landmarks. Acrossair, for example, has a series of iPhone public transit apps that help users navigate the subway systems in select cities. Simply point your iPhone in a given direction, and the app shows the names of subway stations and transit lines located in that direction; it also tells you how far away each station is.
Mobile AR apps usually first list the place of interest that’s closest to you in the direction you’re pointing, followed by other places that are farther away; they tap mapping information from their databases to provide you with directions to more distant objects.
This all sounds pretty nifty, but because of the technological limitations of today’s smartphones, the data these apps display isn’t always correct. The one that most effectively handles local searches and provides the best walking directions is World Surfer, says Gene Becker, a blogger who tracks the latest developments in AR.
A few apps are taking mobile AR in different directions. Metaio’s iLiving app for the iPhone lets you take a picture of a room and place 3D graphic models of furniture into the photo and then share screenshots of this virtual setting with your friends via social networks. It’s a fun way to design a new office or rearrange your living room.
The recently launched Junaio iPhone app, also from Metaio, is a mobile AR platform designed to foster creativity. With Junaio, users can publish their own “scenes” online by sticking 3D graphic models (some of which are animated) over images of locations they capture with their smartphone cameras. They can also share these scenes with friends via Facebook or Twitter. Junaio and iLiving showcase the fun, community-oriented uses of AR, even if they may not be particularly “useful.”
The Juniao mobile AR app with user-inserted dinosaur.
SPRXmobile’s Layar 3.0 platform (for iPhone and Android smartphones) is similar to Junaio, but SPRXmobile is encouraging content developers to contribute a wealth of relevant, practical information about real-world businesses, landmarks and locations in hopes of creating utilitarian apps that would, for example, guide users to a bank’s ATMs or overlay photos and data about historical buildings on present-day locations.
Present mobile AR technology isn’t precise
The potential market for AR products leans toward mobile devices, but, as mentioned above, current smartphones have technological limitations that curb their ability to support AR software. For one thing, their batteries don’t hold a charge long enough; any app that makes heavy use of graphics, GPS technology and networking components uses a lot of juice. But the biggest problem is that the GPS technology used in smartphones needs to be improved.
Wikitude World Browser at London’s Piccadilly Street.
“Most mobile augmented reality apps… are focused on presenting useful information about the physical world, but they fall somewhat short in real-world use because of the limitations of the location and orientation technologies they rely on,” says Becker.
Data displayed by mobile AR apps is not always accurate because the GPS sensors in smartphones are not able to zero in on buildings and objects that are within a few meters or so of where you stand. The range of GPS sensors in smartphones is the same as that of a typical GPS car navigation device — about 20 meters (a little over 65 feet). While this is good enough for relaying information about streets when you’re in a moving vehicle, it isn’t tight enough to home in on — and, thus, convey information about — buildings and landmarks that you walk by in real time.
Additionally, the response time of the electronic compasses found in smartphones often lags, which can make certain objects (like 3D graphic models) seem to “float” in the air on a device’s screen, rather than staying pinpointed on an exact location.
Another way the technology falls short — in the United States, anyway — is that most AR apps are limited to using a smartphone’s GPS technology and compass to present a single-level perspective. Apps that can search for objects which are higher or lower than ground level — the upper stories of buildings, or floors located below ground — use accelerometers to add tilt information to directional data, according to Pam Kerwin, who’s in charge of strategic business development at GeoVector Corp., the company behind the World Surfer mobile app. “But most data has no height coordinate in the database,” she adds. “This is pretty much a Japan feature at the moment.”
A final problem is the lack of “depth perception.” A mobile AR app might be able to guide you to the nearest McDonald’s, but if the restaurant is on the next block, behind the building you’re currently standing in front of, the information about the McDonald’s might appear to be superimposed on the image of the building in front of you.
Not much mobile AR technology has been developed specifically for enterprise or IT use — so far
Mobile augmented reality applications have been designed mainly for the consumer and manufacturing markets. There has been a notable lack of mobile AR apps for the enterprise. “Augmented reality has proven to be useful in product manufacturing and design environments, and for repair of complex machinery. However, most IT and enterprise applications of augmented reality are still a few years off,” says Becker.
“I can’t see any applications of augmented reality in the office environment,” says Dominique Bonte, an analyst at ABI Research. Though he does say that he could foresee outdoor uses of AR apps in sectors such as real estate.
A Layar app shows the closest ING bank ATMs in Amsterdam.
Kerwin explains that mobile AR is best for office workers who don’t sit in cubicles all day. “Augmented reality is very useful for any worker who needs to operate outside: maintenance workers trying to locate machines, sales reps visiting unfamiliar places, [employees trying to find their way around] the corporate campus, workers in the field trying to locate one another to collaborate or send safety instructions to employees in dangerous areas,” she says.
The practical value that mobile AR technology offers the enterprise is that it presents information in relation to the real world. Developers of mobile AR apps will have to do this not only conveniently and quickly, but also economically. Says Gartner analyst Jackie Fenn: “One of the challenges for enterprises will be connecting up data to its relevant location or collecting relevant image databases. It’s often harder to cost-justify compared to the economies of scale of consumer apps.”
Unlike virtual worlds, mobile AR may actually stand a chance in the market
A big question at present is whether the demand for mobile augmented reality will endure and grow over the long haul or whether AR will follow in the footsteps of other overhyped technologies, such as virtual worlds.
Even Peter Meier, chief technology officer at Munich-based Metaio GmbH, developer of the Junaio and iLiving apps, acknowledges that the hardware capabilities of smartphones and the interfaces of AR apps themselves need improvement for augmented reality to succeed. “Unless it works well, it is not ready for the general audience. Would you use a touch screen that only sometimes works? When the technology is right, when it delivers the exact expectations of the consumer, then it will prove to be useful,” he says.
GeoVector’s World Surfer pulls info on AT&T Park from Flickr, Google, YouTube and more.
Meier’s company and the other major developers of AR applications appear to be in it for the long term, though. Most of them plan to improve the accuracy of data displayed (by implementing technologies such as markerless tracking) and add more sophisticated contextual information and image recognition to their apps within a year, observes Gartner’s Fenn.
Still, he says, “there are so many useful and compelling applications for delivering context-sensitive data. But like many of these technologies, figuring out how to make money from it will be a challenge for everyone.”
Jeremiah Owyang, an analyst at Altimeter Group in San Mateo, Calif., says that the key to avoiding the fate of virtual worlds is for AR developers “to look for the intersections between the real world and the existing Internet — not creating a new virtual world. Consumers will find value [in augmented reality] when Internet content is provided to them in context when they need it in the real world, such as seeing reviews about restaurants from people they trust as they make a decision while walking down the street.”
A natural extension of human perception
At the moment, the idea of augmented reality for mobile devices might be regarded as hype, since many of these apps focus on the technology’s fun and gimmicky aspects. And, because of the limitations of current GPS sensors in handheld gadgets, the location accuracy of objects displayed in most AR browsers is not quite up to par.
World Surfer displays San Francisco historical info.
Within the next few years, though, it’s inevitable that more advanced smartphones and mobile devices will hit the market. This should lead to the development of mobile AR apps that are better able to recognize and track a wide range of objects and landmarks.
As for overcoming the stigma that it’s nothing more than marketing hype, AR might succeed where virtual worlds faltered because it’s grounded in real life. Becker muses that AR could become accepted as a natural extension of our perception of, and interaction with, everyday surroundings.
“Humans are driven to augment our reality, and to augment our own capabilities. From the earliest cave paintings to modern-day urban graffiti, we overlay our world with expressions of our inner selves,” he says. “Architecture, street signs, billboards, fashions — these are all visual and functional augmentations of the physical world.”
Howard Wen reports for several technology publications.
Source: Computerworld.com