JoAnn Hackos, Comtech Services, Inc.

Is Augmented Reality in the future of technical information development? That was the question at the heart of the Augmented Reality Think Tank, sponsored by Huawei in Dallas in October 2014. The Think Tank brought together a collection of knowledgeable individuals from academe and industry to learn about Augmented Reality (AR) and assess its readiness as a new kind of customer-facing information deliverable.

The conference was kicked off by Kit Brown-Hoekstra, Comgenesis, who is the current STC president. She asked the key questions: What is the biggest challenge in developing content for augmented reality now? Besides gaming, which industries have the greatest potential for AR?

She suggested that industries with many physical assets to manage such as the automotive, heavy-equipment, and medical-device industries might be important candidates to use AR. Technical content in general is moving away from the book. However, if the source content is not ready, is not well prepared, it may not be ready for a move to a visually enhanced platform.

Kit’s concerns were echoed in Steve Feiner’s presentation. Feiner is a professor of computer science at Columbia University and has worked on Augumented Reality for many years. He and his graduate students began in the mid-90s testing AR solutions for the US military. Their goal was to overlay information in the user’s visual field as they perform a task.

As a result of the tests, they found that the users oriented their heads closer to the ideal position for viewing the machinery. With the paper manual, they were constantly moving their heads back and forth between the manual and the machine. Unfortunately, they were wearing heavy and clumsy headgear.

Perhaps most interesting to information developers, however, was the need for the researchers to “enhance” the government manuals. Feiner mentioned that they were so bad that they could not use them in the test and had to rewrite them. The content was clearly not ready to support a move to AR.

In subsequent experiments, in a more controlled environment, the researchers were able to show that AR was faster for alignment and pinning tasks, that the alignment task was performed more accurately, and the AR presentation was preferred by the subjects and seemed to them to be more intuitive.

When we asked what it would take to produce a first-class AR task presentation, Feiner stated that an expert must be videotaped performing the task. Note that many information developers would love to have an expert perform a complex task for them so that they could write the best procedure. I also notice that the AR segment we viewed indicated to the users when they made mistakes. That information, the immediate troubleshooting information for a task, is extraordinarily valuable to users but rarely available to the writers.

Andy Lowery and John Trozak from DAQRI, a technology company developing AR software and lightweight, ergonomically efficient helmets, showed us more about what AR might look like. They believe that AR is the 21st-century way of communicating, aiding perhaps 10 percent of industrial applications with this technology. They believe that the focus should be on applications that promise a very large return on investment.

They have conducted several promising studies using AR for assembly instructions. In one study of an airplane-wing assembly, they compared an AR instruction using a tablet against the written instructions. The average assembly time was reduced by 30 percent, and errors were reduced by 94 percent. In addition, the ability of the software to track the technicians’ behavior enabled them to find and correct steps in the procedure that were causing problems. The industrial analytics provide a heat map of how people are engaging with the machine.

They assert that with their 4D studio program, people who develop work instructions can be trained to develop AR using drag-and-drop tools.

Cecilia Abadie is a well-known AR developer, who joined the conference wearing her Google glass. She believes that wearables will be the next wave of technology for the consumer and in the workplace. She provided examples of AR used to teach medical procedures and bring patient data into the surgeon’s field of view. She mentioned firefighting applications that provide information on a building to the first responders. She has developed museum displays that provide the viewer with information on the exhibits and tourism displays that help tourists navigate a city.

Opportunities for AR abound in education and sports and can assist people with disability better function. Cecilia is known, by the way, for winning her case in California to allow people to wear Google glass while driving.

By focusing AR development on highly complex and rarely performed procedures, Christine Perey, an AR consultant from Switzerland, believes that companies can gain a competitive advantage. She believes that manufacturing, including prototyping, assembly, and control, is an ideal environment for AR. She pointed to examples that used AR for factory-floor planning to ensure that no problems in the design would be found after it was too late to correct them.

A representative from a second technology developer, Alexander Oser of Metaio, demonstrated their experience developing applications for automotive, retail, and healthcare applications. In fact, you may have already seen their development of 3D modeling for Lego in a Lego store near you. Children can hold the Lego box of their choice up to a screen and see a 3D model of the finished object.

Several of the applications that Alexander illustrated seemed like magic. You could, for example, point to a button on a device in your car and view the owner information in your field of view. You might use a camera on a drone to map a landscape and then use the images to assist with navigation and development planning. He talked about thermal touch, which might be able to discover if someone has touched an object in the real world. He showed us the output of an RGB-D camera that could allow us to place a virtual object in our real world. He showed two different chairs being placed virtually in a room so that we might see how they would look.

Metaio, based in Munich Germany, has been working with Volkswagen to develop Mobile Augmented Reality Training Assistance. The steps of the maintenance procedure are placed on a tablet used to view the machine, the real machine, not a picture of it. Andrew showed us a Mitsubishi Electric example in which field technicians access information from the Cloud that recognizes the specific device they need to repair. The maintenance instructions are then superimposed on the actual object. As a result of the AR instructions, technicians were able to complete their work more quickly and take one additional house call per day. That productivity gain resulted in significant cost savings for the company.

The challenge, as Alexander pointed out, is that AR currently depends on the existence of a 3D CAD model of the machine. The software uses the 3D model to map the visual field and superimpose instructional cues like blinking lights, colors, and moving arrows. Unfortunately, many companies don’t have the 3D models.

Consequently, Metaio is working on technology that would be able to map the edges of an actual machine and essentially build what was needed of a 3D model, opening up the technology to many more companies.

Although most of the examples we saw during the Think Tank required either downloadable information to a tablet or helmet or used information accessed through a cloud database, Clark Dodsworth of Osage Consulting explained how we might use AR locally, especially with the capabilities of cell phones. By computing with other phones in the network, our cell phones could provide us with information about our locations, the weather, what is around us, our calendars, and more.

Tilanka Chadrasekera, an interior design professor from Oklahoma State University, showed us how he has been adding AR tools into the design classroom. Along with virtual reality tools, AR gives students the ability to do more realistic and fulfilling design work.

Robert Atkins, who teaches game design at Southern Methodist University, exampled how virtual and augmented reality will be added to the gaming world. He also cautioned us to avoid over complication. I thought he was a genuine minimalism advocate. His descriptions of the golden rules of AR design were the same as the rules for User Interface design. He told us to

  • Be consistent in the rules and style of interaction
  • Keep it simple by avoiding putting too much cognitive load on the user
  • Draw from the familiar, what the user already knows about
  • Use one button for one function and allow users to focus on the experience, not the controls
  • Structure the learning curve so that users get the primary actions right before being introduced to complexities
  • Document everything you design because the human memory is fallible and selective

Christine Perey provided some important cautions that those of us just learning about this technology must take into account. At present, there are no standards for AR. Each developer is doing something different, which results in many proprietary technology silos and no interoperability. She is working with an AR Community, established in 2009, with about 50 people helping to create a reference specification in ISO to relate AR to other standards. Those standards include 3D compression and transmission, AR browser interoperability, and others. You can find out more about this organization at

During the Q&A session we discussed the challenge for information developers in building a new skill set. There are degrees in AR taught primarily through computer science, graphic design, and communications. It seemed most interesting for an organization to develop a group of experts to work together to investigate AR opportunities, build a business case, gather the expertise required to create content, and ensure that the development is an integral part of the corporate content strategy.

Without planning in which AR becomes part of the whole rather than an outlier, it appears likely that AR could become another niche development project somewhere in the organization, manned by enthusiasts but outside of, once again, a corporate-wide strategy.

I want to thank Rhonda Truitt, Global Director of Documentation Innovation and Best Practices at Huawei, and her colleagues for planning and sponsoring the AR Think Tank. It was an eye-opening (almost literally) experience. I even got to ride a roller coaster through Oculus Rift, the first time I’ve been on a roller coaster since my devastating experience as a five-year old.