AFRL reimagines tech development with virtual reality

WRIGHT-PATTERSON AIR FORCE BASE, Ohio – In a move the Wright Brothers likely never anticipated but surely would approve, the Air Force Research Laboratory Aerospace Systems Directorate is using a bicycle to demonstrate how virtual and augmented reality could revolutionize aircraft maintenance, as well as collaborative partnerships.

At the Air Vehicles Technology Symposium, held Sept. 10-12 in Dayton, the Virtual, Augmented, and Mixed Reality for Aircraft Maintenance, or VAMRAM, team demonstrated a unique potential use for a technology most commonly associated with video gaming. Using a Wright Cycle Company bicycle, commercially-available smartglasses, and a software application developed in conjunction with small business partner Energective, the team showed off a mixed reality display tool that can provide crews step-by-step instructions and visual cues that guide them through aircraft maintenance procedures.

This tire change project is part of the team’s larger effort to streamline the acquisition and development process through an open data- and idea- sharing community driven largely by technology end users.

Using the bicycle as a basic vehicle surrogate, the AFRL team gave symposium attendees the opportunity to use the tool to engage in the process of changing a component, from start to finish. Through the headset, users were able to see real-time visual aids laid over the actual bicycle wheel, guiding them to the correct part, providing direction, and enabling them to complete the task.

This tool holds the potential to streamline the sometimes cumbersome aircraft maintenance process, which often requires crews to refer back and forth to a set of paper instructions while simultaneously interpreting and executing the actions onboard the aircraft.

“Typical content delivery of the technical data is more prescriptive,” said VAMRAM team co-lead Capt. David Eisensmith. “It says ‘do this’ and you interpret it from the pictures and do it in the physical space. With this system, AR gives you visual cues to show you what to do, so it’s more instructive than prescriptive.”

Although the team emphasizes that this technology is in the early exploratory stages, Eisensmith said it holds the potential to be a game-changing tool for the maintenance community. Beyond the obvious benefits of simplifying maintenance and repair actions, the tool could also be used to direct digital workflows, capture and record maintenance data, and share that data across the user community. In this way, it can be a big enabling technology for the Air Force’s flight line of the future and condition-based/predictive maintenance, helping crews anticipate maintenance needs rather than rely on pre-planned maintenance schedules.

Ashtin Hicks, VAMRAM integrated product team lead, said the system can also be a valuable training tool. Much like a flight simulator is used to train pilots, this system could be used to instruct maintainers on new aircraft or procedures, without the need for physical hardware.

“This could be used to get technicians trained and operational faster and with more accuracy from the first day,” said Hicks.

In fact, the team said the potential uses for this technology are far-reaching. But to move the technology from prototype to product requires collaboration. Their mission is to put together a roadmap for acquiring the expertise, partnerships, data, and ideas that can turn concept into reality.

“We’re trying to teach the Air Force about this technology and how to integrate it and produce content,” said Eisensmith. “As AFRL, we are trying to achieve not the final product, but the knowledge and data management that goes behind it.”

He explained that through a 2018 AFWERX Challenge, the VAMRAM team ran a tech sprint in conjunction with the Wright Brothers Institute. The idea of creating an Innovator’s Guide to advance collaboration and expedite development arose out of that event. This guide is intended to walk end-users through the process of identifying a problem that this technology could potentially solve, working through a possible solution, testing that solution, and evaluating the efficacy of the solution. The VAMRAM team sees the Innovator’s Guide as a tool to enable users to gather and “crowdsource” data that will benefit the entire technical community.

“Putting this guide out there with a process to work through helps turn the Air Force into our laboratory and allows for that crowdsourcing mentality to happen,” said Eisensmith.

Additionally, the team is conducting product development case studies, with the end goal of developing and documenting a tailored acquisition process. The bicycle tire change project is one such case study through which the team hopes to test out a tailored development approach, bringing in contracted partners to develop software and work hand-in-hand with the AFRL team. “We’re working with a company that is willing to learn with us,” Hicks said of the application developers.

Moving toward technology sharing and tailored acquisition is important to the Air Force because it eliminates needless redundancy, expedites development, and saves time and money. In other words, it helps turn a good idea into a viable product faster and less expensively.

“Think of it like Wikipedia,” said Eisensmith. “We’re letting users produce their own content, publish their own lessons learned, identify their own gaps. We want it to be crowdsourced. We want everyone to be able to contribute to that.”

He notes that developing a product for a task even as simple as the bike wheel change involves many variables, including hardware acquisition, data access, hardware and software support, and cybersecurity, to name only a few. It’s a daunting task, and precisely why rethinking basic acquisition is so important.

“Acquisition is challenging,” said VAMRAM team co-lead Dr. Pam Kobryn. “How do you tell the developer what your requirements are, how do you test, how do you determine what it is going to cost, how do you evaluate proposals? That’s what we’re bringing in.”

Now that the team has completed the technical effort behind the mixed reality bicycle tire change tool, their next step is to conduct an effectiveness study and capture lessons learned, all of which will be incorporated into the Innovator’s Guide. They will also continue to raise awareness and recruit partners through events such as the Air Vehicles Technology Symposium.

“We want to find interested parties that want to collaborate with us,” said Eisensmith. “It is our hope to bring more people in to be a part of the Innovator’s Guide, exercise it, publish case studies, and provide feedback. Our technology partners are our best resource.”

Dr. Michael Gregg (left), director of the Air Force Research Laboratory Aerospace Systems Directorate, uses a mixed-reality application to change the tire on a Wright Cycle Company bicycle as Capt. David Eisensmith guides the process. The demonstration, on display at the 2019 Air Vehicles Technology Symposium, is part of the Virtual, Augmented, and Mixed Reality for Aircraft Maintenance team’s effort to show the technology’s potential for future aircraft maintenance. (U.S. Air Force Photo)