How virtual reality could change the way NASA does science

Doug Morton is an earth scientist at NASA’s Goddard Space Flight Center and he knows what it’s like to go out into the field (generally a rainforest) and gather information. He has all kinds of tools to collect data — LIDAR sensors, cameras and more. But at the end of the day, Morton can only gather so much, and once back at Goddard, he must rely on the observations he collected to continue his science.

“I often think, ‘If only I could go back and do blank,’” Morton said. Simply looking back at photographs has obvious limitations.

But what if there were a way — short of getting back on a flight to the Amazon — that Morton could revisit the site of his study?

Enter: LandscapesVR, one of six augmented reality (AR) and virtual realty (VR) pilot programs that were run out of a lab at Goddard in fiscal 2017. The aim is to create a proof-of-concept app where scientists can “revisit” and explore locations like remote rainforests or lava tubes.

The prospect of being able to return to his gathered data in a way that is more intuitive and rich than scrolling through a camera roll is what got Morton interested in the project. That measurement he forgot to take? Now he can “go back” and take it.

“I’m excited to do something I’ve never done before,” he said. And it doesn’t hurt, he quipped, that in virtual fieldwork there are no insects.

Tom Grubb, who leads these AR and VR pilot projects, is excited to see Morton excited.

An avid gamer in addition to a NASA computer programmer, Grubb has been following AR and VR technology for years. “I personally could immediately see how this would be useful for NASA,” he said. So in the summer of 2016 he applied for an internal research grant and received “quite a bit” of funding to explore the possibilities of the virtual world.

Working with stakeholders across Goddard, Grubb settled on six pilot projects that can broadly be divided into two categories: science and engineering

The LandscapesVR project is among the former. The latter, meanwhile, includes projects where users can immerse in VR to practice and measure the assembly of small satellites, one where users can test how hardware responds to the conditions of space and another AR project where augmented reality is used to help mission controllers pay attention to important information while refueling and servicing satellites already in orbit.

After a year of early development, these three engineering pilots have been combined into one project (a virtual and augmented reality engineering toolkit, if you will) for fiscal 2018. The team, which generally consists of five or six people working part time, will also continue working on LandscapesVR and a project called Space Weather, where VR is used to visualize Earth’s magnetosphere.

Grubb has worked at NASA for 29 years, but “last year was the best year of my career,” he said. Being able to work in this new technology has him energized.

But it’s also true that there are still limitations. After just a few minutes in virtual reality these became clear — the headset was heavy, the movements awkward and dizzying. Can scientists and engineers spend significant time in this environment? Is it really so user-friendly?

Grubb admitted that the technology has a way to go before prime time. “It’s early days in some senses,” he said.

But Billy Gallagher, a robotic operator and engineer who works in satellite servicing and is building out AR applications as a side job, argued that it is in NASA’s DNA to try things early. “NASA’s always trying to do things we can’t do yet,” he said. “Since these technologies are just coming out, no one really knows what they’re useful for.” The point of this pilot lab? To come up with, and explore, ways the technologies could be useful.

According to Grubb and Gallagher, the value of AR and VR in engineering seems clear. In the sciences, meanwhile, the exact application is a bit more of an open question.

And there’s another benefit to NASA involvement in AR and VR beyond exploration, Grubb said. A lot of innovation in the industry is currently driven by gaming — a discipline that has different needs from those of NASA scientists. To the extent that NASA can be involved early, he suggested, it can help mold what the technology becomes. So that’s precisely what Grubb and Gallagher are doing.

“Absolutely I think it’s going to change how we do work here at NASA,” Grubb said. “VR and AR has a really profound potential to change all of our lives.”