MITRE researched air traffic language AI tool for FAA, documents show

MITRE, a public interest research nonprofit that receives federal funding, proposed a system for transcribing and studying conversations between pilots and air traffic controllers, according to documents obtained by FedScoop through a public records request. 

A presentation dated August 2023 and titled “Advanced Capabilities for Capturing Controller-Pilot dialogue” shows that MITRE engaged in a serious effort to study how natural language processing could be used to help the Federal Aviation Administration, and, in particular, to help with “understanding the safety-related and routine operations of the National Airspace System.”

MITRE, which supported the project through its Center for Advanced Aviation System Development, told FedScoop that the prototype is currently being transitioned to the FAA for “potential operational implementation.” Otherwise, it’s not clear what the current status of the tool is, as the agency’s artificial intelligence use case inventory was last updated in July 2023, according to a DOT page. The FAA did not respond to a request for comment and instead directed FedScoop to the Department of Transportation. 

“Communications between pilots and air traffic controllers are a crucial source of information and context for operations across the national airspace,” Greg Tennille, MITRE’s managing director for transportation safety, said in a statement to FedScoop in response to questions about the documents. “Collecting voice data accurately, efficiently and effectively can provide important insights into the national airspace and unearth trends and potential hazards.” 

The August 2023 presentation describes several ways that natural language processing, a type of AI meant to focus on interpreting and understanding speech and text, could be fine-tuned to understand conversations between air traffic controllers and pilots. The project reported on the performance of different strategies and models in terms of accuracy and provided recommendations. At the end, it also describes a brief exploration of how ChatGPT might be able to help with comprehension of Air Traffic Control sub-dialogues, noting that “the results were surprisingly good.”  

The presentation reveals how the often-overwhelmed aviation agency might try to take advantage of artificial intelligence and comes as the Biden administration continues to push federal agencies to look for ways to deploy the technology. 

At the same time, it also outlines potential interest in ChatGPT. While the Department of Transportation said it doesn’t have a relationship with OpenAI, other documents show that officials within the agency are interested in generative AI.  

The Department of Transportation and ChatGPT

The reference to ChatGPT in the project, though it appears to be provisional and not a core part of the research, is more evidence of how the Department of Transportation might use generative AI tools in the future. FedScoop previously reported, for example, that the DOT’s Pipeline and Hazardous Materials Safety Administration had disclosed a use case — described as “planned” and “not in production” — involving “ChatGPT to support the rulemaking processes.”

PHMSA, which said it’s continuing to study the short- and long-term risks and benefits associated with generative AI, has said it does not plan on using the technology for rulemaking. The agency also said that it has an agreement with Incentive Technology Group, worth about several hundreds of thousands of dollars, to explore generative AI pilots. 

PHMSA said that the project did not involve ChatGPT, but instead involved “Azure OpenAI Generative LLM version 3.5.” (OpenAI explains on its website that GPT-3.5 models can be used to understand and generate natural language or code, but PHMSA did not explain whether the reference to ChatGPT in the AI use case disclosure was a mistake or a distinct project from its work with Incentive Technology Group.)

Notably, while other agencies are beginning to develop policies for generative AI, the Department of Transportation has not responded to questions from FedScoop about what policies or guidance it might have surrounding the technology. 

Emails obtained by FedScoop through public records requests show that the Chief Data Officer Dan Morgan had on-hand a “generative AI” guidance document attributed to the government of New Zealand. An email last summer to the Department of Transportation’s AI Task Force from Matt Cuddy, operations research analyst at the DOT’s Volpe National Transportation Systems Center, shows that the agency had made large language models a topic of interest.

A publicly available document from 2019 said that through the task force, the DOT had made transportation-related AI “an agency research & development priority.” 

Last year, FedScoop reported that the Department of Transportation had disclosed the use of ChatGPT for code-writing assistance in its inventory, but then removed the entry and said it was made in error. The department has not responded to questions about how that error actually occurred. Emails obtained by FedScoop show that the incident attracted attention from Conrad Stosz, the artificial intelligence director in the Office of the Federal Chief Information Officer. 

In regard to this story, the Department of Transportation told FedScoop again that the FAA ChatGPT entry was made in error and that the “FAA does not use Chat GPT in any of its systems, including air traffic systems.” It also said that the use case was unrelated to the MITRE FAA project.