JAIC creates new position for ‘AI assurance’

The Pentagon’s Joint Artificial Intelligence Center is creating a new position to lead “AI assurance,” overseeing AI ethics policy and test and evaluation.

Jane Pinelis will take on the role in a part-time capacity while splitting time as the chief AI engineer at Johns Hopkins University’s Applied Physics Laboratory.

The JAIC’s new AI assurance role will incorporate both quantitative testing and verification of AI systems but also includes work to ensure that those using the systems have confidence in the algorithms.

“This enables us to tailor assurance arguments to the specific audience of interest: a warfighter needs a different set of evidence to trust a system than an acquisition official or an international ally,” Pinelis said in a statement to FedScoop. “It also enables us to go beyond formal test methods and also use evidence-based arguments to establish trustworthiness that can incorporate test metrics but also formal mathematical reasoning, as well as qualitative logical arguments.”

Pinelis was an early leader at the JAIC, joining as its first chief of test and evaluation in January 2020. Before announcing her departure from the role in December, she spoke frequently about the importance of testing AI to ensure it measures up to safety and reliability needs to gain the trust of operators.

As head of AI assurance, she will report directly to Lt. Gen. Michael Groen, the JAIC’s director, and oversee the new heads of test and evaluation and responsible AI, once both positions are hired.

“One of the goals of combining [test and evaluation] and [responsible AI] is to ensure the operationalization of the Responsible AI principles through a robust assurance process,” JAIC spokeswoman Sarah Flaherty said in a statement.

The JAIC has been looking for a new chief of responsible AI since October following the departure of Alka Patel. Until then, the JAIC’s policy lead, Sunmin Kim, has stepped in on an acting basis.

The restructure comes as JAIC’s oversight is also changing. The office currently reports directly to the deputy secretary of defense, but there will soon be a new chief digital and AI officer within the DOD to oversee its activities, along with the chief data officer.