2020 in review: Pentagon leads federal DevSecOps efforts
The Department of Defense accounted for most of the federal movement on DevSecOps in 2020, while civilian agencies generally were just getting started in using the development philosophy that is popular in private industry.
In a few well-publicized projects, DOD took the greatest strides attempting to integrate the work of developers, security experts and operations specialists — the Dev, the Sec and the Ops. Progress in using the DevSecOps philosophy was less noticeable, however, on the civilian side, experts say. Many federal coding teams and contractors have embraced DevOps practices but still tack the “Sec” onto the end of the software development lifecycle.
“Agencies are in various stages of maturity in DevSecOps, sometimes even within the same agency itself,” ATARC founder Tom Suder told FedScoop in October. “Most agencies have at least started the DevSecOps journey with the purchase of stand-alone tools.”
DevSecOps is so highly regarded because it “bakes” security into the software development process and allows for developers to recognize and address vulnerabilities as they work. The most advanced versions of the philosophy allow not only for the security work be integrated, but also for dynamic, technology-assisted assessments of the code. Google, Microsoft, Apple and Facebook this year all said they had begun dynamic analysis of their code.
Although it most federal agencies might not get to that level for awhile, experts say advanced DevSecOps eventually could help them reduce their technical debt. The National Institute of Standards and Technology began exploring possible development of a DevSecOps framework in March that could help agencies close the gap with industry.
Traditional, static analysis of code uses continuous integration, continuous delivery (CI/CD) pipelines to perform automated status checks that tend to report more false positives than the advanced techniques now available, said Richard Bae, director of solutions at ForAllSecure, in an interview. The Pittsburgh-based software company specializes in fuzzing, a type of dynamic analysis that sends a bunch of inputs at target code — executing it hundreds or thousands of times per second — searching for bugs in performance.
Google’s Project Zero revamped its DevOps pipeline to incorporate fuzzing of Chrome, while Microsoft’s Project OneFuzz restructured its codebase to fuzz every endpoint. Other Silicon Valley companies are joining suit, but federal agencies have ground to make up, Bae said.
“If you do good software development, most of our security problems will go away because all of the nagging vulnerabilities that we see in software — a lot of those are attributed to people not using secure coding techniques and things we should be doing,” said Ron Ross, a NIST fellow, in March.
Like many ongoing efforts in federal IT, progress on DevSecOps also could have been affected by the COVID-19 pandemic. Chief information officers and IT teams have had to focus limited resources on more urgent problems, like the shift to telework and remote access security architectures.
At least one agency — the Department of Veterans Affairs — put the accountability for DevSecOps practices in the hands of a specific leader. The department appointed Todd Simpson as its first head of DevSecOps in July.
And ATARC launched a source code repository on GitLab in October to help agencies begin using DevSecOps. A DevSecOps Project Team is creating an automated CI/CD pipeline allowing agency IT personnel to practice source code management.
DOD pockets of DevSecOps
The Pentagon meanwhile has more resources to put toward DevSecOps, with several service branches creating their own coding units and programs this year. But that doesn’t mean DevSecOps is part of DOD’s overall tech culture just yet, Bae said. So far, the progress has been within projects like Kessel Run and Platform One that have mandates to “hyper-modernize,” Bae said.
“Those are pretty isolated, junior,” he added. “There’s still a long way to go to have that be applied DOD-wide.”
The Platform One team, based out of the Air Force, consists of about 180 software developers using DevSecOps practices to develop military tools. The methodology helps limit the chance of adversaries probing their networks, especially during pandemic telework.
Meanwhile the Army launched a software factory to bring “true DevSecOps” to the branch in July, and the National Geospatial-Intelligence Agency has a “relatively advanced” DecvSecOps pipeline as well, Bae said.
Even the newly minted Space Force has a DevSecOps coding unit dubbed the Kobayashi Maru.
DOD officials further announced an Adaptive Acquisition Framework pathway for buying software and securely developing code between government and contractor teams using DevSecOps in October.
Federal watchdogs have shown an interest in monitoring the DOD’s progress on DevSecOps. The department has at least 22 weapons programs using agile software development — where iterative updates are pushed rapidly — but none of them used a DevSecOps methodology, according to an annual Government Accountability Office assessment released in June.
Such programs don’t have time to implement DevSecOps practices because their contract requirements only cover producing features. Instead, DOD has to create specific programs to work down the tech debt for analyzing weapons systems, Bae said. In those cases, vulnerability researchers often have limited knowledge of what they’re evaluating, he said.
The best example of this is Section 1647 of the 2016 National Defense Authorization Act, which provided DOD $200 million to give to weapons system developers to find bugs post-development.
