The FBI is looking to the cloud to stop the next terrorist attack

The FBI is in the midst of a data crisis.

Of course, there’s no shortage of data for the bureau to process. The problem, as Christine Halvorsen explained Tuesday, is that criminal investigations generate more data than the bureau can handle manually.

“The solution is the adoption of cloud technology,” said Halvorsen, deputy assistant director for the FBI’s Counterterrorism Division.

Speaking at Amazon Web Services’ re:Invent conference in Las Vegas, Halvorsen gave a benchmark for the proliferation data in terrorism investigations. After the Boston Marathon bombings in 2013, the FBI collected more than 50 terabytes of information on Tsarnaev brothers who conducted the attacks. Just four years later, investigating the mass shooting in Las Vegas generated more than a petabyte of data, or 20 times as much. That’s the new norm, Halvorsen said.

“To get through that data is a challenge for us,” she said. “We threw resources at it. The answer for the FBI was to throw bodies at it. But the investigators and analysts were completely overwhelmed by the volume of data collected in that short amount of time for us to get through it quickly. They were using boards to photos up, cardboard boxes.”

Had the FBI been in the cloud at that point, though, it would’ve been a different story, Halvorsen believes, especially with developments in artificial intelligence and machine learning.

After the 2017 Las Vegas massacre,  she said, “we had agents and analysts, eight per shift, working 24/7 for three weeks going through the video footage of everywhere Stephen Paddock was the month leading up to him coming and doing the shooting.”

Artificial intelligence and machine learning would’ve totally changed that dynamic. “If we had loaded that up into the cloud, the estimate is it would’ve taken us a day using Amazon Rekognition to recognize where he was in the videos. That’s all we were trying to do: narrow down where in the videos he was and who he was meeting with to make sure there wasn’t anybody else part of the conspiracy.”

And that frees agents and analysts to focus on the next threat. “And think about it too, you take that manpower and you put it on something like that — the other cases we have, they don’t stop going,” Halvorsen said. “The subjects don’t just sit back and say ‘The FBI is busy over there, we’re going to stop doing bad things while they’re busy.’ The threat keeps going.”

The cloud culture shift

The FBI since then has made progress in moving to the cloud. Halvorsen said the Counterterrorism Division, in particular, has shut down its data center and moved its data and services to AWS. That’s resulted in a 98 percent reduction in manual work for analysts and 70 percent cost reductions.

But it hasn’t come without growing pains culturally. “You can’t continue to use the same processes you used before,” Halvorsen explained. “You can’t lift and shift.”

So her division had to restructure its teams to reflect the new cloud paradigm. “This was key to our success,” she said. “We’ve embedded data scientists with engineers, with agents, with analysts, sitting together now in the Counterterrorism Division to go after the threat.” And it took some time to get it right — the division failed miserably at its first attempt to shift its makeup. “The good news is we failed fast. And by failing fast and correcting that, we’re actually stronger for it today.”

Now, everybody is a builder, Halvorsen explained. “Now we have teams working in an integrated fusion center-type concept.” The bureau has embedded data scientists at command centers to assist operators as they respond in real-time to attacks. It also is offering its personnel the opportunity to training in data science through an education training program. Halvorsen said 64 members of her division have jumped at the opportunity.

This focus on pairing data science to the mission is” working very well,” she said, pointing to the recent threats of pipe bombs mailed to prominent political officials. “Which, if you noticed, we solved that pretty quickly.”

But despite the shift to the cloud, major challenges persist for the FBI. While it has improved its ability to grasp its own data because of cloud, sharing with others remains something that lets terrorists slip through the cracks, Halvorsen said.

The Tsarnaev brothers and Paddock “were the unknowns of the knowns … someone had them in their holdings. They were in the data,” she said. “But we didn’t put the puzzle together quick enough to stop these tragic events from happening. When we talk about the puzzle pieces, the pieces of data…its not just in the FBI’s holdings. Think bigger than that, Halvorsen said, referencing the intelligence community, state and local authorities and the private sector. “It’s all there, but we don’t have the ability to put the pieces of the puzzle together right now because it’s all being held separately.”

Her hope is that the cloud and emerging technologies like AI will help destroy those silos — though, like the FBI’s own modernization, the transformation is as much about culture as it is adopting new tech.

Until then — until the FBI can end the constant threat of terrorism in the U.S. — Halvorsen said she is “failing.”

“Please help me not fail.”