How federal agencies can use agile development to apply open data

Over the past three years, a small, dedicated team of federal innovators has been steadily transforming the way government agencies apply open government data. The Opportunity Project (TOP), housed in the Census Bureau at the U.S. Department of Commerce, has adopted agile development principles to help government, communities, and the technology industry build high-impact, data-driven digital tools and platforms.

TOP facilitates 12- to 14-week technology development cycles that bring together tech teams from industry, academia, nonprofits, and user communities to work with federal data stewards on public challenges in health care, education, transportation, and other areas. This innovative approach has produced more than 70 digital tools since TOP was launched in 2016 and is now being replicated by others in government. Some of the most recent TOP products include:

• A risk assessment tool using satellite data to find areas where poor infrastructure may be vulnerable during natural disasters;
• A model using geospatial and emergency medical services (EMS) data to predict where and when the risk of opioid overdose will be high;
• Online tools that use artificial intelligence (AI) and occupational data to match veterans with job apprenticeships; and
• A website that uses federal spending and audit data to maximize the use of funds for addressing homelessness.

Today, the IBM Center for the Business of Government published our report on TOP and the lessons of its success. The report, “Agile Problem Solving in Government: A Case Study of the Opportunity Project,” describes what has made TOP successful, how TOP can be further developed, and how government agencies can adopt the TOP model. The Center for Open Data Enterprise (CODE) participated in the 2018 TOP technology development cycle and wrote the report as part of our organization’s mission to maximize the value of open government data for the public good.

As the report explains, The Opportunity Project combines three key elements: the power of open government data, public-private collaboration, and high-energy agile approaches to software development. By adopting and adapting TOP’s approach to their own missions and programs, government agencies can apply agile, collaborative, data-driven solutions to a wide range of public problems. Here are some of the elements that TOP has shown to be key to success.

Collaboration based on shared benefits 

The Opportunity Project recruits diverse participants by delivering benefits for all stakeholders. Government agencies benefit from the resources that tech teams contribute, and from the experience of participating in agile, user-focused projects. Companies that contribute their work through tech teams can showcase their capabilities while learning about government data resources. And user advocates participating in TOP help ensure that their needs have priority in government data programs and the resulting platforms and tools.

Organizing projects around common themes

In 2018, The Opportunity Project used a common theme to organize and provide focus for a number of their projects. About half the projects last year were organized through a “geo-cohort” that included experts in geospatial data to tackle a number of related problems using the federal government’s geospatial data assets. This year, TOP is organizing most of its work around two major themes: workforce development and increasing participation in the 2020 U.S. Census.

Commitment to user engagement

From the beginning, TOP has engaged user advocates who can help shape projects to ensure that end-users’ needs are met. Recent user advocates have represented groups as diverse as military veterans and the Choctaw Nation of Oklahoma. The user perspective has influenced many TOP projects and contributed to their success.

Combining virtual and in-person collaboration

For most of the past three years, TOP has functioned almost exclusively through virtual meetings and participation. Federal agency representatives and tech teams have connected by phone and online conferencing, and have met in person only after their work is complete. In 2018, however, TOP supplemented this virtual collaboration with two in-person meetings: a meeting in Puerto Rico to convene participants working on geospatial projects, and a user engagement workshop for projects led by the U.S. Department of the Treasury and the White House Office of Management and Budget. The experience showed that combining virtual and in-person meetings is a powerful model for collaboration, and TOP plans to continue this approach with an initial workshop on July 2 in Chicago.

Showcasing success

Every year, TOP showcases the results of its work with a Demo Day, in which teams present the products they have built. The March 2019 Demo Day launched more than 20 digital tools and platforms produced during the 2018 development cycle for an audience of more than 200 people. The Demo Day combined dynamic speakers, effective presentations, and keynotes from government leaders, all live-streamed and made available online. These annual events encourage further adoption of the TOP methodology, participation in upcoming projects, and potential partnerships to make the products developed through TOP sustainable and scalable.

Over the past three years, TOP has developed a network of government agency partners and a clear, replicable methodology that set the stage for applying its agile approach to data-driven projects across government. Last October, the U.S. Department of Health and Human Services launched the first federal agency effort specifically based on the TOP methodology. The success of this HHS Health Tech Sprint shows that the TOP approach can be adapted by individual agencies to address complex public problems in service of their mission.

The Opportunity Project’s website includes many examples of its work and a detailed toolkit for using its methodology. We hope that our new report will help increase awareness of TOP’s work and encourage more federal agencies to follow this innovative, successful model.

Joel Gurin is president of the Center for Open Data Enterprise (CODE), a Washington-based nonprofit that works to maximize the value of open government data for the public good. Katarina Rebello is CODE’s director of programs.