Open source high-performance computing more accessible to agencies

A new generation of open source solutions for high-performance computing are helping federal agencies tackle complex data and modeling challenges, such as climate simulations and fraud detection, according to a leading open source software expert.

High-performance computing (HPC) solutions, together with available cloud services, are allowing government agencies to not only generate high-resolution models, but also have a scalable, sustainable and manageable way to store the research, says Brent Schroeder, chief technology officer at SUSE, the world’s largest independent open source software company.

“Thinking about the huge amount of data that agencies are flooded with in nearly every program, it’s not humanly possible to analyze this information by hand, or even with traditional computerized checks. These problems are particularly well suited to HPC and machine learning,” he says.

Schroeder discusses trends in HPC technology, open source software and why agency leaders should consider integrating modern HPC solutions into their operations during a new FedScoop podcast underwritten by SUSE Federal:

How HPC has changed since the days of supercomputers

“Today [HPC systems] can be scaled to any size from departmental and developmental systems, to very large parallel clusters,” explains Schroeder. “There’s a huge diversity in computing technology. Containerization is helping with manageability and portability, and now machine learning is blending the HPC by analyzing interim results during long running simulations, to avoid wasting computing resources if a simulation is flawed.”

Schroeder describes how technological advances have improved HPC’s capabilities, allowing projects to leverage a range of techniques including simulation programs; automation that uses artificial intelligence; machine learning applications that look for patterns; and large-scale analytics.

Trends in technology that open the door for HPC projects

HPC modeling and simulation software is evolving rapidly in three key areas, says Schroeder: the HPC platform, containerization and workloads in the cloud.

At the platform level, there’s a huge amount of diversity in CPU architectures with improved processors and accelerators such as GPUs and FPGAs.

“The rate of change in HPC is higher than ever before, and it is accelerating. This makes it so much more important and vital to navigate and stay current with the change that is happening,” he says.

Additionally, containerizing HPC workloads will increase the likelihood that scientific results can be reproduced by other researchers and reproduced after the original HPC cluster has been decommissioned. And with workloads moving to the cloud, the capacity is more flexible to support more researchers and accelerate results.

How agency leaders can take greater advantage of HPC workloads

Schroeder shares three recommendations for leaders to consider if they want to implement HPC projects.

First, to ensure flexibility, it’s important to review what the agency has done in the past and to be open to deploying new technologies so that the program can adapt to the rapidly changing technology landscape.

Secondly, he encourages leaders to consider a technology review as an opportunity to implement technologies that are available today but weren’t during previous reviews for the HPC stack.

And finally, leaders should assess if the project needs to be done on-premise.

“With the advancements in technology, it’s a great opportunity to take advantage of the ability to deploy where best fits the requirements of the problem and really optimizes both the delivery and the cost of the solutions,” Schroeder says.

Brent Schroeder is responsible for shaping SUSE’s technology and portfolio strategy to support emerging use cases in areas such as hybrid cloud, high-performance computing, IoT and AI/ML. Brent brings 30 years of technology innovation and development experience, having led software strategy development at Dell, in the office of the CTO, and before that, serving in various management and engineering roles with NCR, Compaq and HP. 

Listen to the podcast for the full conversation on open source solutions for high-performance computing. You can hear more coverage of “IT Modernization in Government” on our FedScoop radio channels on Apple Podcasts, Spotify, Google Play, Stitcher and TuneIn.

This podcast was produced by FedScoop and underwritten by SUSE Federal.