Using Containers to Accelerate HPC

If you will be at SC22 in Dallas this year, don’t forget to join Pawsey’s Alexis Espinosa in this tutorial, along with colleagues Shane Canon (Lawrence Berkeley National Laboratory), Eduardo Arango Gutierrez (Red Hat Inc), Sameer Shende (University of Oregon, ParaTools Inc) and Andrew Younge (Sandia National Laboratories)

Within just the past few years, the use of containers has revolutionized the way in which industries and enterprises have developed and deployed computational software and distributed systems. The containerization model has gained traction within the HPC community as well with the promise of improved reliability, reproducibility, portability, and levels of customization that were not previously possible on supercomputers. This adoption has been enabled by a number of HPC Container runtimes that have emerged including Singularity, Shifter, Enroot, Charliecloud, and others.

This hands-on tutorial looks to train users on the usability of containers on HPC resources. We will provide a detailed background on Linux containers, along with introductory hands-on experience building a container image, sharing the container and running it on a HPC cluster. Furthermore, the tutorial will provide more advanced information on how to run MPI-based and GPU-enabled HPC applications, how to optimize I/O intensive workflows, and how to setup GUI enabled interactive sessions. Cutting-edge examples will include machine learning and bioinformatics. Users will leave the tutorial with a solid foundational understanding of how to utilize containers on HPC resources using Podman, Shifter, and Singularity, and in-depth knowledge to deploy custom containers on their own resources.

More information about the tutorial can be found here: https://sc22.supercomputing.org/presentation/?id=tut111&sess=sess201