A flexible, open-source package management tool that can support multiple versions and configurations of software will be adopted as part of Pawsey Supercomputing Research Centre’s configuration of software and hardware for Setonix phase 1.
To enable easy and performant software installations for staff and researchers, Pawsey experts are adopting Spack, a package manager developed by Lawrence Livermore National Laboratory (LLNL) that has become a widely adopted tool.
Spack is simple, non-destructive (in that a new installation does not remove older versions) and has been built with community contributions making it ideally suited for High-Performance Computing, with installation recipes for more than 4000 software packages.
Spack is not tied to a particular language. Researchers can build a software stack in Python or R, link to libraries written in C, C++, or Fortran, easily swap compilers and target specific microarchitectures.
Pawsey team is working closely with Spack original developers at LLNL and HPE Cray Compass group, tuning its configuration on Setonix Phase 1.
Researchers who work with several external software libraries or packages will find a live space where they can work and even collaborate on this new automated package management tool.
Setonix users will be able to install their software packages with a straightforward command-line interface, find solutions to installation problems from others within the community, and be supported by documentation that allows easy onboarding and is open to contributions.
As part of the Setonix development process, Pawsey worked with AMD, the vendor for the new supercomputer processors and accelerators, to confirm that all tools and features in the technologies hosted on Setonix are ready to support diverse Pawsey computational and data processing workloads.
The Centre played an active role during the testing of the AMD Instinct MI100 GPUs and the AMD Milan CPUs, and feedback informed the implementation of newest versions. Our participation has now been featured during AMD stakeholder presentations and product announcements.
To provide a smooth transition from Maali — our current software compiler developed in house — to Spack, specific documentation will become available soon.
More information will be part of the migration training developed for researchers moving from Magnus and Galaxy to Setonix. A new software policy framework will also be released, to provide researchers with a detailed view on software installation services on Setonix.
Please note that now that Pawsey experts are focused on Setonix commissioning, no new software installation requests on Magnus, Galaxy, and Zeus will be processed.
To know more about Spack or get involved, visit Spack on GitHub, and take the tutorial or look at the docs. Join the discussion on Slack, and learn how to contribute packages.
You can also check out the Upcoming Events page for tutorials, workshops and other events.
