Three times more storage and performance for SKA pathfinders

As part of the Pawsey Capital Refresh, the Pawsey Supercomputing Centre is boosting high-speed storage for researchers using CSIRO’s Australian Square Kilometre Array Pathfinder (ASKAP)and the Murchison Widefield Array (MWA) radio telescopes by providing three times as much storage and three times as much performance.

Supercomputing is often associated with raw processing power, the ability to do a lot of calculations very quickly. In addition to supporting such activities, the Pawsey centre is in the unique position of being a key part of an operational radio astronomy observatory, supporting ASKAP and MWA. These instruments gather large volumes of data, which is sent to Pawsey for analysis in real-time, making storage space as important as processing power.

The Galaxy supercomputer, a Cray XC30, is dedicated to supporting these radio telescopes. Originally it had 1.9 PB of high-speed storage for this purpose. Now that ASKAP and MWA are operational, this amount of space can be filled with raw data in roughly a week! Processing the data requires plenty of spare room and must occur while new data are still being recorded, putting a large strain on the disks.

To ensure a smooth operation of these new telescopes, we realised that it would be necessary to significantly upgrade our high-speed storage capacity.

Parallel distributed file systems are required to keeping the compute nodes fed or deal with the tsunami of data coming out of them. High-speed filesystems are also designed to deal with thousands of users accessing them at the same time.

The new high-speed storage was delivered in two parts. The astronomy filesystem, which will be dedicated to MWA, has been upgraded from 1.8 PB to 2.7 PB of usable space and capable of reading/writing at 30 GB/s. The astronomy filesystem is powered by HPE and is made up of three DDN Storage arrays, and uses Lustre, a high-performance parallel filesystem.

This upgrade has been followed by the procurement of a new Lustre filesystem, called Buffer. Buffer will be a dedicated resource for ASKAP operations. Manufactured by Dell, it provides 3.7 PB of usable space and is capable of reading/writing at 40 GB/s.

The technology provided by Dell consists of twenty-four 960 GB solid state disks and six hundred and seventy-two 8 TB mechanical spinning disks.

This upgrade will help ensure the speed and capacity required by astronomers will be available to carry out vital research projects. This will enable them to continue increasing our understanding of the Universe more efficiently.

The new ASKAP filesystem is mounted on the Galaxy Supercomputer and the data ingest nodes. It will be used to stream data from  ASKAP’s 36 antennas located at CSIRO’s Murchison Radio Observatory (MRO) in Mid-West Western Australia. The data ingest nodes have four dedicated 10 Gbit connections to the MRO and are connected to the Pawsey InfiniBand fabric.

The Pawsey Capital Refresh project is supported by the Australian Government through a $70 million grant. Pawsey is also supported by the Australian Government under the National Collaborative Research Infrastructure Strategy (NCRIS) and related programs through the Department of Education. The Centre would also like to acknowledge the support provided by the Western Australian Government and its Partner organisations.

CSIRO acknowledges the Wajarri Yamatji people as the traditional owners of the Murchison Radio-astronomy Observatory site.

Tiles at the Murchison Widefield Array

Tiles at the Murchison Widefield Array