Pawsey Capital Refresh Project
In 2018 the Australian Government awarded $70 million to upgrade Pawsey’s supercomputing infrastructure, on top of the $80 million granted in 2009 to establish a petascale supercomputing facility.
The Pawsey upgrade, as a major part of the national HPC infrastructure, is ensuring Australia continues to enable computationally-intense research.
Pawsey capital refresh is a complex upgrade will be a staged process. Some ancillary systems, including storage and network infrastructure, have been procured prior to the main system.
The delivery of the main supercomputing is expected to occur in two-phases.
The first phase will provide researchers with a system that is at least equivalent in capacity to what they are currently using, with the latest generation of processors and increased memory per node.
During this phase, researchers with an active allocation on Magnus will transition to the new system.
Phase one is due to be commissioned in mid-2021.
The second phase is expected to be in production by mid-2022. It will provide an exponential expansion in capacity and state-of-the-art technology.
You can see all the parts that make the Capital Refresh Project on the infographics displayed at the bottom of this page or they can be download here.
News Stories
To read the stories published related to the project milestones referred to the information below:
- Pawsey Capital Refresh Boosts Cloud Infrastructure 21/11/2019
- Tender released for Australia’s new research supercomputer 14/11/2019
- Three times more storage and performance for SKA pathfinders 11/11/2019
- Pawsey Capital Refresh – Reference Groups Established 5/04/2019
- New funding to accelerate science and innovation 28/04/2018
Pawsey is committed to engage with its diverse stakeholders and keep it update regarding the procurement. Some of the channels the Centre has established to achieve this are the Pawsey user forums, Capital Refresh Update for potential vendors, Pawsey newsletters and more recently our podcasts.
You can access the outcomes from the Pawsey user forums via this link, read the Centre’s newsletters on our website.
Podcasts
You can listen to the Capital Refresh Podcast from the list below
- Episode 1: What is the Pawsey Capital Refresh?
- Episode 2: Capital Refresh Status Update
- Episode 3: HTC Cloud Procurement
Monthly update
December 2019
The project reached a significant milestone on 14th November with the release of the Pawsey Supercomputer System (PSS) tender on AusTender. The PSS will replace the functionality of both the:
- Magnus supercomputer — a petaflop-performance, Cray XC40, commissioned in 2014, that provides a general, research supercomputing service
- Galaxy supercomputer — a smaller Cray XC30, commissioned in 2013, which provides operational support for two Square Kilometre Array (SKA) precursor radio telescopes: the Australian Square Kilometre Array Pathfinder (ASKAP) and the Murchison Widefield Array (MWA), each of which are located in the remote mid-west of Western Australia,
with a single general-purpose supercomputer, the capability of which can be effectively and dynamically partitioned into special-purpose environments under software control.
In scope for this procurement is the PSS hardware, software and support services comprising:
- Servers (for computation, visualisation, management, data movement, file serving, and login functions)
- The high-performance interconnect and a dedicated, separate system management fabric for the out-of-band management of the system nodes and network switches
- The supercomputer storage and filesystem environment, e.g., the scratch, intensive I/O scratch, home, and application (/pawsey) filesystems
- Software, including all necessary licences for the operating system, the system scheduler, a complete and fit-for-purpose software stack, a comprehensive application development environment, and filesystem software/licences, together with relevant management and monitoring tools
- Installation
- Ongoing support and maintenance.
A number of the supporting procurements have also progressed in November:
- Deliveries for the Cloud Compute Upgrade have commenced arriving at Pawsey. Installation is expected to occur in December 2019 and January 2020.
- The MWA compute cluster evaluation has progressed and is expected to be finalised in December.
- A paper recommending user requirements for the Long-term storage has been endorsed by the User Reference Group. An RFI document is being prepared and will go through peer review in December 2019.
- Scoping of the building works required to support the Pawsey Supercomputer System continued in November.
- Development of the draft RFQ for the ASKAP ingest nodes commenced in November and will continue through December 2019.
- High-speed storage for ASKAP and MWA went live in October
The Change Management Plan has been developed and will be submitted to the Change Management Reference Group in December. Work has also commenced to identify the training and engagement that will be required to support the user transition from Nimbus to the new Cloud Compute System.
Infographic
Find below an infographic regarding the current status of the Project (last updated on 21/11/2019). They can also be download here
Pawsey Capital Refresh Status
Tape library expansion Additional tape storage has been procured to expand the existing tape libraries from 50 to 63 Petabytes in each library. Long term storage Server and hard disk storage and networking will be refreshed, along with the tape library’s total maximum storage capacity. Both ingest and egress data transfer services will be upgraded, improving upload and download times. |
Pawsey is partnering with Dell EMC to expand its current cloud system with 5x more memory and 25x more storage to form a new cutting-edge flexible compute system. This expansion aims to better service emerging research areas and communities who benefit more from a high throughput compute. |
|
Astronomy high speed storage: 3x more storage and performance. The existing Astro filesystem was expanded to service the MWA community. Powered by HPE, it has been upgraded to 2.7 PB of usable space and capable of reading/writing at 30 GB/s. The New buffer filesystem, a dedicated resource for ASKAP researchers, provides 3.7 PB of usable space and is capable of reading/writing at 40 GB/s. It is manufactured by Dell. High speed storage filesystems Designed to deal with thousands of users accessing them at the same time, high speed filesystems will be procured to increase speed and storage capability to general purpose science. |
The MWA cluster will be a resource better tuned to MWA’s needs. It will be procured ahead of the Main Supercomputer, allowing ASKAP to use the full CPU partition of Galaxy. When available, the new MWA cluster will provide: |
|
The high-speed interconnect ties all the pieces of the Pawsey ‘puzzle’ together. When procured, all parts will sit on the same fabric as first-class citizens, allowing Pawsey researchers to run their workflows quicker. |
The remote visualisation capability will be procured as part of the main supercomputer. When the new capabilities become available, researchers will be able to visualise their science in real-time, while being processed. This new capability will allow researchers to steer their visualisation while the data is processing and fine-tuned to the desired outcome. |
|
Delivered in two phases, the Pawsey Supercomputer phase 1 will be delivered by mid-2021, it will provide researchers with a system that is at least equivalent in capacity to what they are currently using. Phase 2 is expected to be in production by mid-2022; it will provide an exponential expansion in capacity and the latest state-of-the-art technology. |
Pawsey Data Workflow