MWA-data-reduction
This project has been used for processing and analysis of data from the precursor and prototype instruments of the low-frequency Square Kilometre Array (SKA-Low) telescope. Primarily, the Murchison Widefield Array (MWA) and the two SKA-Low prototype stations, the Engineering Development Array (EDA2) and Aperture Array Verification System 2 (AAVS2). The typical applications included formation of all-sky images, i.e. horizon to horizon images of the entire visible hemisphere to look for radio-transients in 2-second all-sky images from these stations [1]. Recently the Nimbus virtual machines have been used by Chris Lee during his ICRAR summer studentship (2020/2021), later 3rd year Physics Project and currently Curtin Summer studentship (2021/2022) to verify pulsar detections with the SKA-Low prototype stations and measure mean flux density of selected pulsars (Figure1.pdf and Figure2.pdf) [2]. I have also used Nimbus instances to calculate sensitivity of the SKA-Low prototype stations (EDA2 and AAVS2) over entire sky and over full range of Local Sidereal Times (LSTs), and the resulting data products have been used to develop software, create and publish a sensitivity database, and a publicly available web-interface [3]. Further, the Nimbus instances were also used for processing of the MWA to verify its polarimetric sensitivity (sensitivity in Stokes I,Q,U and V polarisations) and demonstrate a general and accurate method for calculations of polarimetric sensitivity of low frequency arrays [4]. Finally, the project has also been used to look for signals from known repeating and non-repeating Fast Radio Bursts (FRBs) in the MWA data [5]. This work resulted in several publications, which will be listed in the later sections of this report
Area of science
Astronomy
Systems used
Nimbus
Applications used
Custom develop bash and python scripts. Standard software for data reduction and imaging (mwa-reduce, WSCLEAN, MWA 2016 beam model etc.). CASA and MIRIAD software packagesThe Challenge
The main projects for which the Nimbus instances have been used for:
– Formation of all-sky images using data from SKA-Low prototype stations (EDA2 and AAVS2) and searches for radio-transients in these images [1]
– Verification and measurements of mean flux densities of selected pulsars using SKA-Low prototype stations (EDA2 and AAVS2) – continuing student project [2]
– Simulations of the SKA-Low station sensitivity to develop software, create a sensitivity database, and a publicly available web-interface [3].
– Processing of the MWA data to verify sensitivity of a low-frequency polarimetric radio interferometer [4]
– MWA data analysis and searches for dispersed radio transients, such as Fast Radio Bursts (FRBs) or low radio-frequency counterparts of short Gamma-ray Bursts (GRBs) [5]
– Processing of data from drift scan observations with the SKA-Low prototype stations and comparison with the predictions of the simulations.
– Simulations of the SKA-Low station beam, simulations of total power detected by the stations during drift scan observations for comparisons with the real-data
– Simulations of the MWA primary beam.
The Solution
These challenges have been solved by dedicated software implemented in python or C++ and executed on Nimbus instances.
The main advantage of using Nimbus instances is that processes can be executed without any time limits, which is the case on all the other supercomputers available at Pawsey (Garrawarla, Magnus and earlier Galaxy).
The lack of time limits makes the required data processing much simpler and more flexible than on other HPC systems
The Outcome
The nimbus instances offer a flexible environment where programs and scripts can be executed in their original form (without necessity of “wrapping” them in special scripts as required on other HPC machines such as Garrawarla, Zeus, Galaxy etc) and they can be left running for long time without interruptions and time limits (as mentioned earlier). Therefore, the Nimbus instances are ideal for solving small and medium scale problems like the ones required by my projects listed above.
The main outcomes are publications listed in the next window.
List of Publications
1] Sokolowski et al., “A Southern-Hemisphere all-sky radio transient monitor for SKA-Low prototype stations”, Accepted for publication in PASA (April 2021), https://ui.adsabs.harvard.edu/abs/2021arXiv210402940S/abstract
[2] Lee et al., in preparation
[3] Sokolowski et al., “What is the SKA-Low Sensitivity for Your Favourite Radio Source ?”, accepted for publication in PASA
[4] Sutinjo et al., “System equivalent flux density of a low-frequency polarimetric phased array interferometer”, submitted to Astronomy & Astrophysics
[5] Tian et al., “High time resolution search for prompt radio emission from the long GRB 210419A with the MurchisonWidefield Array”, submitted to PASA
