Alessandro Ridolfi's home page






SPLICE_DADA on GitHub

Latest version: 1.0.2 (22Jul2019)

git clone https://github.com/alex88ridolfi/splice_dada.git



SPLICE_DADA




Requirements:
SPLICE_DADA allows you to join several .dada files together along the frequency direction, i.e. to join adjacent subbands.


A few backends and/or experiments acquire data in baseband mode and store them in the DADA format (file extension ".dada"). This is the most powerful and flexible data format possible, as it represents the "raw" complex voltages directly measured from the two receiver's feeds. Often times, mostly for computational reasons, the whole observing bandwidth is split into several subbands, which are processed independently and stored onto separate files by different machines. As a result, the observation will be composed by several groups of files, each group being relative to a separate subband.


SPLICE_DADA allows you to stitch the subbands together, so as to produce a single file which embodies the whole observing bandwidth. This is very useful especially when one wants to produce a coherently-dedispersed search-mode data file from the original baseband DADA files.





DOCUMENTATION


1) Installation

No actual installation procedure is needed to use SPLICE_DADA itself. You just need to have Python 2.7 and Numpy installed. Other than that, just:

  1. Download the code from GitHub using the "git clone" command above
    git clone https://github.com/alex88ridolfi/splice_dada.git

  2. Move to the newly dowloaded directory
    cd splice_dada

  3. Run the code with:
    ./splice_dada

Obviously, to have SPLICE_DADA at hand from any directory, you can add the folder where the code is located to your PATH environment variable.



2) Basic usage

To successfully use SPLICE_DADA, you need:

  1. DADA files from N adjacent subbands of the same observation.

Let's imagine that we have observed the bright pulsar B0329+54 for 30 seconds in the P band (about 350 MHz) in baseband mode, with a total observing bandwidth of 80 MHz, split into 5 separate subbands. The resulting files with which we will have to deal with will be:

  • ./subband1/2019-04-16-20:32:32_0000000000000000.000000.dada
  • ./subband1/2019-04-16-20:32:32_0000000640000000.000000.dada
  • ./subband1/2019-04-16-20:32:32_0000001280000000.000000.dada
  • ./subband2/2019-04-16-20:32:32_0000000000000000.000000.dada
  • ./subband2/2019-04-16-20:32:32_0000000640000000.000000.dada
  • ./subband2/2019-04-16-20:32:32_0000001280000000.000000.dada
  • ./subband3/2019-04-16-20:32:32_0000000000000000.000000.dada
  • ./subband3/2019-04-16-20:32:32_0000000640000000.000000.dada
  • ./subband3/2019-04-16-20:32:32_0000001280000000.000000.dada
  • ./subband4/2019-04-16-20:32:32_0000000000000000.000000.dada
  • ./subband4/2019-04-16-20:32:32_0000000640000000.000000.dada
  • ./subband4/2019-04-16-20:32:32_0000001280000000.000000.dada
  • ./subband5/2019-04-16-20:32:32_0000000000000000.000000.dada
  • ./subband5/2019-04-16-20:32:32_0000000640000000.000000.dada
  • ./subband5/2019-04-16-20:32:32_0000001280000000.000000.dada

where each subband has its own subdirectory, and each .dada file, for each subband, contains 10 seconds of data.

In order to merge the subbands together, we use SPLICE_DADA with the following command:

splice_dada -dadafiles "subband*/*.dada" -o spliced_B0329+54


As output, the program returns one merged .dada file for each file found in each subband.
The suffix "subint_N" in the ouput file names indicates that file number N was merge across all subbands. In this example:
  • Output file "spliced_B0329+54_subint_0000.dada" corresponds to file "2019-04-16-20:32:32_0000000000000000.000000.dada" merged across all 5 subbands
  • Output file "spliced_B0329+54_subint_0001.dada" corresponds to file "2019-04-16-20:32:32_0000000640000000.000000.dada" merged across all 5 subbands
  • Output file "spliced_B0329+54_subint_0002.dada" corresponds to file "2019-04-16-20:32:32_0000001280000000.000000.dada" merged across all 5 subbands

NB: Even if your files are not organized in subdirectories, each cointaining a different subband, SPLICE_DADA is smart enough to reconstruct and figure out which file belongs to which subband.


Left: folding done on individual subbands / Right: folding done on the spliced .dada file




3) Command line options

You can use a number of options to adapt SPLICE_DADA to specific applications and cases. Here follows a list of all the available options:

  • -dadafiles "<dadafiles*.dada>"
    Merge subbands of these files.

  • [-o "outfilebasename"]
    Specify output file basename.
    Default: "spliced_<original_name>"

  • [-blocksize {N_bytes,xMB,xGB}]
    Specify the size of each block to process at each iteration.
    Default: 100MB

  • [{ -Q | -v | -V}]
    -Q: quiet mode. Do not print anything onto the standard output. Useful to call SPLICE_DADA within scripts or pipelines.
    -v: verbose mode. Print additional useful information.
    -V: very verbose mode. Print a lot of additional information. Useful for debugging purposes.

  • [{-version | --version }]
    Show code version.

  • [{ -h | -help | --help }]
    Show a simple usage help and the list of options.




4) Acknowledgements

Big thanks to Cees Bassa, who shared a very similar code ("joindada.c"), by which "SPLICE_DADA" was directly inspired and thanks to which I understood important details on the DADA file format. Many thanks to Marta Burgay for testing the code and providing excellent feedback and suggestions.





Porto Tramatzu, Sardinia (Italy) Atmospheric CO2