Science data acquisition

The IMPISH flight-like data acquisition is more autonomous than the debug acquisition. The same tools are composed together in the flight scripts as in the debug scripts, but the flight code is designed to be less interactive.

Prerequesites: set up the IMPISH flight computer using the deployment scripts, and configure a local Python environment for analysis--see here.

Acquiring flight data

There are a couple of different ways to command IMPISH. One way is in an interactive shell. Another is via the remote commanding interface. The remote commanding interface uses packetized commands and sends them to the IMPISH flight system over a radio link (Starlink, I think).

Both systems use the same script files. We will make, or have made, a separate guie on translating Linux script invocations into flight-safe commanding. So, for the purposes of this guide, we will just assume we're using an interactive terminal for data acquisition.

Begin acquisition steps (interactive shell)

All of the scripts here are from the main repository: impisc/scripts.

0. Run stop_science to reset the system to a known state.
   • The science program is automatically stopped by a systemd-networkd hook if the DAQBOX interface goes down, but we stop the program first anyway.
1. Run start_science. This script does a few things
   • Turns on the preamplifiers, DAQBOX, and Bubba bias;
   • Configures the DAQBOX network (on enp1s0);
   • Sets the Bubba bias voltage;
   • Recalibrates the DAQBOX baseline;
   • Starts the nominal-science systemd service.

Checking that things are working

After the start_science script successfully executes, you can check the status of the systemd task:

sudo systemctl status nominal-science

If it's running, you should see no error outputs. You can also check the health data to make sure the power lines are on, and that the bias has been set properly. Finally, you can check the science quicklook Grafana dasboard. It should have some counts coming in once the nominal-science systemd service starts.

If there are any errors in the service initialization, record them and notify someone who can fix the errors.

Parsing the data that comes out

Files should be appearing on the IMPISH data drive in /data/science/. The files are all ~30s of compressed data.

# Check the most recent files, if you want
ls -lth /data/science | head -n10

Each file is a sequence of data records . The format is different from the native DAQBOX format. It is a fixed-size record, but the size depends on the current system configuration. This is because the science data can be optionally rebinned. The rebinning is defined by an environment variable in variables.env.

The files can be read in a similar way to what is described in the taking debug data guide, with a few substitutions:

• Change the with open... to with bz2.open if the files are bzip compressed, and use a different open function if a different compression utility is used in flight. Alternatively, decompress the files before opening them.
• Do not parse the packets using daq_box_api.parse_spectrum_packet. You'll need to parse them using a different class (discussed below).

Turning the packed binary records into a comprehensible data structure

You can each record from the science files into a ctypes Python struct.

import ctypes

# Get this from the `variables.env` file
NUM_BINS = 1000
NUM_DETECTORS = 4
class ScienceRecord(ctypes.LittleEndianStructure):
    _pack_ = 1
    _layout_ = "ms"
    _fields_ = (
        ("timestamp", ctypes.c_uint32),
        ("frame", ctypes.c_uint8),
        ("spectra", NUM_DETECTORS * (NUM_BINS * ctypes.c_uint32)),
    )

The records can then be parsed into a numpy.ndarray, an astropy.Quantity, or an ndcube object (if you wanna figure that out).

Struct details

Each record from the science control contains the following:

• A UNIX timestamp (32B unsigned)
• The DAQBOX frame number (0 to 31, 1B unsigned)
• The 4x [N x 32B unsigned], where N corresponds to the number of bins defined in variables.env.

Hopefully we can leave N = 1000 for the whole flight, but if we need to reduce the number of bins to save disk space, so be it. The 4x multiplier is because there are four detector channels.

The UNIX timestamp and DAQBOX frame number define the left time bin edge for each spectrum packet. The frame number indicates the number of 32ms steps that have passed since the last roll-over to zero. The accuracy of absolute timing is almost certainly better than a microsecond.

Bonus: parsing health data in addition to science data

... TBD. Here's the basic idea:

• Figure out the interval you want to analyze
• Query the IMPISH ground database for the system health packets you need
• Coordinate that information into e.g. your instrument response generation.