This is the most up to date version of this file format produced by Borealis version 0.7, the current version.
For data files from previous Borealis software versions, see here.
The pyDARNio format class for this format is BorealisRawrf found in the borealis_formats.
The rawrf format is intended to hold high bandwidth, non-filtered raw data from every antenna.
This format is only produced in a site-style, record by record format and is only available to be produced on request. Please note that this format can cause radar operating delays and may reduce number of averages in an integration, for example.
Site files are produced by the Borealis code package and have the data in a record by record style format. In site files, the hdf5 group names (ie record names) are given as the timestamp in ms past epoch of the first sequence or sampling period recorded in the record.
The naming convention of the rawrf site-structured files are: :
[YYYYmmDD].[HHMM].[SS].[station_id].rawrf.hdf5.site
For example: :
20191105.1400.02.sas.rawrf.hdf5.site
This is the file that began writing at 14:00:02 UT on November 5 2019 at the Saskatoon site, and it provides data the experiment that ran at that time. Since rawrf is not filtered, this data does not need a slice identifier because it contains all the samples being taken at that time. Some familiarity with the experiment may be necessary to understand the data, or some access to the other file types produced concurrently. This is primarily a debug format for engineering purposes and should only be produced for special cases.
These files are often bzipped after they are produced.
The file fields under the record name in rawrf site files are:
Field name
| type |
description |
---|---|
agc_status_word uint32 |
AGC status word. Bit position corresponds to the USRP motherboard/ transmitter. A '1' indicates an agc fault occurred at least once during integration |
blanked_samples uint32 [number of blanked samples] |
Samples that should be blanked because they occurred during transmission times, given by sample number (index into decimated data). Can differ from the pulses array due to multiple slices in a single sequence. |
borealis_git_hash unicode |
Identifies the version of Borealis that made this data. Contains git commit hash characters. Typically begins with the latest git tag of the software. |
data [complex64, ] |
A contiguous set of samples (complex float) at given sample rate. Needs to be reshaped by data_dimensions to be correctly read. |
data_descriptors [bytes, ] |
Denotes what each data dimension represents. = ‘num_sequences’, ‘num_antennas’, ‘num_samps’ for rawrf |
data_dimensions [uint32, ] |
The dimensions of the data. Dimensions correspond to data_descriptors. |
experiment_comment unicode |
Comment provided in experiment about the experiment as a whole. |
experiment_id int16 |
Number used to identify the experiment. |
experiment_name unicode |
Name of the experiment file. |
gps_locked bool |
Designates if the local GPS had a lock during the entire integration period. |
gps_to_system_time_diff float32 |
The max time difference between box_time GPS time) and system time (NTP) during the integration. Negative when GPS time is ahead of system time. |
int_time float32 |
Integration time in seconds. |
intf_antenna_count uint32 |
Number of interferometer array antennas |
lp_status_word uint32 |
Low power status word. Bit position corresponds to the USRP motherboard/ transmitter. A '1' indicates low power occurred at least once during integration |
main_antenna_count uint32 |
Number of main array antennas |
num_samps uint32 |
Number of samples in the sampling period. Each sequence has its own sampling period. Will also be provided as the last data_dimension value. |
num_sequences int64 |
Number of sampling periods (equivalent to number sequences transmitted) in the integration time. |
num_slices int64 |
Number of slices used simultaneously in this record by the experiment. If more than 1, data should exist in another file for this time period for the other slice. |
rx_center_freq float64 |
Center frequency of the sampled data in kHz. |
rx_sample_rate float64 |
Sampling rate of the samples in this file's data in Hz. |
samples_data_type unicode |
C data type of the samples, provided for user friendliness. = 'complex float' |
scan_start_marker bool |
Designates if the record is the first in a scan (scan is defined by the experiment). |
scheduling_mode unicode |
The mode being run during this time period (ex. 'common', 'special', 'discretionary'). |
sqn_timestamps [float64, ] |
A list of GPS timestamps corresponding to the beginning of transmission for each sampling period in the integration time. These timestamps come from the USRP driver and the USRPs are GPS disciplined and synchronized using the Octoclock. Provided in seconds since epoch. |
station unicode |
Three-letter radar identifier. |
tx_antenna_phases [complex64, ] |
The complex phase for each antenna for transmission, normalized such that full- power has magnitude 1. |
File restructuring to array files is not done for this format.