Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiple Recordings Extension #99

Closed
wants to merge 2 commits into from
Closed

Multiple Recordings Extension #99

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
de02b43
Renaming 'volatile' extension to 'multirecordings'.
bhilburn Oct 2, 2018
b085bd4
Implementing the multirecordings extension.
bhilburn Oct 2, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
9 changes: 9 additions & 0 deletions extensions/antenna.sigmf-ext.md
View file
Open in desktop
Expand Up @@ -35,3 +35,12 @@ the `annotations` object:
|`azimuth_angle`|false|float|degrees|Angle of main beam in azimuthal plane from North.|
|`elevation_angle`|false|float|degrees|Angle of main beam in elevation plane from horizontal.|
|`polarization`|false|float|string|E.g. `"vertical"`, `"horizontal"`, `"slant-45"`, `"left-hand circular"`, `"right-hand circular"`.|

### `multirecordings` Compatibility

If both the `antenna` and `multirecordings` extensions are supported, the
following fields in the `antenna` namespace MAY use the `recording` datatype in
the `annotations` object:

* `azimuth_angle`
* `elevation_angle`
103 changes: 103 additions & 0 deletions extensions/multirecordings.sigmf-ext.md
View file
Open in desktop
@@ -0,0 +1,103 @@
# Multi-Recordings Extension v0.0.1

This extension makes it possible to link multiple SigMF recordings together to
represent a single data-capture event. This is primarily used for two purposes:
representing aspects of one SigMF recording with another recording, and
capturing multi-channel data.

For some recordings, there are fields that are continuously variable. Examples
include the geolocation of a radio receiver that is on a moving vehicle, the
azimuth of a spinning antenna aperture, and the temperature readings of a
thermal sensor. If these were stored as metadata in the "primary" signal
recording, they would bloat and complicate the use of that recording.
Furthermore, it would be difficult to process any aspect of the data without
parsing all of the recorded data. This extension enables you to treat such
fields as *simply another `SigMF Recording`*.

For recordings that digitize multiple streams of information (e.g., a
phased-array receiver with multiple channels), this extension enables you to
capture the multiple streams and link them such that each one can be treated as
a separate `SigMF Recording` or they may be treated together as one.

Per the SigMF specification, all recordings must be independently evaluated and
be able to stand-alone.

## Datatypes

| type | long-form name | description |
| --------- | --------------------- | ------------------------------------------------------- |
| recording | string recording name | A string that indicates the name of a `SigMF Recording` |

The `recording` datatype is the primary mechanism by which SigMF Recordings are
linked. This datatype is the name of a SigMF Recording, **without** extensions
(i.e., given a recording comprising `N.sigmf-meta` and `N.sigmf-data`, the
`recording` datatype value would simply be `N`).

The metadata and dataset files of the recording MUST be in the local directory
(i.e., local to the SigMF Recording in which they are referenced, and thus in
the same SigMF Archive).

### `core` Namespace Support
bhilburn marked this conversation as resolved.
Show resolved Hide resolved

If the `multirecordings` extension is supported, it MAY be used in the following
name/value pairs within the `core` namespace.

#### Capture Segment Objects

The following are existing name/value pairs in capture segment objects in the
`core` namespace:

| name | required | type | description |
| -------------- | -------- | --------- | ------------ |
| `frequency` | false | recording | The center frequency of the signal in Hz.|

### `multirecordings` Namespace

This extension adds the following name/value pairs under the `multirecordings`
namespace.

#### Global Object

| name | required | type | description |
| ------------- | -------- | --------- | ----------------------------------- |
| `master` | false | recording | The primary recording this recording is linked to.|

The `multirecordings:master` field in the `global` object allows the user to
indicate another SigMF recording that acts as the 'master' or 'parent' of this
one, effectively providing a mechanism for bi-directional link between SigMF recordings.
bhilburn marked this conversation as resolved.
Show resolved Hide resolved

#### Capture Segment Objects

| name | required | type | description |
| -------------- | -------- | --------- | ------------ |
| `streams` | false | object | List of SigMF recordings that represent multiple streams.|

The `multirecordings:streams` field in a capture segment object is a JSON array
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this work if there are multiple segments at nonzero offsets? What does that imply for channels other than 'this' channel?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jacobagilbert - I don't think I understand the gap you're pointing out. Can you explain it in a bit more detail for me?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure - I think my question here has to do with this streams object being part of a captures segment, which starts at a user-defined sample (not necessarily 0) and extends until the next segment or EOF. That seems like something that might be a challenge. It also might be well thought out, so this is not to say 'this is a bad idea' -- rather 'is this what was intended'? Or perhaps this should be in global?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I removed this from my re-write of the extension, and then realized why it was there 😅

This is needed to point from the primary recording down to children. That said, I agree with your comment about it being in captures. The new PR moves this to global.

of `recording` strings, as defined by this extension, that indicate multiple
streams of data that were captured as part of the same event.

An example of using the `streams` field in a recording called `example-channel-0`:

```json
...
"captures": [
{
"core:sample_start": 0,
"core:frequency": 900000000,
"core:time": "2017-02-01T11:33:17053240428Z",
"multirecordings:streams": [
"example-channel-0",
"example-channel-1",
"example-channel-2",
"example-channel-3"
]
}
],
...
```

In the example above, the metadata in the only `capture segment object`
indicates that the full data capture event consists of four recorded streams,
the first one is "this" recording (`example-channel-0`), and that there are
three other SigMF recordings in the local directory that represent the other
three channels.
Empty file removed extensions/volatile.sigmf-ext.md
View file
Open in desktop
Empty file.
160 changes: 82 additions & 78 deletions sigmf-spec.md
View file
Open in desktop
Expand Up @@ -28,34 +28,34 @@ This document is available under the [CC-BY-SA License](http://creativecommons.o

## Table of Contents

<!-- markdown-toc start - Don't edit this section. Run M-x markdown-toc-generate-toc again -->
**Table of Contents**

- [Signal Metadata Format Specification v0.0.1](#signal-metadata-format-specification-v001)
- [Abstract](#abstract)
- [Status of this Document](#status-of-this-document)
- [Copyright Notice](#copyright-notice)
- [Table of Contents](#table-of-contents)
- [Introduction](#introduction)
- [Conventions Used in this Document](#conventions-used-in-this-document)
- [Specification](#specification)
- [Files](#files)
- [Archive Format](#archive-format)
- [Dataset Format](#dataset-format)
- [Metadata Format](#metadata-format)
- [Datatypes](#datatypes)
- [Namespaces](#namespaces)
- [Global Object](#global-object)
- [Captures Array](#captures-array)
- [Capture Segment Objects](#capture-segment-objects)
- [Annotations Array](#annotations-array)
- [Annotation Segment Objects](#annotation-segment-objects)
- [Dataset Licensing](#dataset-licensing)
- [SigMF Compliance by Applications](#sigmf-compliance-by-applications)
- [Example](#example)
- [Acknowledgements](#acknowledgements)

<!-- markdown-toc end -->
* [Signal Metadata Format Specification v0.0.2](#signal-metadata-format-specification-v002)
* [Abstract](#abstract)
* [Status of this Document](#status-of-this-document)
* [Copyright Notice](#copyright-notice)
* [Table of Contents](#table-of-contents)
* [Introduction](#introduction)
* [Conventions Used in this Document](#conventions-used-in-this-document)
* [Specification](#specification)
* [Files](#files)
* [SigMF Archives](#sigmf-archives)
* [Dataset Format](#dataset-format)
* [Metadata Format](#metadata-format)
* [Datatypes](#datatypes)
* [Namespaces](#namespaces)
* [Extension Namespaces](#extension-namespaces)
* [Canonical Extension Namespaces](#canonical-extension-namespaces)
* [Global Object](#global-object)
* [The `extensions` Field](#the-extensions-field)
* [Captures Array](#captures-array)
* [Capture Segment Objects](#capture-segment-objects)
* [The `datetime` Pair](#the-datetime-pair)
* [Annotations Array](#annotations-array)
* [Annotation Segment Objects](#annotation-segment-objects)
* [Dataset Licensing](#dataset-licensing)
* [SigMF Compliance by Applications](#sigmf-compliance-by-applications)
* [Example](#example)
* [Citing SigMF](#citing-sigmf)
* [Acknowledgements](#acknowledgements)

## Introduction

Expand Down Expand Up @@ -129,12 +129,13 @@ into a file archive. A `SigMF Archive` may contain multiple `SigMF Recordings`.

1. The archive MUST use the `tar` archive format, as specified by POSIX.1-2001.
2. The archive file's filename extension MUST be `.sigmf`.
3. The archive MUST contain the following files: for each contained recording
with some name given here meta-syntactically as `N`, files named `N` (a
directory), `N/N.sigmf-meta`, and `N/N.sigmf-data`.
4. The archive MUST NOT contain any other files unless their pathnames begin
with `N/N`, for some `N` which has `.sigmf-meta` and `.sigmf-data` files as
described above.
3. The archive MAY be compressed. If it is compressed, it MUST have a double
extension, where the second extension reflects the compression format
(e.g., `N.sigmf.gz`).
4. It is RECOMMENDED that if there is a "primary" or "top-level" recording in
an archive, that it have the same name as the archive directory (given an
archive name `N.sigmf`, `N/N.sigmf-meta` and `N/N.sigmf-data` represent the
primary recording in the archive).
5. It is RECOMMENDED that if recordings in an archive represent a continuous
dataset that has been split into separate recordings, that their filenames
reflect the order of the series by appending a hyphenated zero-based index
Expand Down Expand Up @@ -193,7 +194,7 @@ be namespaced.

The format of the name/value pairs is:

```
```json
"namespace:name": value,
```

Expand All @@ -212,16 +213,16 @@ The format of the name/value pairs is:

The values in each name/value pair must be one of the following datatypes:

|type|long-form name|description|
|----|----|-----------|
|int|integer|Signed 64-bit integer.|
|uint|unsigned long|Unsigned 64-bit integer.|
|double|double-precision floating-point number|A 64-bit float as defined by IEEE 754.|
|string|string|A string of characters, as defined by the JSON standard.|
|boolean|boolean|Either `true` or `false`, as defined by the JSON standard.|
|null|null|`null`, as defined by the JSON standard.|
|array|JSON `array`|an `array` of other values, as defined by the JSON standard.|
|object|JSON `object`|an `object` of other values, as defined by the JSON standard.|
| type | long-form name | description |
| ------- | -------------------------------------- | ------------------------------------------------------------- |
| int | integer | Signed 64-bit integer. |
| uint | unsigned long | Unsigned 64-bit integer. |
| double | double-precision floating-point number | A 64-bit float as defined by IEEE 754. |
| string | string | A string of characters, as defined by the JSON standard. |
| boolean | boolean | Either `true` or `false`, as defined by the JSON standard. |
| null | null | `null`, as defined by the JSON standard. |
| array | JSON `array` | an `array` of other values, as defined by the JSON standard. |
| object | JSON `object` | an `object` of other values, as defined by the JSON standard. |

#### Namespaces

Expand Down Expand Up @@ -258,11 +259,13 @@ namespaces may be defined by the user as needed.
the canonical extension namespaces.

##### Canonical Extension Namespaces

This is a list of the canonical extension namespaces defined by SigMF:

* `antenna` - Used to describe the antenna(s) used to for the recording.
* `modulation` - Defines how to describe modulations used in wireless communications systems.
* `volatile` - Allows for continously time-varying fields, such as a moving receiver or rotating antenna.
* `antenna` - Used to describe the antenna(s) used to for the recording.
* `modulation` - Defines how to describe modulations used in wireless communications systems.
* `multirecordings` - Allows for continously time-varying fields, such as a
moving receiver or rotating antenna, and multi-channel recordings.

#### Global Object

Expand All @@ -274,29 +277,30 @@ about the recording itself.
The following names are specified in the `core` namespace and should be used in
the `global` object:

|name|required|type|description|
|----|--------------|-------|-----------|
|`datatype`|true|string|The format of the stored samples in the dataset file. Its value must be a valid SigMF dataset format type string.|
|`sample_rate`|false|double|The sample rate of the signal in samples per second.|
|`version`|true|string|The version of the SigMF specification used to create the metadata file.|
|`sha512`|false|string|The SHA512 hash of the dataset file associated with the SigMF file.|
|`offset`|false|uint|The index number of the first sample in the dataset. This value defaults to zero. Typically used when a recording is split over multiple files.|
|`description`|false|string|A text description of the SigMF recording.|
|`author`|false |string|The author's name (and optionally e-mail address).|
|`recorder`|false|string|The name of the software used to make this SigMF recording.|
|`license`|false|string|A URL for the license document under which the recording is offered; when possible, use the canonical document provided by the license author, or, failing that, a well-known one.|
|`hw`|false |string|A text description of the hardware used to make the recording.|
|`extensions`|false|object|A list of extensions used by this recording.|
| name | required | type | description |
| ------------- | -------- | ------ | ----------------------------------- |
| `datatype` | true | string | The format of the stored samples in the dataset file. Its value must be a valid SigMF dataset format type string.|
| `sample_rate` | false | double | The sample rate of the signal in samples per second.|
| `version` | true | string | The version of the SigMF specification used to create the metadata file.|
| `sha512` | false | string | The SHA512 hash of the dataset file associated with the SigMF file.|
| `offset` | false | uint | The index number of the first sample in the dataset. This value defaults to zero. Typically used when a recording is split over multiple files.|
| `description` | false | string | A text description of the SigMF recording.|
| `author` | false | string | The author's name (and optionally e-mail address).|
| `recorder` | false | string | The name of the software used to make this SigMF recording.|
| `license` | false | string | A URL for the license document under which the recording is offered; when possible, use the canonical document provided by the license author, or, failing that, a well-known one.|
| `hw` | false | string | A text description of the hardware used to make the recording.|
| `extensions` | false | object | A list of extensions used by this recording.|

##### The `extensions` Field

The `core:extensions` field in the `global` object is JSON array of name/value
pairs describing `SigMF Extension` namespaces, where the name is the namespace
provided by an extension and the value is a string that specifies whether the
extension is `optional` or the version of the extension required to properly
parse & process the SigMF Recording. In the example below, `extension-01` is
used, but not required, and `version 1.2.3` of `extension-02` *is* required.

```JSON
```json
"global": {
...
"core:extensions" : {
Expand Down Expand Up @@ -324,12 +328,12 @@ pairs apply.
The following names are specified in the `core` namespace and should be used in
capture segment objects:

|name|required|type|description|
|----|--------------|-------|-----------|
|`sample_start`|true|uint|The sample index at which this segment takes effect.|
|`length`|false|uint|The length of this capture segment, in number of samples.|
|`frequency`|false|double|The center frequency of the signal in Hz.|
|`datetime`|false|string|An ISO-8601 string indicating the timestamp of the sample index specified by `sample_start`. More details, below.|
| name | required | type | description |
| -------------- | -------- | ------ | ------------ |
| `sample_start` | true | uint | The sample index at which this segment takes effect.|
| `length` | false | uint | The length of this capture segment, in number of samples.|
| `frequency` | false | double | The center frequency of the signal in Hz.|
| `datetime` | false | string | An ISO-8601 string indicating the timestamp of the sample index specified by `sample_start`. More details, below.|

###### The `datetime` Pair

Expand Down Expand Up @@ -377,16 +381,16 @@ name/value pairs apply.
The following names are specified in the `core` namespace and should be used in
annotation segment objects:

|name|required|type|description|
|----|--------------|-------|-----------|
|`sample_start`|true|uint|The sample index at which this segment takes effect.|
|`sample_count`|true|uint|The number of samples that this segment applies to. |
|`generator`|false|string|Human-readable name of the entity that created this annotation.|
|`comment`|false|string|A human-readable comment.|
|`freq_lower_edge`|false|double|The frequency (Hz) of the lower edge of the feature described by this annotation.|
|`freq_upper_edge`|false|double|The frequency (Hz) of the upper edge of the feature described by this annotation.|
|`latitude`|false|| |
|`longitude`|false|| |
| name | required | type | description |
| ----------------- | -------- | ------ | --------------------------------------------------------------------------------- |
| `sample_start` | true | uint | The sample index at which this segment takes effect. |
| `sample_count` | true | uint | The number of samples that this segment applies to. |
| `generator` | false | string | Human-readable name of the entity that created this annotation. |
| `comment` | false | string | A human-readable comment. |
| `freq_lower_edge` | false | double | The frequency (Hz) of the lower edge of the feature described by this annotation. |
| `freq_upper_edge` | false | double | The frequency (Hz) of the upper edge of the feature described by this annotation. |
| `latitude` | false | |
| `longitude` | false | |

There is no limit to the number of annotations that can apply to the same group
of samples. If two annotations have the same `sample_start`, there is no
Expand Down