Generalize and Improve Annotation Handling #32

aromanielloNTIA · 2022-08-15T23:15:52Z

Overview

This update enhances the implementation of SigMF annotation segments and measurement metadata. Hard-coded metadata values are removed to ensure that actions provide accurate metadata as required by the SigMF specifications. In general, metadata is no longer pulled by the annotation classes directly from a measurement_result dictionary, which makes annotation use more explicit. Additionally, actions no longer need to store metadata in the measurement_result.

These changes extend the functionality of the annotation classes, allowing better support for future actions. For example, the annotation class TimeDomainDetection can now support time domain captures other than raw IQ captures.

All existing actions are updated to make use of the refactored metadata classes.

Changelog

Add imports to scos_actions/metadata/annotations/__init__.py so that all annotations are importable from scos_actions.metadata.annotations
Refactor annotations as Python dataclasses. This makes their definitions easier to write, more legible, and more consistent.
Refactor annotations so that they don't depend on specific keys being present in measurement_result dictionaries, instead requiring each metadata value to be specified as a parameter for the annotation instance
Adds the AnnotationSegment base dataclass, which annotations can be built from. This base class implements the required and optional keys associated with any possible SigMF annotation segment. This replaces the old Metadata base class.
Change the MeasurementMetadata class to a dataclass, and implement previously missing keys which are part of the SigMF specification. Also adds checks that required values are specified.
Update all annotation classes to:
- Inherit from AnnotationSegment
- Contain all possible keys defined in the related SigMF specification (many were previously missing)
- Check that all keys required by the related SigMF specification are given values
- Add docstrings describing the parameters of the annotation segment.
Note on CalibrationAnnotation:
- Keep old functionality: has a parameter to specify the gain_sensor key, which is actually not a part of the ntia-sensor specification
Notes on FrequencyDomainDetection annotation:
- Moved from ../fft_annotation.py to ../frequency_domain_detection.py
- Keep old functionality: prepend "fft_" to the detector parameter, to make it easy to use with the new generalized detector implementation
Notes on TimeDomainDetection:
- Moved from ../time_domain_annotation.py to ../time_domain_detection.py
Update actions (acquire_single_freq_fft, acquire_single_freq_tdomain_iq, and acquire_stepped_freq_tdomain_iq) and sigm_builder for the above changes. Note that the changes now require a classification key in the measurement metadata (required by the SigMF specification), and all three actions are hard-coded to record their results as "UNCLASSIFIED"

aromanielloNTIA · 2022-08-16T20:50:16Z

Saving as draft for now- let's save this for after the incoming SigMF-NS-NTIA updates happen, and make all the corresponding SigMF-related updates in one PR.

aromanielloNTIA · 2022-08-30T18:01:55Z

Leaving out ProbabilityDistributionAnnotation (which was waiting for its ntia-algorithm spec to be finalized) makes this PR ready for review. Will add in the ProbabilityDistributionAnnotation with #27 when it's ready.

dboulware

Looks good. I think we need to revisit the metadata handling, but I'll create issues to do it at a later date.

dboulware · 2022-09-08T17:13:58Z

scos_actions/actions/acquire_single_freq_tdomain_iq.py

@@ -99,11 +97,19 @@ def execute(self, schedule_entry, task_id) -> dict:
        measurement_result["description"] = self.description
        measurement_result["sigan_cal"] = self.sigan.sigan_calibration_data
        measurement_result["sensor_cal"] = self.sigan.sensor_calibration_data
+        measurement_result["classification"] = "UNCLASSIFIED"


Would this be better to pull from YAML?

Yes, definitely better than hard-coding this in.

Commenting here to keep this traceable: #41 is this issue

dboulware · 2022-09-08T17:14:25Z

scos_actions/actions/acquire_stepped_freq_tdomain_iq.py

@@ -115,6 +115,7 @@ def __call__(self, schedule_entry_json, task_id):
            measurement_result["name"] = self.name
            measurement_result["sigan_cal"] = self.sigan.sigan_calibration_data
            measurement_result["sensor_cal"] = self.sigan.sensor_calibration_data
+            measurement_result["classification"] = "UNCLASSIFIED"


Would this be better to pull from YAML?

dboulware · 2022-09-08T17:25:36Z

scos_actions/actions/acquire_single_freq_fft.py

@@ -191,6 +188,7 @@ def execute(self, schedule_entry, task_id) -> dict:
        measurement_result["measurement_type"] = MeasurementType.SINGLE_FREQUENCY.value
        measurement_result["sigan_cal"] = self.sigan.sigan_calibration_data
        measurement_result["sensor_cal"] = self.sigan.sensor_calibration_data
+        measurement_result["classification"] = "UNCLASSIFIED"


Might be better to pull from YAML, but we can add an issue to do that.

dboulware · 2022-09-08T17:27:03Z

scos_actions/actions/acquire_single_freq_fft.py

@@ -243,8 +241,19 @@ def description(self):
    def get_sigmf_builder(self, measurement_result) -> SigMFBuilder:
        sigmf_builder = super().get_sigmf_builder(measurement_result)
        for i, detector in enumerate(self.fft_detector):
-            fft_annotation = FrequencyDomainDetectionAnnotation(
-                detector.value, i * self.fft_size, self.fft_size
+            fft_annotation = FrequencyDomainDetection(


This is a lot of arguments. I agree that it helps to make it explicit what it requires but I think we need to revisit this later. Maybe later we move to a builder or fluent interface.

Agreed that we should improve this later. Hopefully along with more metadata handling improvements overall (maybe tied to addressing #29?)

dboulware · 2022-09-08T17:27:43Z

scos_actions/actions/acquire_single_freq_tdomain_iq.py

@@ -99,11 +97,19 @@ def execute(self, schedule_entry, task_id) -> dict:
        measurement_result["description"] = self.description
        measurement_result["sigan_cal"] = self.sigan.sigan_calibration_data
        measurement_result["sensor_cal"] = self.sigan.sensor_calibration_data
+        measurement_result["classification"] = "UNCLASSIFIED"


Same as above, but we don't need to fix now.

dboulware · 2022-09-08T17:32:40Z

scos_actions/actions/interfaces/measurement_action.py

@@ -39,15 +36,50 @@ def __call__(self, schedule_entry, task_id):
    def get_sigmf_builder(self, measurement_result) -> SigMFBuilder:


This is still relying on measuremen_result containing everything. Don't need to fix now, but we should create an issue and revisit.

Figuring out how/when to rely on measurement_result should be a main priority of #40

dboulware · 2022-09-08T17:51:28Z

scos_actions/metadata/annotation_segment.py

+            assert (
+                self.freq_lower_edge is not None and self.freq_upper_edge is not None
+            ), err_msg
+        # Define SigMF key names


We should probably move to using a json schema in the future, but not needed for now.

aromanielloNTIA added 6 commits August 15, 2022 16:43

Generalize time domain annotation

6b99a15

Generalize FFT annotation

f582ce3

Remove dependence on measurement result

0e446be

Update FFT action to new annotation parameters

95ee933

Update FFT action to new annotation parameters

ab9b90c

Cleanup action diffs

03cf8a3

aromanielloNTIA added the enhancement New feature or request label Aug 15, 2022

aromanielloNTIA requested a review from dboulware August 15, 2022 23:15

Make count and num_samps distinct

5f24865

aromanielloNTIA marked this pull request as draft August 16, 2022 20:49

aromanielloNTIA added 6 commits August 17, 2022 11:59

APD annotation progress

66a6cb8

Merge branch 'dsp-refactor' into generalize-annotate

a973bb7

Merge branch 'dsp-refactor' into generalize-annotate

63a0648

Merge branch 'dsp-refactor' into generalize-annotate

5a8044b

Merge branch 'dsp-refactor' into generalize-annotate

dcac854

Merge branch 'dsp-refactor' into generalize-annotate

a158e6a

Base automatically changed from dsp-refactor to master August 23, 2022 16:48

aromanielloNTIA added 12 commits August 23, 2022 10:50

Merge branch 'master' into generalize-annotate

163e3c9

Merge upstream blacken

5f960db

Merge branch 'master' into generalize-annotate

97d163d

Use dataclasses for annotation segments

66fdb3d

Removed default param value

f207bfb

Decouple MeasurementMetadata from AnnotationSegment

6c15126

Update FFT action for new annotation code

93acac1

Update CalibrationAnnotation to dataclass

8e20039

Switch sensor annotation to dataclass

c72040b

Update measurement_action for new annotations

7f3893e

import all annotations to .annotations

511d505

Update create_metadata

3c0fbf4

aromanielloNTIA added 4 commits August 26, 2022 14:04

Added docstring for measurement metadata

f306070

Remove unused parameter

1f18026

Fixed typo

43eda90

Merge branch 'master' into generalize-annotate

a9d2839

aromanielloNTIA changed the title ~~Generalize Time and Frequency Domain Annotations~~ Generalize and Improve Annotation Handling Aug 30, 2022

Remove WIP ProbabilityDistributionAnnotation

05dee3a

aromanielloNTIA marked this pull request as ready for review August 30, 2022 18:01

aromanielloNTIA added 5 commits September 1, 2022 20:07

Make calibration keys optional as in spec

0350f0e

Add metadata exception class

d0a3fe0

Allow setting datatypes other than cf32_le

05900e3

Skip attempting to load values if not provided

00d93e1

Merge branch 'fix-rfpath-notset' into generalize-annotate

14a1f60

dboulware approved these changes Sep 8, 2022

View reviewed changes

Merge branch 'master' into generalize-annotate

b4b5df8

dboulware merged commit 4b3ead1 into master Sep 8, 2022

aromanielloNTIA deleted the generalize-annotate branch September 8, 2022 18:51

aromanielloNTIA mentioned this pull request Sep 8, 2022

Require classification metadata in YAML configs #43

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Generalize and Improve Annotation Handling #32

Generalize and Improve Annotation Handling #32

aromanielloNTIA commented Aug 15, 2022 •

edited

Loading

aromanielloNTIA commented Aug 16, 2022

aromanielloNTIA commented Aug 30, 2022

dboulware left a comment

dboulware Sep 8, 2022

aromanielloNTIA Sep 8, 2022

aromanielloNTIA Sep 8, 2022

dboulware Sep 8, 2022

dboulware Sep 8, 2022

dboulware Sep 8, 2022

aromanielloNTIA Sep 8, 2022

dboulware Sep 8, 2022

dboulware Sep 8, 2022

aromanielloNTIA Sep 8, 2022

dboulware Sep 8, 2022

		@@ -39,15 +36,50 @@ def __call__(self, schedule_entry, task_id):
		def get_sigmf_builder(self, measurement_result) -> SigMFBuilder:

Generalize and Improve Annotation Handling #32

Generalize and Improve Annotation Handling #32

Conversation

aromanielloNTIA commented Aug 15, 2022 • edited Loading

Overview

Changelog

aromanielloNTIA commented Aug 16, 2022

aromanielloNTIA commented Aug 30, 2022

dboulware left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aromanielloNTIA commented Aug 15, 2022 •

edited

Loading