Adds Sensor column in Fido QueryResponseTable if present in SOAR metadata by daraghhollman · Pull Request #206 · sunpy/sunpy-soar

daraghhollman · 2026-04-14T20:07:22Z

PR Description

The EPD instrument on Solar Orbiter uses the 'sensor' key instead of 'detector' in the SOAR metadata. A 'Detector' column hence doesn't exist in the QueryReponseTable output from Fido. This makes it difficult to only source data from a specfic sensor, with the only current workaround being to filter by string in the 'Data product' column (see #205).

These (ableit small) changes create a 'Sensor' column in the table output if the 'sensor' key is present in the metadata, allowing for simple table filtering based on column values. e.g.

from sunpy.net import Fido
from sunpy.net import attrs as a

import sunpy_soar

search = Fido.search(
    a.Time("2021-11-20", "2021-11-30"),
    a.Level(2),
    a.Instrument.epd,
)["soar"]

filtered_search = search[search["Sensor"] == "EPT"]

This PR partially addresses #205, however, I think a full solution should enable filtering by sensor during the Fido query (i.e. implementing these sensors in a.Detector) - though I think this is outside my ability / current understanding of the codebase.

AI Assistance Disclosure

AI tools were used for:

Code generation (e.g., when writing an implementation or fixing a bug)
Test/benchmark generation
Documentation (including examples)
Research and understanding
No AI tools were used

Regardless of AI use, the human contributor remains fully responsible for correctness, design choices, licensing compatibility, and long-term maintainability.

…n metadata EPD uses the 'sensor' key instead of 'detector' in pulled metadata. Exposing this in the search table allows for easier filtering for this instrument.

hayesla · 2026-04-14T20:31:52Z

thanks for this PR @daraghhollman! it would be nice to also be able to build a query with the sensor in it, i.e. liek a.soar.Sensor . We could register this as a new attrs. I wonder why some instruments have it listed as a detector or sensor?

It looks like these are unique sensor names in the table it queries, from the v_sc_data_item table

BIAS
EAS
EAS1
EAS2
EPT
EPTHET1
EPTHET2
FDT
FSI
FSI174
FSI304
HET
HFR
HIS
HRI
HRI1216
HRI174
HRT
IBS
LFR
NARROW
OBS
PAS
SIS
STEP
TDS
TNR
UV
VL
WIDE

daraghhollman · 2026-04-15T07:25:29Z

Yeah I think that would be ideal.

I'm also curious about sensor vs. detector. I wonder if there are any products which define both simultaneously. Maybe it's just a semantics difference.
Have you an idea on how we could check for that @hayesla?

If there are no overlaps, do you still think it's worth creating a new attr, a.Sensor, or would it be better to register these under a.Dectector in these cases?

daraghhollman · 2026-04-15T08:51:43Z

The above commit implements a.soar.Sensor to enable Fido queries as follows:

e.g.

from sunpy.net import Fido
from sunpy.net import attrs as a

import sunpy_soar

search = Fido.search(
    a.Time("2021-11-20", "2021-11-22"),
    a.Level(2),
    a.Instrument.epd,
    a.soar.Sensor("EPT"),
)

I guess this will need some documentation additions too?

…ch_attrs.py` Sensors 'NARROW' and 'WIDE' are still missing descriptions

daraghhollman · 2026-04-15T10:31:21Z

The above commit adds a.soar.Sensor to the 'Available search attributes' documentation page (examples/search_attrs.py). I've populated a sensor_attrs.json file with descriptions from what I could find by searching SOAR (from the list @hayesla made with v_sc_data_item), however I was not able to find anything for NARROW or WIDE. These have been left blank.

Screenshot from docs build:

hayesla

This is looking great @daraghhollman - a lil test would be nice, and a small example, even just the EPD/EPT would be great!

hayesla · 2026-04-15T10:48:09Z

@@ -0,0 +1,32 @@
+{
+  "BIAS": "Radio and Plasma Waves Bias Unit",


did you pull these from the data table with a tap query? we coul dmaybe add this to the tools on how this was generated :)

No this was sadly a manual process. Searching product names via the SOAR search page directly, and looking at the metadata for matching files.

ah ok, I'm hesitant to putting this hardcoded attrs into the repo. You can see https://github.com/sunpy/sunpy-soar/blob/main/tools/update_data.py is how we generate the attrs info data. We can maybe do something liek

soar = TapPlus(url="http://soar.esac.esa.int/soar-sl-tap/tap") def get_all_sensors(): print("Updating sensor descriptors...") job = soar.launch_job("SELECT * FROM soar.sensor") res = job.get_results() sensor_names = {} for row in res: sensor_names[row["name"]] = row.get("long_name", "") return sensor_names

which i think is correct, and returns:

{'EPT': np.str_('Electron Proton Telescope'), 'HET': np.str_('High-Energy Telescope'), 'SIS': np.str_('Suprathermal Ion Spectrograph'), 'STEP': np.str_('SupraThermal Electrons and Protons'), 'EPTHET1': np.str_('Electron Proton Telescope and High-Energy Telescope 1'), 'EPTHET2': np.str_('Electron Proton Telescope and High-Energy Telescope 2'), 'FSI': np.str_('Full Sun Imager'), 'FSI174': np.str_('Full Sun Imager at 174 Angstrom'), 'FSI304': np.str_('Full Sun Imager at 304 Angstrom'), 'HRI': np.str_('High Resolution Imager'), 'HRI174': np.str_('High Resolution Imager at 174 Angstrom'), 'HRI1216': np.str_('High Resolution Imager at 1216 Angstrom'), 'IBS': np.str_('Inboard Sensor'), 'OBS': np.str_('Outboard Sensor'), 'VL': np.str_('Visible Light'), 'UV': np.str_('Ultraviolet'), 'FDT': np.str_('Full Disk Telescope'), 'HRT': np.str_('High Resolution Telescope'), 'LFR': np.str_('Low Frequency Receiver'), 'TDS': np.str_('Time Domain Sampler'), 'TNR': np.str_('Thermal Noise Receiver'), 'HFR': np.str_('High Frequency Receiver'), 'BIAS': np.str_('Biasing Unit'), 'NARROW': np.str_('Narrow Slits'), 'WIDE': np.str_('Wide Slit'), 'EAS': np.str_('Electron Analyser System'), 'EAS1': np.str_('Electron Analyser Sensor 1'), 'EAS2': np.str_('Electron Analyser Sensor 2'), 'HIS': np.str_('Heavy Ion Sensor'), 'PAS': np.str_('Proton and Alpha Particle Sensor')}

Yeah that makes a lot of sense! I'll give it a whirl and verify.

I've added this function to tools/update_data.py and call it in the __main__ step, writing to sunpy_soar/data/sensor_attrs.json.

Returns a Dict[str, str] containing sensor names and descriptions. This is called at the end of this file and writes to `sunpy_soar/data/sensor_attrs.json`

Checks if the sensor `ept` is in the registered sensors.

hayesla

This is looking great @daraghhollman - just some small suggestions

hayesla · 2026-04-16T19:38:18Z

@@ -0,0 +1,3 @@
+Added ``a.soar.Sensor`` to attrs, enabling users to query with a specific instrument sensor, e.g. the EPT sensor on the EPD instrument.
+This was required as ``a.Detector`` does not provide a comprenhensive listing of all available sensors on Solar Orbiter.


dont need this sentence

Suggested change

This was required as ``a.Detector`` does not provide a comprenhensive listing of all available sensors on Solar Orbiter.

hayesla · 2026-04-16T19:41:34Z

 # L0, L1, L2, L3, LL01, LL02, LL03
 #
+# For specific instrument detectors or sensors, see ``a.Detector``. However, some SOAR products require the use of ``a.soar.Sensor`` attribute instead.
+#####################################################


I'd put this after the .products.

hayesla · 2026-04-16T19:43:19Z

@@ -0,0 +1,34 @@
+"""
+===========================================================
+Searching for Solar Orbiter data using the Sensor attribute


Suggested change

Searching for Solar Orbiter data using the Sensor attribute

Searching for EPD/EPT data with the `a.soar.Sensor` attribute

Move `a.soar.Sensor` to after `a.soar.Product` Update `a.soar.Sensor` EPD/EPT example title Update changelog, removing redundant line

Include Sensor column in Fido QueryResponseTable if 'sensor' exists i…

4b50a93

…n metadata EPD uses the 'sensor' key instead of 'detector' in pulled metadata. Exposing this in the search table allows for easier filtering for this instrument.

Adds a.soar.Sensor to enable query by Sensor name with Fido

2dbf93a

Adds documentation for a.soar.Sensor descriptions in `examples/sear…

35c3d8e

…ch_attrs.py` Sensors 'NARROW' and 'WIDE' are still missing descriptions

hayesla reviewed Apr 15, 2026

View reviewed changes

daraghhollman added 4 commits April 15, 2026 14:23

Adds get_all_sensors function to tools/update_data.py

e380dfa

Returns a Dict[str, str] containing sensor names and descriptions. This is called at the end of this file and writes to `sunpy_soar/data/sensor_attrs.json`

Adds test for sensor attributes

ceebe05

Checks if the sensor `ept` is in the registered sensors.

Adds simple documentation example for working with a.soar.Sensor

3857857

Add changelog

4227e66

hayesla approved these changes Apr 16, 2026

View reviewed changes

Updates based on reviewer suggestions

8f0694d

Move `a.soar.Sensor` to after `a.soar.Product` Update `a.soar.Sensor` EPD/EPT example title Update changelog, removing redundant line

		@@ -0,0 +1,32 @@
		{
		"BIAS": "Radio and Plasma Waves Bias Unit",

		@@ -0,0 +1,3 @@
		Added ``a.soar.Sensor`` to attrs, enabling users to query with a specific instrument sensor, e.g. the EPT sensor on the EPD instrument.
		This was required as ``a.Detector`` does not provide a comprenhensive listing of all available sensors on Solar Orbiter.

	Searching for Solar Orbiter data using the Sensor attribute
	Searching for EPD/EPT data with the `a.soar.Sensor` attribute

Uh oh!

Conversation

daraghhollman commented Apr 14, 2026

PR Description

AI Assistance Disclosure

Uh oh!

hayesla commented Apr 14, 2026

Uh oh!

daraghhollman commented Apr 15, 2026

Uh oh!

daraghhollman commented Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daraghhollman commented Apr 15, 2026

Uh oh!

hayesla left a comment

Choose a reason for hiding this comment

Uh oh!

hayesla Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

daraghhollman Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hayesla Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daraghhollman Apr 15, 2026

Choose a reason for hiding this comment

Uh oh!

daraghhollman Apr 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hayesla left a comment

Choose a reason for hiding this comment

Uh oh!

hayesla Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

hayesla Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

hayesla Apr 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

daraghhollman commented Apr 15, 2026 •

edited

Loading

daraghhollman Apr 15, 2026 •

edited

Loading

hayesla Apr 15, 2026 •

edited

Loading

daraghhollman Apr 15, 2026 •

edited

Loading