Skip to content

Idea Jot List

Jump to bottom
jvandegriff edited this page Sep 27, 2026 · 3 revisions

Tasks to consider for HAPI development

Tasks in random order -

Just add things here as soon as you think of it, even if it's crazy, or hard, etc. This is the capture page.

  • add contribution guidelines and expectations for all our various HAPI repos

  • still need to normalize and automate our release process

  • a way to validate all existing HAPI info responses (and maybe the SPASE records we make from them?) and let people know if something from their server seems to be incorrect

  • writing an auto-cadence determiner - this woudl help with the SPASE cadence field, plus cadence is actually pretty essential for finding and analyzing time series data (why didn't we require this? maybe we shoudl just require in now..)

  • a HAPI amalgamator - "create my dataset" that fuses multiple HAPI sources with many options, like match other datasets to cadence of a primary dataset, or average everything into 5 min bins, or fuse spectral data of different energies (or frequencies, etc) into a common spectrogram, add ephemeris data to this primary data (this is really an instance of the first item), etc. Also, have lots of output options for saving the data. Could even have a "save for HAPI access later" option that puts data in an arrangement that a local HAPI Server could read. Most people will likely just want files, either CDF, etCDF, HDF, FITS, etc, or possible Python pickle files

  • eventually: a HAPI to SPASE 3.0 converter; currently, the HAPI-to-SPASE converter makes SPASE 2.0 records, and there will be a 2.0 to 3.0 converter for SPASE. But this will miss things that HAPI has that can go into SPASE 3.0 but did not fit into 2.0, so the converter will need revisiting, eventually.

  • finding ways to more directly support people who are using AI to write HAPI scripts

    • validation datasets: this HAPI call should result in exactly the following JSON content: " { # JSON content here }"
    • agents.md file describing how to write HAPI python code for data retrieval
    • MCP servers, or custom GPTs

  • while looking at file listing items, we noticed that the HAPI spec never actually states that HAPI times must only ever increase. So we need to add that to the spec!

Work with the ISTP standards group

Make sure there is enough in ISTP to drive a basic HAPI Server so that data providers just need to implement one kind of metadata, ISTP, and then the rest flow from it, i.e., HAPI and then SPSE.

Side loading of datasetSchema objects

We could create a repo of parameter mappings that allow the transformation of a HAPI dataset into something that conforms to a known entity as defined by our datasetSchema objects. All the ground mag datasets are close, so they might be low hanging fruit. Availability information woudl also be a possibility, as would event lists. More complex data like energetic particles or plasma wave dat might be harder

We could also write pass-through HAPI servers to handle more complex data types. The EPI-Lo instrument on PSP has very complex particle data, and takes lots of custom code to use.

Thoughts about the upcoming PyHC Winter school:

  • Python notebooks and tutorials and scripts (and maybe an entire AI-focused workspace) for analyzing the Gannon storm. These could be used at the upcoming PyHC Winter School or just given out in general as evidence of the usefulness of HAPI for multi-source analysis
  • For the PyHC winter school, some capabilities we could try to make easy with HAPI: An analysis of the Gannon storm, including: How compressed was the magnetosphere? Propagation from moon distance to magnetosphere Waves generation EMIC waves eventually - auroral effects

Clone this wiki locally