New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Fido client and filetype attr #31
Add Fido client and filetype attr #31
Conversation
This is great! It looks like the data server issues are finally fixed now, so I will go ahead and merge this in and work on updating the docs to match. For now, I think it is best to keep the One question for future consideration, is it possible to connect a database of metadata to a Fido client? There is a lot of coordinate and wavelength information in the catalog that I assume would be useful to include as search parametersin Fido (although the official EIS as-run catalog is a bit of a maze, so we might need to generate a reduced database for the just the HDF5 files). |
90270c9
to
a9d128a
Compare
I think this is finally ready (only about a year late!) Please squash this before merging. The commit history is a bit all over the place. |
bool | ||
True if this client can handle the given query. | ||
""" | ||
required = {a.Time, a.Instrument, a.Source} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you need to make Source required?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There could be another future instrument called EIS? 🤷
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That doesn't mean you need to support source, if someone didn't provide source then they would get both. They could optionally use source to disambiguate.
(It just needs moving to optional).
@MJWeberg we talked about this a bit offline a few months ago, but just wanted to record my thoughts here for posterity. This is definitely possible. The difficulty lies in the fact that the EIS catalogue is not a hosted SQL database, but rather a local file that one needs to download. This means it's not just a matter of translating a Fido query into a query against the EIS catalogue, but also downloading the catalogue and making sure it is up to date, with the assumption that there is always an existing correspondence between this local catalogue and what is being provided on the remote server. I think this is an interesting direction and would enable much more detailed queries, but is more complicated than what is proposed here. I would recommend spinning this out as a separate issue. |
Finally getting back to this now. This is great, Will! Thanks you very much! I agree that linking in to the SQLite database would be nice, but is out of the scope of this PR. For now, people can still use the eis_catalog GUI to query and download observations based on the metadata. |
This pull request adds a new Fido client called
EISClient
that enables searches of the level 1 HDF5 and FITS files via Fido, the sunpy data retrieval interface. An additional attr is also provided to discriminate between the HDF data and header files and FITS files.Though I've not deleted any code, I would recommend that this supersede the code that is in
download/download_hdf5_data.py
.A few things that could still be done:
Removedownload/download_hdf5_data.py
as this provides duplicate functionalitynet
subpackageAt the moment, this is difficult to test because eis.nrl.navy.mil is experiencing SSL certificate issues.This PR should be squash merged.