Skip to content

Latest commit

 

History

History
32 lines (21 loc) · 2.92 KB

README.md

File metadata and controls

32 lines (21 loc) · 2.92 KB
Raw

TMT-IRIS metadata

This repository attempts to collect and organize the IRIS metadata.

  • Many of IRIS metadata come from Observatory Telemetry, these are defined in "model files" and control which telemetry from other instruments and systems IRIS will be subscribed to. This list is maintaned on Github in the subscribe-model.conf in the tmt-icd/IRIS-Model-Files repository in a text format
  • IRIS metadata are then saved to FITS headers so they are available for offline processing. However the FITS files also contain other metadata which do not come from telemetry but are created during the observations.
  • Moreover we need to track which FITS headers go in each of the different type of FITS files generated by the IRIS processing pipeline.

We plan to have a single Google Sheet which collects all of this information in 1 place and is human and machine readable. Then provide software tools in this repository to generate the subscribe-model.conf file and to generate the metadata templates that go into the Python package iris_pipeline and define what FITS headers are expected to be in each file.

Once these tools are available, the Google Sheet will be the master copy and when edited, we can propagate the changes to the rest.

Master sheet on Google

Google master sheet

After the initial import, this should be considered the master copy where this information is kept updated. Then this will be exported to hocon model files for the subscribe-model.conf file and be a unique reference for all the metadata that goes into the FITS files produced by the observatory.

Versions

Versions are tagged both in the repository and using "Name current version" on Google Sheets. See https://github.com/oirlab/iris_metadata/releases

What is availble in this repository

  • import.ipynb Jupyter Notebook used to merge the telemetry from the model files and the FITS metadata spreadsheet
  • [Metadata_IRIS_new - icd.csv](Metadata_IRIS_new - icd.csv) Dump of the FITS metadata spreadsheet
  • IRIS_telemetry_and_FITS_metadata.csv, initially created from import.ipynb, it will be kept synchronized with the google spreadsheet.
  • csv_to_google_sheet.ipynb: Jupyter Notebook to take the CSV and upload it to Google Sheet overwriting what is there
  • generate-subscribe-model.ipynb: Jupyter Notebook to generate subscribe-model.conf in HOCON format from IRIS_telemetry_and_FITS_metadata.csv
  • download_google_sheet.sh: Script to download the Master Google sheet to IRIS_telemetry_and_FITS_metadata.csv