While the Gen3 Butler provides some intrinsic structure to its data repositories, considerably more is left to convention (often encoded in higher-level packages, like obs_base). This document will be - at least at first - a proposal for how to organize data repositories in detail, focusing on collection naming conventions, filesystem locations, and developer workflows. The immediate focus will be the environment at NCSA, but it is hoped that much of this will hold for the IDF and USDF as well.
After consultation with other stakeholders and ultimately RFC, at least some of the content here should probably be moved to the DM Developer Guide.
Links:
- Publication URL: https://dmtn-167.lsst.io
- Alternative editions: https://dmtn-167.lsst.io/v
- GitHub repository: https://github.com/lsst-dm/dmtn-167
- Build system: https://github.com/lsst-dm/dmtn-167/actions/
You can clone this repository and build the technote locally with Sphinx:
git clone https://github.com/lsst-dm/dmtn-167
cd dmtn-167
pip install -r requirements.txt
make html
Note
In a Conda environment, pip install -r requirements.txt
doesn't work as expected.
Instead, pip
install the packages listed in requirements.txt
individually.
The built technote is located at _build/html/index.html
.
You can edit the index.rst
file, which is a reStructuredText document.
The DM reStructuredText Style Guide is a good resource for how we write reStructuredText.
Remember that images and other types of assets should be stored in the _static/
directory of this repository.
See _static/README.rst
for more information.
The published technote at https://dmtn-167.lsst.io will be automatically rebuilt whenever you push your changes to the master
branch on GitHub.
This technote's metadata is maintained in metadata.yaml
.
In this metadata you can edit the technote's title, authors, publication date, etc..
metadata.yaml
is self-documenting with inline comments.
The bibliography files in lsstbib/
are copies from lsst-texmf.
You can update them to the current lsst-texmf versions with:
make refresh-bib
Add new bibliography items to the local.bib
file in the root directory (and later add them to lsst-texmf).