This page contains information about the whole code stored in the gamma-cat repository.
There is a command line interface to run the gamma-cat scripts, the make.py
file in the top-level folder.
To see the available sub-commands and options:
$ ./make.py --help
To run the full pipeline, i.e. generate all output files and run all checks:
$ ./make.py all
After adding/ changing data in the input folder, one should always execute:
$ ./make.py checks
which checks the format/ structure of the input files.
The make.py
command line interface just imports and executes functions and classes from the gammacat
Python package (i.e. the .py
files in the folder with name gammacat
).
We list the modules in gammacat
and comment on the code organisation.
The following are more basic modules:
utils.py
has some helper utility functions (e.g. for JSON / YAML / ECSV I/O)modeling.py
has aParameter
andParameterList
class to help process input spatial and spectral source models from the YAML files.info.py
has some helpers for versions, filenames, ...sed.py
has a class to process and validate the spectral energy distributions (SEDs) in the input folder. The SEDs in the output folder can be read directly withgammapy.spectrum.FluxPoints
.lightcurve.py
has a class to process and validate the lightcurves in the input folder. The lightcurves in the output folder can be read directly withgammapy.time.LightCurve
.
In additions there are classes in gammapy.catalog.gammacat
that are used in the gammacat
scripts to process the data: GammaCatResource
, GammaCatResourceIndex
, GammaCatDatasetCollection
.
Then there is a hierarchy of higher-level modules (that import from the basic modules and modules representing lower-level steps in the processing pipeline):
input.py
has classes to read / clean up / process the data in theinput
folder.collection.py
has classes to create the files in theoutput
folder (only the dataset files and index files, not the catalog files).cat.py
is the code to create the catalog fileschecks.py
is the code to run checks. At the moment the methods there just dispatch to methods calledvalidate
orcheck
in lower-level modules (such asgammacat.input
), and the actual checks are thus scattered throughout thegammacat
modules. There's also checks on data content ingammacat/tests
(which is probably a bad idea, but pytest is convenient to have asserts)
There is a folder gammacat/tests
with some unit tests for the code in the gammacat
package, that can be executed via python -m pytest gammacat/tests
We don't have the relation quite figure out, what goes where:
gammacat/tests
gammapy/catalog/tests/test_gammacat.py
- The various check / validate methods throughout
gammacat
and executed via./make.py check
.
The following tool is helpful to lint YAML files:
TODO: it's too picky, showing errors for things that are OK. Figure out how to make it less picky and document that here.
The gamma-cat website is a static website generated by Python and Sphinx.
We have a Sphinx test page where we can try out things locally and check if they also work on ReadTheDocs. It's an orphan page, i.e. doesn't show up for normal users.
We use several Sphinx extensions, and also have our own in gammacat/sphinx/exts.
More info soon ... for now this is just a link collection:
sphinxcontrib.rawfiles
- Useful example. Not very useful directly, because can't control destination of the copy!?
- Code: https://bitbucket.org/birkenfeld/sphinx-contrib/src/master/rawfiles/sphinxcontrib/rawfiles.py
- PyPI: https://pypi.python.org/pypi/sphinxcontrib-rawfiles/
- Although for now, this is working for what I need: http://www.sphinx-doc.org/en/1.5.1/config.html#confval-html_extra_path
- Looks very useful! http://sphinxcontribdatatemplates.readthedocs.io/en/latest/
- A good Sphinx table extension would be useful: sphinx-doc/sphinx#786
- Not sure if this is useful! https://pythonhosted.org/sphinxcontrib-restbuilder/
- Maybe: https://pypi.python.org/pypi/sphinxcontrib-jsoncall
- Looks interesting, probably not useful here: https://jsdoc-toolkit-rst-template.readthedocs.io/en/latest/