Validate kadi states

[taldcroft]

Description

This is the modern Py3 version of validate_states. Instead of updating that project and making a new Ska3 package, this PR makes a new sub-package of kadi (with a corresponding app) to handle state validation.

This requires sot/cheta#242.

Known issues / To do

• Add per-class HTML generation for use in other apps
• Config item to allow kadi commands to use In-work command events
• Test with telemetry gap during comm (mix of back-orbit and realtime)
  • Add ability to mock a gap (probably in cheta or maybe maude)

Optional

• Roll error > 4 deg 2022:273:15:55:40.253 - 2022:273:17:11:26.128 (will be fixed by update to Ska.Sun.position())
• Commanded attitude validation not implemented
• ACIS power not implemented
• Web-kadi validation app

Interface impacts

This will supersede the Py2 validate_states app.

Testing

Unit tests

• No unit tests
• Mac
• Linux
• Windows

Independent check of unit tests by Jean. I ran with sot/cheta#242, https://github.com/sot/maude/pull/44, sot/cxotime#31 in my path.

• Linux

Functional tests

Run over full 2022:293 IU-reset and Safe mode

python -m kadi.scripts.validate_states --days=7 --stop=2022:297

This gives the expected results.

Run within 2022:293 IU-reset and Safe mode (at day 295:0000z)

This creates a data gap corresponding to being in a real-time comm after an 8-hour gap.

env CHETA_FETCH_DATA_GAP="--exclude=*ephem?_* --start=2022:294:15:00:00 --stop=2022:294:23:00:00" \
python -m kadi.scripts.validate_states --days=3 --stop=2022:295

This gives the expected results.

[javierggt]

• I went over the code and it seems fine. There are only two minor comments.
• I successfully ran unit tests.
• I ran the validate script, but the maude.config submodule is not there, so I had to manually comment it out (the file seems to be missing in maude's PR #44).

I noticed that the setup.py still lists all the templates, so I just want to point out that those templates were removed. Currently, the only templates in kadi are the ones added by this PR.

If this is implemented in the kadi web server, most likely the templates will not be used, because we will want to use the kadi style. I suppose it is ok to leave them here, because this is currently a standalone application.

[javierggt]

this argument does not exist

[taldcroft]

Fixed

[javierggt]

This is possible with plotly figures:

json.dumps(fig, indent=2)

so in principle one can always recreate the figure from the json. That means the web app has the choice of rendering the html in the server or sending the json and rendering in the browser.

One option for a web page could be to request each validation plot separately in json and load them as they are done, but it might not be a good idea because there might be repeated calls to maude spread over several python instances.

[taldcroft]

I noticed that the setup.py still lists all the templates, so I just want to point out that those templates were removed. Currently, the only templates in kadi are the ones added by this PR.

Fixed.

[taldcroft]

If this is implemented in the kadi web server, most likely the templates will not be used, because we will want to use the kadi style. I suppose it is ok to leave them here, because this is currently a standalone application.

Yes, we can cross that bridge later with the app. Likewise the interesting idea about returning the validation plot as JSON in order to have a more responsive app.