Refresh regression testing and fix ruff / improve variable naming

[taldcroft]

Description

This refreshes the basic regression testing of cheta/update_archive.py which is run on HEAD. The existing scripts and documentation were out of date, including several that only existed as legacy Python 2 scripts. The initial driver was to test ruff updates to that module, but having functional regression testing is independently important.

This also fixes a few ruff warnings. While in the code, I took the opportunity to document a couple of key functions and use modern standards for naming variables. The renaming was done using VS code so there is high confidence in correctness. Nevertheless there is no coverage of these changes in unit tests so I will run the manual regression testing procedure.

This should have been two PR's but that isn't the way things happened...

Interface impacts

None

Testing

Unit tests

• Linux

ska3-kady$ git rev-parse --short HEAD
264d50a

ska3-kady$ pytest 
============================================================= test session starts =============================================================
platform linux -- Python 3.11.8, pytest-7.4.4, pluggy-1.4.0
rootdir: /data/baffin/tom/git/cheta
plugins: anyio-4.3.0, timeout-2.2.0
collected 174 items                                                                                                                           

cheta/tests/test_comps.py ............................................................                                                  [ 34%]
cheta/tests/test_data_source.py .........                                                                                               [ 39%]
cheta/tests/test_fetch.py ................................                                                                              [ 58%]
cheta/tests/test_intervals.py .........................                                                                                 [ 72%]
cheta/tests/test_orbit.py .                                                                                                             [ 72%]
cheta/tests/test_remote_access.py ......                                                                                                [ 76%]
cheta/tests/test_sync.py ........                                                                                                       [ 81%]
cheta/tests/test_units.py ...........                                                                                                   [ 87%]
cheta/tests/test_units_reversed.py ...........                                                                                          [ 93%]
cheta/tests/test_utils.py ...........                                                                                                   [100%]
================================================ 174 passed, 335 warnings in 127.56s (0:02:07) ================================================

Independent check of unit tests by Jean

• Mac:

(ska3-flight-latest) flame:cheta jean$ pytest
================================================================================== test session starts ===================================================================================
platform darwin -- Python 3.11.8, pytest-8.0.2, pluggy-1.4.0
PyQt5 5.15.9 -- Qt runtime 5.15.8 -- Qt compiled 5.15.8
rootdir: /Users/jean/git
configfile: pytest.ini
plugins: astropy-0.11.0, qt-4.4.0, cov-5.0.0, timeout-2.2.0, remotedata-0.4.1, anyio-4.3.0, filter-subpackage-0.2.0, doctestplus-1.2.1, astropy-header-0.2.2, hypothesis-6.112.0, arraydiff-0.6.1, mock-3.14.0
collected 174 items                                                                                                                                                                      

cheta/tests/test_comps.py ............................................................                                                                                             [ 34%]
cheta/tests/test_data_source.py .........                                                                                                                                          [ 39%]
cheta/tests/test_fetch.py ................................                                                                                                                         [ 58%]
cheta/tests/test_intervals.py .........................                                                                                                                            [ 72%]
cheta/tests/test_orbit.py .                                                                                                                                                        [ 72%]
cheta/tests/test_remote_access.py ......                                                                                                                                           [ 76%]
cheta/tests/test_sync.py ........                                                                                                                                                  [ 81%]
cheta/tests/test_units.py ...........                                                                                                                                              [ 87%]
cheta/tests/test_units_reversed.py ...........                                                                                                                                     [ 93%]
cheta/tests/test_utils.py ...........                                                                                                                                              [100%]

==================================================================================== warnings summary ====================================================================================
cheta/cheta/tests/test_comps.py::test_cmd_states
  /Users/jean/miniforge3/envs/ska3-flight-latest/lib/python3.11/site-packages/setuptools_scm/git.py:308: UserWarning: git archive did not support describe output
    warnings.warn("git archive did not support describe output")

cheta/cheta/tests/test_intervals.py::test_fetch_MSID_intervals
  /Users/jean/miniforge3/envs/ska3-flight-latest/lib/python3.11/site-packages/django/utils/encoding.py:266: DeprecationWarning: 'locale.getdefaultlocale' is deprecated and slated for removal in Python 3.15. Use setlocale(), getencoding() and getlocale() instead.
    encoding = locale.getdefaultlocale()[1] or 'ascii'

cheta/cheta/tests/test_intervals.py::test_fetch_MSID_intervals
  /Users/jean/miniforge3/envs/ska3-flight-latest/lib/python3.11/site-packages/django/http/request.py:1: DeprecationWarning: 'cgi' is deprecated and slated for removal in Python 3.13
    import cgi

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
====================================================================== 174 passed, 3 warnings in 115.41s (0:01:55) =======================================================================
(ska3-flight-latest) flame:cheta jean$ git rev-parse HEAD
dd2940a6d2878561bca14c7698ca3562a1f88817

Functional tests

Regression testing on HEAD by using the script regression_test_eng_archive.sh:

• No unexpected warnings in processing.
• No errors in processing.
• Regression comparison of test to flight values (test/compare_regr_vals.py) all OK.

[Copilot]

Copilot reviewed 5 out of 13 changed files in this pull request and generated no suggestions.

Files not reviewed (8)

• NOTES/NOTES.test: Language not supported
• make_test_eng_archive.csh: Language not supported
• make_test_eng_archive.sh: Language not supported
• pyrightconfig.json: Language not supported
• regression_test_eng_archive.sh: Language not supported
• .pre-commit-config.yaml: Evaluated as low risk
• cheta/tests/test_orbit.py: Evaluated as low risk
• cheta/update_archive.py: Evaluated as low risk

Comments skipped due to low confidence (1)

test/make_plots.py:4

• The comment should be 'sanity check' instead of 'sanity check'.

# Make a few plots for a sanity check of test archive correctness