Skip to content

Ska3 (2026.1 with Python 3.13) testing

Jump to bottom Edit New page
Jean Connelly edited this page Mar 3, 2026 · 20 revisions

Summary

This document describes testing plans and results for the transition to 2026.1, with Python 3.13 and updates to most of the core packages such as NumPy and Astropy. See last year's

Core functionality (Ska3 team responsibility)

Core testing is done via the Ska3 integration testing package which consists of running available package unit tests and regression test scripts using machinery in the testr package. This is done with the run_testr command, where some tests may be skipped on standalone platforms.

Current results for 2026.1

See the test outputs are shown in:

FOT testing

We probably need to include testing reminders during the public test phase on GRETA:

  • To run the CCDM software from the test environment
  • To run fotmp notebooks
  • To confirm expected behavior on chimchim with regard to notebooks and any apps used by the engineers.
  • To confirm thermal trending can make reports and plots (2026.1 included an update to kaleido that requires chromium engine not installed on GRETA)
  • To ask FOT thermal to test out xija fitting.

GRETA functionality

Date:

Confirm jupyter notebook/lab works on chimchim via local browser.

Load Review Checks

Date: 29-Jan-2026

  • Load review tools pass all tests (unit/regression)
  • sparkles/proseco/hopper pass all unit tests.
  • starcheck - no substantive diffs seen in year of schedules used as regression set in https://icxc.cfa.harvard.edu/aspect/test_review_outputs/starcheck/hope/ (the updated starcheck in test includes one PR that creates small expected diffs in the outputs).
  • Do a functional test to confirm that interactive plotting works from ipython on OSX (arm64) and Linux.
JC confirmed interactive plotting with macosx and qt backends on OSX arm64 and tkagg backend on Linux - 28-Jan-2026

Utilities

aca_view

Date: 02-Feb-2026

Tested following most of the examples listed in Aca-view's wiki. Found some issues and reported them in the repository. All the new issues are already present in the flight version and are not major issues. For this reason I don't think it is ok to proceed.

find_attitude

Date: 30-Jan-2026

Tested on /proj/sot/ska3/aca-test . The DeprecationWarning is OK in this case - will do an update that is not a lien.

In [1]: import find_attitude

In [2]: find_attitude.test()
============================================================================ test session starts ============================================================================
platform linux -- Python 3.13.11, pytest-9.0.2, pluggy-1.6.0
rootdir: /proj/sot/ska3/aca-test/lib/python3.13/site-packages
plugins: anyio-4.12.1, timeout-2.4.0, typeguard-4.3.0
collected 24 items

find_attitude/tests/test_find_attitude.py ........................                                                                                                    [100%]

============================================================================= warnings summary ==============================================================================
find_attitude/tests/test_find_attitude.py: 618 warnings
  /proj/sot/ska3/aca-test/lib/python3.13/site-packages/find_attitude/find_attitude.py:443: DeprecationWarning: `in1d` is deprecated. Use `np.isin` instead.
    ok = np.in1d(ap["pix0"], constraints.healpix_indices) | np.in1d(

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=============================================================== 24 passed, 618 warnings in 493.87s (0:08:13) ================================================================
Out[2]: False

In [3]: find_attitude.__version__
Out[3]: '3.5.2'

Ska3 cron processing

acdc

Date: 30-Jan-2026

Run with /proj/sot/ska3/aca-test not /proj/sot/ska3/test .

c3po-v$ cd /tmp
c3po-v$ mkdir -p acdc/regress_hope/VC2
# Get enough data to make the last two reports
c3po-v$ cp /dsops/GOT/input/2025_[23]*VC2* acdc/regress_hope/VC2
c3po-v$ acdc_process_vc2_to_l0 --vc2-root=acdc/regress_hope/VC2 --data-root=acdc/regress_hope
c3po-v$ acdc_process_l0_to_quads --data-root=acdc/regress_hope --start 2025:290
# At this step manually edit vc2_index.dat to just have entries after 2025:298
# and mark any cals FAIL to match flight
c3po-v$ acdc_process_quads_to_cals --data-root=acdc/regress_hope
c3po-v$ acdc_make_reports --data-root=acdc/regress_hope
mkdir /proj/sot/ska/www/ASPECT_ICXC/test_review_outputs/acdc/hope
cp -Ruva acdc /proj/sot/ska/www/ASPECT_ICXC/test_review_outputs/acdc/hope
cd /proj/sot/ska/www/ASPECT_ICXC/test_review_outputs/acdc/hope
rm -R acdc/regress_hope/VC2
rm -R acdc/regress_hope/2025/

Checks:

Note on comparing outputs:

The temperature values assigned to cals vary compared to flight. This is likely because flight processing is done soon after the cal and can be using available MAUDE telemetry from realtime. This will generate differences in temperatures and mask values because the temperatures are not available during the VC2 readout interval.

Diffs:

For the 2025:339 cal the diffs are:


In [1]: from astropy.io import fits
In [2]: dat1 = fits.open('/proj/sot/ska/data/acdc/cals/flight/2025/339/cal_2025_339_02_43_12.fits.gz')[0].data
In [5]: dat2 = fits.open('/proj/sot/ska/www/ASPECT_ICXC/test_review_outputs/acdc/hope/acdc/regress_hope/cals/flight/2025/339/cal_2025_339_02_43_12.fits.gz')[0].data
In [6]: diff = dat1 - dat2
In [8]: import numpy as np
In [9]: argmax = np.argmax(np.abs(diff)).flatten()[0]

In [10]: print(
    ...:     f"Mean diff: {np.mean(diff)}",
    ...:     f"Diff std dev: {np.std(diff)}",
    ...:     f"Max diff: {np.max(np.abs(diff))} ({dat1.ravel()[argmax]}, {dat2.ravel()[argmax]})",
    ...: )
Mean diff: -0.05555091053247452 Diff std dev: 1.1226611137390137 Max diff: 47.55810546875 (7954.017578125, 7906.45947265625)

acis_taco

Date: 28-Jan-2026

Reprocess current week (JAN2726.pkl) with Ska3-flight and 2026.1:

cd /tmp
mkdir acis_taco
cd acis_taco
mkdir flight
mkdir hope
/proj/sot/ska3/flight/bin/skare /proj/sot/ska3/flight/share/acis_taco/make_esaview_data.py --nweeks=1 --data-root=flight
/proj/sot/ska3/test/bin/skare /proj/sot/ska3/test/share/acis_taco/make_esaview_data.py --nweeks=1 --data-root=hope

Then from ipython

import pickle
import numpy as np                                                                                                 
dat1 = pickle.load(open('/tmp/acis_taco/flight/JAN2726.pkl', 'rb'))                                         
dat2 = pickle.load(open('/tmp/acis_taco/hope/JAN2726.pkl', 'rb')) 
np.all(dat1['illums'] == dat2['illums'])  # True
np.all(dat1['times'] == dat2['times'])  # True

Result:

  • OK

arc

Date: 29-Jan-2026

Setup:

# 

rm /proj/sot/ska3/test/data/arc3
# Make a new dir for hope testing
mkdir /proj/sot/ska3/test/data/arc3
# Copied the /proj/sot/ska/data/arc3 contents into that directory
rsync -aruvz /proj/sot/ska/data/arc3/* /proj/sot/ska3/test/data/arc3/

ln -s /proj/sot/ska/www/ASPECT/arc3_hope_test /proj/sot/ska3/test/www/ASPECT/arc3


cd ~/git/arc
git checkout 4.10.3
# Install as aca user
source /proj/sot/ska3/test/bin/ska_envs.csh
make -n install # check paths 
make install

Run task pieces by hand to look for errors at the console:

jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/get_iFOT_events.pl
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/get_web_content.pl
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/get_solar_flare_png.py --image-cache-dir /proj/sot/ska3/test/data/arc3/solar_flare_image_cache --out-file /proj/sot/ska3/test/www/ASPECT/arc3/solar_flare.png
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/get_goes_x.py --h5=/proj/sot/ska3/test/data/arc3/GOES_X.h5
Downloading https://hpiers.obspm.fr/iers/bul/bulc/Leap_Second.dat
|=================================================================================================| 1.3k/1.3k (100.00%)         0s
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/plot_goes_x.py --h5=/proj/sot/ska3/test/data/arc3/GOES_X.h5 --out=/proj/sot/ska3/test/www/ASPECT/arc3/goes_x.png
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/get_ace.py --h5=/proj/sot/ska3/test/data/arc3/ACE.h5
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/get_hrc.py --h5=/proj/sot/ska3/test/data/arc3/hrc_shield.h5 --data-dir=/proj/sot/ska3/test/data/arc3
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/plot_hrc.py --h5=/proj/sot/ska3/test/data/arc3/hrc_shield.h5 --out=/proj/sot/ska3/test/www/ASPECT/arc3/hrc_shield.png
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/make_timeline.py --data-dir=/proj/sot/ska3/test/data/arc3
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/arc.pl 
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/arc.pl -config arc3:arc_ops
jeanconn-c3po-v% /proj/sot/ska3/test/share/arc3/arc_time_machine.pl

Examined output at https://cxc.cfa.harvard.edu/mta/ASPECT/arc3_hope_test/ and checked:

  • HTML has right dates
  • snapshot content appears correct
  • plots match flight bye eye

cheta

Date: 29-Jan-2026

  • cheta_sync from 2026.1 ran with no errors from a laptop.

Manual run of regression test script:

jeanconn-kady> sh regression_test_eng_archive.sh
Updating archive, do 'tail -f test/make_eng_archive.log' in another window
Fetch file is /proj/sot/ska3/aca-test/lib/python3.13/site-packages/cheta/fetch.py
ENG_ARCHIVE is None
Getting 1crat
Getting fptemp_11
Getting orbitephem0_x
Getting sim_z
Getting tephin
Getting cvcductr
Getting dp_dpa_power
fetch: using ENG_ARCHIVE=/proj/sot/ska/jeanproj/git/cheta/test/eng_archive for archive path
Fetch file is /proj/sot/ska/jeanproj/git/cheta/test/../cheta/fetch.py
ENG_ARCHIVE is /proj/sot/ska/jeanproj/git/cheta/test/eng_archive
Getting 1crat
Getting fptemp_11
Getting orbitephem0_x
Getting sim_z
Getting tephin
Getting cvcductr
Getting dp_dpa_power
[OK] 1crat dat
[OK] 1crat dat
[OK] 1crat dat5
[OK] 1crat dat5
[OK] 1crat dat5
[OK] 1crat dat5
[OK] 1crat dat5
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] 1crat datd
[OK] fptemp_11 dat
[OK] fptemp_11 dat
[OK] fptemp_11 dat5
[OK] fptemp_11 dat5
[OK] fptemp_11 dat5
[OK] fptemp_11 dat5
[OK] fptemp_11 dat5
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] fptemp_11 datd
[OK] orbitephem0_x dat
[OK] orbitephem0_x dat
[OK] sim_z dat
[OK] sim_z dat
[OK] sim_z dat5
[OK] sim_z dat5
[OK] sim_z dat5
[OK] sim_z dat5
[OK] sim_z dat5
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] sim_z datd
[OK] tephin dat
[OK] tephin dat
[OK] tephin dat5
[OK] tephin dat5
[OK] tephin dat5
[OK] tephin dat5
[OK] tephin dat5
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] tephin datd
[OK] cvcductr dat
[OK] cvcductr dat
[OK] cvcductr dat5
[OK] cvcductr dat5
[OK] cvcductr dat5
[OK] cvcductr dat5
[OK] cvcductr dat5
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
[OK] cvcductr datd
All OK: True

Regression and unit tests provide full coverage of package user functionality and daily cron updates.

kadi

No extra tests. Regression and unit tests provide full coverage of package user functionality and daily cron updates.

arc5gl

Date: 28-Jan-2026

  • confirm cmdline works fine in test environment to fetch files

mica

No extra tests. Regression and unit tests provide full coverage of package user functionality and daily cron updates.

skawatch3

Date:

  • jobwatch unit tests pass
  • skawatch.py and hourly_watch.py run without error in the test environment.
  • html outputs look reasonable to visual inspection
  • OK

task_schedule

Date: 30-Jan-2026

  • task_schedule3.pl ran without errors

I ran task schedule to run the test arc3 job for a day and confirmed the test page (https://cxc.cfa.harvard.edu/mta/ASPECT/arc3_hope_test/) updated and the logs looked reasonable.

xija

Date: 28-Jan-2026

  • xija_gui_fit at xija version 4.36.0 was confirmed to successfully start and run a fit.
Add a custom footer

For Users

For Maintainers

Continuous Integration

Old Pages

Clone this wiki locally