Skip to content

v3.2.0 - Flux Calibration, Corrected Tables and Documentation

Choose a tag to compare

@m-samland m-samland released this 02 Nov 15:03
· 11 commits to main since this release
42b1c4b

spherical 3.2.0 corrects the observation tables, makes flux calibration saturation-aware, and adds a documentation site. Full detail is in CHANGELOG.md.

Before you upgrade

Upgrade spherical first, then sync the tables. Versions before 3.2.0 cannot detect the new tables, so a 3.0.x or 3.1.0 sync without --force skips the changed files and keeps v3.0.0. After upgrading, spherical-sync-tables replaces previously synced tables by itself. Add --force only to overwrite tables you rebuilt locally.

Re-run existing reductions. Reductions now use daytime backgrounds by default and a corrected inverse variance on both instruments. Re-run IRDIS from irdis_calibration and IFS from extract_cubes.

pip uninstall trap  # pip installs only: trap-hci provides the same package
pip install --upgrade "spherical[pipeline] @ git+https://github.com/m-samland/spherical.git@v3.2.0"
spherical-sync-tables --dest ~/data/sphere/database

With pixi, run pixi install.

New database tables

The tables are on Zenodo as version v3.2.0, 10.5281/zenodo.23884922, complete to 2026-10-30.

Observation table v3.0.0 rows v3.2.0 rows Set aside in the new "other" table
IFS 4733 4384 135
IRDIS 6101 5585 210
IRDIS polarimetry 1205 1155 48
IFS-SAM 152 152 0
IRDIS-SAM 203 203 0
  • Observation tables hold only stellar high-contrast targets. Solar-system bodies, galaxies, other non-stellar targets and sequences with no matching star move to table_of_other_observations_*.fits, where a CATEGORY column gives the reason (#224).
  • Each science file belongs to one target. Rows of v3.0.0 that repeated the same frames under several stars are gone, which removes further rows beyond those set aside. Other catalogued stars in the field are in the new FIELD_TARGETS column, and a name lookup for such a star returns that row (#224, #214).
  • FLUX frames no longer compete for PRIMARY_SCIENCE. Rows labelled FLUX fall from 275 to 74 (IFS) and from 370 to 81 (IRDIS), and the remaining ones are sequences with only flux frames (#179, reported by @tomasstolker).
  • ROTATION is corrected. It no longer jumps by 360° when the sidereal time wraps, which affected 110 sequences of v3.0.0, and field-tracking sequences now report about 0 (#167, reported by @tomasstolker; #187).
  • Mixed flux DIT no longer blocks HCI_READY. The new columns FLUX_ND_FLAG and FLUX_DIT_SPREAD report mixed setups (#173).

No columns were removed. SIMBAD queries for the table update went through the Harvard mirror.

Seven rows switch reduction path

These short sequences have CORO frames but more CENTER exposure time, so WAFFLE_MODE changes from False to True. If you reduced one of them with an earlier version, re-run it with force={"process_extracted_centers"}, which also repeats the steps after it. A resume alone can leave stale centres and TRAP results without an error (#179).

Instrument Target Night Filter OBS_ID
IFS HD 108639 2015-05-26 OBS_YJ 200356914
IFS RX J1844.5-3723 2018-07-04 OBS_YJ 1962012
IFS V* AF Lep 2022-12-15 OBS_H 3379336
IRDIS HD 7199 2016-11-16 DB_H23 200410077
IRDIS HD 35974 2026-01-11 BB_Ks 4712909
IRDIS RX J1844.5-3723 2018-07-04 BB_H 1962012
IRDIS V* AF Lep 2022-12-15 DB_K12 3379336

Reduction changes

  • Daytime backgrounds by default. IRDIS and IFS use the daytime DARK,BACKGROUND calibrations and fall back to the sequence's SKY frames only when none match. Pass prefer_sky=True to retrieve_observation_metadata to use SKY frames first (#250).
  • Corrected inverse variance. IRDIS {type}_ivar_cube.fits was up to 20× too high in background-limited pixels, most of all in K band (#239). IFS needs charis 2.3.0 for the same reason, and spherical now installs it (charis-dep#62).
  • Saturated flux cubes are left out of the PSF reference. IRDIS drops cubes whose core reaches flux_saturation_adu and warns above flux_nonlinearity_adu. Each decision is written to additional_outputs/flux_cube_selection.csv (#172).
  • Several flux calibration fixes. Sequences crossing LST 0h no longer get a halved flux PSF (#193), and mixed ND filters no longer crash (#171). nd_attenuation.fits changes shape to (n_wave, n_frames).
  • crop_size must be odd and defaults to 257. IRDIS crops right after the background fit, which makes reductions 2 to 3.4× faster and converted/ 3.8× smaller. An even value raises when the config is built (#151).
  • TRAP comes from PyPI as trap-hci>=2.3. The import name is still trap, and temporal reductions run about 1.5 times faster with unchanged results. TRAP detection now stops with an error when a result folder holds detection maps from a reduction with a different number of frames (#208).

New features

  • Documentation site with installation, concepts, how-tos, 51 Eri IFS and IRDIS tutorials and generated references (#196).
  • Template matching for IRDIS broadband. Existing broadband results need force={"run_trap_detection"} (#134).
  • Optional frame alignment. The opt-in align_frames step writes cubes with the star on the centre pixel (#161, #207).
  • Forced extraction at known positions. The opt-in run_trap_forced_extraction step writes contrast spectra at configured positions, and run_trap_on_observations accepts a function returning the TRAP config per observation.
  • Multi-epoch target selection through database.multi_epoch_filter.select_multi_epoch_targets().
  • Cleanup by stage. cleanup_pipeline_products(..., stages=...) replaces the clean_raw, clean_extracted and clean_wavecal flags, and it now works for IRDIS (#218).

Citing

Cite Samland et al. (2025) for the package, and the version DOI 10.5281/zenodo.23884922 for the v3.2.0 database tables. If you use the enrichment columns, please also cite Gaia DR3 and MOCAdb (Gagné et al. 2026).