Skip to content

fix: relabel at-or-below-cap data at the capped pixel scale - #431

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/small-datasets-loader-pixel-scales
Aug 3, 2026
Merged

fix: relabel at-or-below-cap data at the capped pixel scale#431
Jammy2211 merged 1 commit into
mainfrom
feature/small-datasets-loader-pixel-scales

Conversation

@Jammy2211

Copy link
Copy Markdown
Collaborator

Summary

cap_array_2d_for_small_datasets handled one case and silently dropped the other. Data larger than the 16x16 PYAUTO_SMALL_DATASETS cap was cropped and rebuilt at SMALL_DATASETS_PIXEL_SCALES (0.6); data already at-or-below the cap early-returned, keeping the caller's uncapped pixel_scales (0.1). A capped simulator writes its data at 0.6, so the loader mislabelled the frame 6x — ±0.8" instead of ±4.8" for a 16x16 field.

Off-centre galaxies then fell outside the mislabelled frame, their non-negative linear intensity solve correctly returned exactly 0.0, total_luminosity became 0, and min(5 * 0.5 * 0**0.6, 5.0) collapsed a UniformPrior to lower == upper == 0.0. The PriorException surfaced four steps downstream in autolens_workspace/scripts/group/slam.py:321, naming neither the loader nor the pixel scale.

The at-or-below-cap branch now rebuilds the Array2D at the capped scale, mirroring the crop branch. Shape is preserved — that branch must never crop. Rebuilding is required rather than returning a corrected scalar: the Array2D is constructed by the caller before the call and carries its own geometry.

Fixes #430. Fixes the group/slam.py + group/slam.ipynb failures in PyAutoHeart Workspace Smoke run 30790463134.

API Changes

No signature changes. One behaviour change, confined to runs with PYAUTO_SMALL_DATASETS=1:
autoarray.util.dataset_util.cap_array_2d_for_small_datasets now returns SMALL_DATASETS_PIXEL_SCALES (0.6) and a rebuilt Array2D for input already at-or-below the cap, where it previously returned the caller's pixel_scales and the input array untouched. Shape is unchanged in that branch. Reached in practice through Imaging.from_fits, which routes data and noise_map (not psf) through it.

With the cap unset — every normal run — behaviour is byte-identical, and the crop branch is untouched.

See full details below.

Test Plan

  • pytest test_autoarray/ — 929 passed
  • scripts/group/slam.py (autolens_workspace) under the capped smoke profile, cleared dataset + output → exit 0, all six SLaM stages genuinely run (0 cached resumes; note output is namespaced under output/test_mode/)
  • Crop path unchanged — 209x209 cosmos_web_ring gives (16,16) @ 0.6 identically before and after
  • Normal uncapped operation unaffected — scripts/imaging/start_here.py (ENV: full_datasets) exit 0
  • Committed datasets audited: of 31 committed dataset FITS, exactly one is at-or-below the cap (slacs1430+4105/psf.fits, 11x11) and PSFs never route through the capper — so no committed dataset changes scale

Downstream effect on two parked scripts

  • imaging/features/scaling_relation/slam → now exit 0 (6 real searches). Its NEEDS_FIX park in autolens_workspace/config/build/no_run.yaml can be removed in a separate workspace PR.
  • multi_galaxy/features/scaling_relation/slam → gets past the 0.0-luminosity cause (real luminosities now measured) but then hits a separate latent script bug: slam.py:863 computes image_half_width from the script's own hardcoded pixel_scale (0.1) while the mask is built from dataset_full.pixel_scales (now correctly 0.6), producing an empty mask. Not a regression — on unpatched main it fails earlier with the documented 0.0-luminosity error. Stays parked; filed separately.
Full API Changes (for automation & release notes)

Changed Behaviour

  • autoarray.util.dataset_util.cap_array_2d_for_small_datasets(array_2d, pixel_scales) — when PYAUTO_SMALL_DATASETS=1 and array_2d.shape_native is at-or-below SMALL_DATASETS_SHAPE_NATIVE, now returns a rebuilt Array2D at SMALL_DATASETS_PIXEL_SCALES and that scale, instead of returning the inputs unchanged. Shape preserved; no cropping in this branch. Unchanged when the env var is unset, and unchanged for above-cap input.

Removed

  • None

Added

  • None

Migration

  • None required. Callers that already consume the returned pixel_scales (the documented contract, and what Imaging.from_fits does) need no change. Code that ignored the returned pixel_scales and reused its own literal was already relying on the buggy path under the cap; it should consume the returned value or read dataset.pixel_scales.

Generated by the PyAutoLabs agent workflow.

cap_array_2d_for_small_datasets handled one case and silently dropped the
other. Data larger than the 16x16 PYAUTO_SMALL_DATASETS cap was cropped AND
rebuilt at SMALL_DATASETS_PIXEL_SCALES (0.6); data already at-or-below the cap
early-returned, keeping the caller's uncapped pixel_scales (0.1). A capped
simulator writes its data at 0.6, so the loader mislabelled the frame 6x —
+/-0.8" instead of +/-4.8" for a 16x16 field.

Off-centre galaxies then fell outside the mislabelled frame, their non-negative
linear intensity solve correctly returned exactly 0.0, and the resulting
collapsed UniformPrior surfaced four steps downstream as a PriorException in
autolens_workspace scripts/group/slam.py, naming neither the loader nor the
pixel scale.

The at-or-below-cap branch now rebuilds the Array2D at the capped scale,
mirroring the crop branch. Shape is preserved — that branch must never crop.
Rebuilding is required rather than returning a corrected scalar: the Array2D
is constructed by the caller before the call and carries its own geometry.

Two unit tests asserted the early return as intended behaviour; both are
rewritten to assert the relabel-without-cropping. The env-unset test and both
crop-path tests are unchanged, guarding the scope of the fix.

Fixes #430

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Aug 3, 2026
@Jammy2211
Jammy2211 merged commit 17885f3 into main Aug 3, 2026
2 checks passed
@Jammy2211
Jammy2211 deleted the feature/small-datasets-loader-pixel-scales branch August 3, 2026 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pending-release PR queued for the next release build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: cap loader mislabels at-or-below-cap data pixel_scales

1 participant