Skip to content

Fix sign of poisson noise in preprocess.poisson_noise_via_data_eps_from#290

Merged
Jammy2211 merged 1 commit intomainfrom
feature/preprocess-poisson-noise-sign-fix
Apr 27, 2026
Merged

Fix sign of poisson noise in preprocess.poisson_noise_via_data_eps_from#290
Jammy2211 merged 1 commit intomainfrom
feature/preprocess-poisson-noise-sign-fix

Conversation

@Jammy2211
Copy link
Copy Markdown
Collaborator

Summary

autoarray.preprocess.poisson_noise_via_data_eps_from returned the wrong sign of the Poisson noise term. It computed data_eps − noisy_eps instead of noisy_eps − data_eps, so data_eps_with_poisson_noise_added(...) and SimulatorImaging.via_image_from(..., add_poisson_noise_to_data=True) produced 2·data − noisy_data rather than noisy_data. Same mean and variance as a real Poisson sample but with mirrored skew (negative instead of positive), which biases inference on simulated low-count imaging.

Surfaced by /health_check on 2026-04-27.

API Changes

Behaviour-only change to two existing functions (signatures unchanged):

  • preprocess.poisson_noise_via_data_eps_from now returns the true Poisson noise term (noisy − data) instead of its negation.
  • preprocess.data_eps_with_poisson_noise_added and SimulatorImaging.via_image_from(..., add_poisson_noise_to_data=True) therefore produce data with the correct Poisson distribution and skew — same seed, different numerical values.

Implementation also routes the result through data_eps.with_new_array(...) so the return type is an Array2D (not a raw np.ndarray), which the previous in-progress edit had inadvertently changed.

Any code asserting against specific seeded outputs of these functions will need expected values regenerated. Anything using the result as a noisy realisation (modelling, inference) will now see correctly-skewed samples — typically more accurate, not less.

See full details below.

Test Plan

  • pytest test_autoarray/dataset/test_preprocess.py test_autoarray/dataset/imaging/test_simulator.py — 34/34 green
  • Full pytest test_autoarray/ — 748/748 green
  • Workspace smoke tests across PyAutoFit/Galaxy/Lens workspaces (run on PR via /smoke_test)
Full API Changes (for automation & release notes)

Changed Behaviour

  • autoarray.preprocess.poisson_noise_via_data_eps_from(data_eps, exposure_time_map, seed=-1) — sign of returned noise corrected. Was data_eps − noisy_eps, now noisy_eps − data_eps. Result is now correctly typed Array2D (matches input).
  • autoarray.preprocess.data_eps_with_poisson_noise_added(...) — downstream consequence. Result is now a true Poisson realisation of data_eps rather than its mirror image around the mean.
  • autoarray.SimulatorImaging.via_image_from(...) — when add_poisson_noise_to_data=True, simulated data and (when include_poisson_noise_in_noise_map=True) the noise map both shift to correct Poisson statistics.

Migration

  • No code changes required. If you have tests that pin specific values for these functions under a fixed seed, re-derive expected values against the fixed code.

🤖 Generated with Claude Code

The function returned data_eps - noisy_eps, which made
data_eps_with_poisson_noise_added produce 2*data - noisy_data
(noise reflected around the mean — same magnitude, mirrored skew).
Poisson is positively skewed; the buggy version had negative skew,
biasing inference on simulated low-count imaging.

Now returns noisy_eps - data_eps so that data + noise = noisy_data.
Also wraps the result back into the input Array2D type via
with_new_array(), preventing the numpy/Array2D dispatch from
returning a raw ndarray.

Test fixtures regenerated under the same seeds.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Jammy2211 Jammy2211 added the pending-release PR queued for the next release build label Apr 27, 2026
@Jammy2211 Jammy2211 merged commit 3ec01f0 into main Apr 27, 2026
4 checks passed
@Jammy2211 Jammy2211 deleted the feature/preprocess-poisson-noise-sign-fix branch April 27, 2026 09:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pending-release PR queued for the next release build

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant