Skip to content

Hybrid solving: SEP full-frame rescue when cedar-detect finds no stars (light-polluted skies) - #566

Draft
hjoungjoo wants to merge 5 commits into
brickbots:mainfrom
hjoungjoo:pr/cedar-sep-hybrid
Draft

Hybrid solving: SEP full-frame rescue when cedar-detect finds no stars (light-polluted skies)#566
hjoungjoo wants to merge 5 commits into
brickbots:mainfrom
hjoungjoo:pr/cedar-sep-hybrid

Conversation

@hjoungjoo

@hjoungjoo hjoungjoo commented Aug 1, 2026

Copy link
Copy Markdown

Problem

Under a heavily light-polluted sky (city core, bright haze), cedar-detect on the processed 512×512 frame finds 0–2 stars per frame and the PiFinder stops solving entirely. The 12→8-bit stretch crushes faint stars sitting on a bright background into a couple of grey levels, and the centred crop throws away half the sensor's sky. This is the everyday sky for a lot of urban PiFinder users — mine included (Seoul).

What this PR does

The production path is unchanged and keeps priority. Every attempt still runs cedar → tetra3 on the processed 512 frame exactly as today. When — and only when — that solve fails:

  1. The camera has published the exposure's uncropped 12-bit raw alongside the normal pipeline (config-gated; zero extra copies when disabled).
  2. A second detector (SEP, the Source Extractor library) runs on it: 2×2 binning, mesh background subtraction (removes the LP gradient / cloud glow that defeats a global threshold), then field-measured quality gates — edge margin (vignette), saturation guard, point-source shape gate, isolation gate, optional static warm-pixel mask (a bundled CLI builds the map from dark frames).
  3. If ≥ 5 stars survive, the attempt is rescued: tetra3 solves the SEP centroids in the rotated full frame, with FOV / Roll / target_pixel all mapped so the solution carries production 512-frame semantics (solver_frame_map, pinned by an equivalence test that solves the same synthetic sky through both paths — Roll agrees to 0.05°, aligned pointing to a fit residual). Downstream code cannot tell which detector solved.
  4. Failed rescues back off exponentially (a failed fallback burns up to solve_timeout; indoors/under cloud that would recur every attempt), re-arming instantly when the SEP count jumps 1.5× — a cloud gap opening on stars is never delayed.

Both config keys (solver_sep_fallback, solver_shadow_detect) default to off; with them off, no SEP code runs and no full frame is published. There is also a shadow-logging mode that only writes a per-attempt comparison CSV, which is how the path was validated in the field before being allowed to solve.

In-progress alignments are deliberately excluded from the rescue in this PR (production frame only); a follow-up PR (#567) maps the alignment coordinate through the SEP frame as well.

Why a second detector, not just full-frame cedar

Same-frame comparison from the fielded device (imx462, Seoul, 2026-08-01, background at 87% of full scale — 50 stage-dump exposures, each frame run through all three configurations offline, bit-identical production inputs):

cedar 512 (current) cedar full-frame (σ8, hot filter, binned) hybrid (this PR)
solve rate 0/50 9/50 (18%) 44/50 (88%)
detections (median) 1 5 17
matched stars (median) 8 10

Live over the same 15 minutes: 1,632 attempts, 89.5% solved (≈100% once the sky improved), every solve via the SEP path. Under a darker sky two nights earlier the balance flips the way you'd hope: cedar solves directly again (5-min window: 461 attempts, 420 cedar / 41 SEP, 100%), and a 40-minute run solved 95% with the two paths splitting roughly half and half. Accuracy (drift-removed scatter, static tube): 1σ 7–17″ on the darker nights, ~1′ under the bright-sky session above. Zero false solves across all sessions — tetra3's pattern match stays the final arbiter, and junk centroid sets simply fail to match.

The full-frame-cedar column is directly relevant to #554: solving the whole sensor is a real win (we measured 3× matches on a clear night), but under heavy LP the 8-bit domain itself is the bottleneck — full-frame cedar still solved only 18% of frames the SEP path solved 88% of.

Relationship to #554

Complementary, and happy to converge. #554 stops cropping for the primary solve; this PR adds a 12-bit-domain rescue for skies where even that fails. The two overlap in the geometry layer (SolveGeometry there, solver_frame_map here — both centre-scale mappings between solve frame and display frame). If #554 lands first I'll rebase this onto it and unify on its geometry; the SEP path would then be a fallback detector inside that architecture rather than a parallel one. Our field data may also be useful evidence for #554's review.

Field validation write-ups

Open questions for maintainers (why this is a draft)

  1. Default: off (as submitted) vs on. On the fielded device it runs always-on; for upstream I'd suggest keeping it opt-in until it has miles on other cameras.
  2. Sensor generality: geometry (crop, rotation, bit depth, saturation) all comes from CameraProfile, so imx296/HQ should work unmodified — but all field validation is imx462. Guidance welcome on what you'd want tested before default-on.
  3. Warm-pixel map ops: the map is optional and the CLI is bundled; is a docs section enough, or would you want it auto-built?
  4. Convergence with Plate-solve the full sensor, with a cached optical calibration #554 as above.

Testing

  • pytest -m unit (1,170 passed; one pre-existing cedar-shmem test fails only when a live PiFinder holds the segment on the test machine) and pytest -m smoke — green on this branch.
  • New tests: SEP detection/gates/warm-map (21), fallback backoff (5), and the production-vs-SEP solve equivalence proof (2).
  • ruff check / ruff format / mypy clean on all touched files.
  • Field: multiple full nights on an imx462 device as described above.

🤖 Generated with Claude Code

hjoungjoo and others added 5 commits August 2, 2026 00:20
sep is the C Source Extractor library behind the SEP fallback detector
(next commits). Optional at runtime: every import site degrades to the
existing cedar-only behaviour when the module is missing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… mapping

Two self-contained modules, no wiring yet:

* sep_detect: detects stars on the UNCROPPED 12-bit raw frame -- 2x2
  binning, sep mesh-background subtraction (removes light-pollution
  gradients / cloud glow that defeat a global threshold), matched-filter
  extraction, then field-measured quality gates: edge margin (vignette),
  saturation guard, point-source shape gate (semi-major axis / npix;
  cloud texture is extended, stars are compact), isolation gate (sep
  deblends bright cloud edges into tight clumps; real stars at this
  plate scale measured zero neighbours within 50 px), and an optional
  static warm-pixel mask (single-pixel sensor defects dominated
  empty-sky counts in field data).

* solver_frame_map: maps between the production rotated-512 solver
  frame and the rotated full-sensor frame. Because the crop is centred
  and the resize isotropic, target_pixel mapping reduces to a scale
  about the frame centre. Rotation conventions are pinned by tests
  against PIL Image.rotate; the stage-5 rule reads camera_interface's
  SCREEN_ROTATE_AMOUNTS as its single source.

sep stays optional: detect_stars returns None when the module is
missing, callers degrade to cedar-only behaviour.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sep_warm_map builds the static warm-pixel map sep_detect masks with:
same-channel neighbour excess recurring at a fixed position across many
dark frames (stars move with the sky, defects do not). Thresholds were
validated on a field corpus: the map covered every recurring empty-sky
detection cell while masking ~0.14% of the frame.

The equivalence test projects tetra3's own catalog stars onto a
synthetic full sensor and solves the same sky through both paths
(production crop-512 vs rotated full frame); Roll must agree to 0.05
deg and camera/aligned pointing to a fit residual -- the proof that
detecting on a different frame cannot disturb tracking or push-to.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…g-gated)

When solver_shadow_detect or solver_sep_fallback is enabled, the Pi
camera publishes each exposure's UNCROPPED raw mosaic (profile rotation
applied, crop skipped) alongside the existing cropped cam_raw, with the
frame's actual exposure/gain metadata. The solver-side SEP path detects
on this frame; the production 512 pipeline is untouched.

Both keys default to off, in which case no extra copy or publish
happens -- captures are byte-for-byte the current behaviour.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…iled solves

The production path is unchanged and keeps priority: every attempt still
runs cedar-detect on the processed 512 frame first. When that solve
fails and the SEP full-frame detector found enough stars (>= 5), the
attempt is rescued by solving the SEP centroids in the rotated full
frame; the solution carries production 512-frame semantics (Roll,
target_pixel pointing) via solver_frame_map, so the pointing chain
downstream cannot tell which detector solved.

Extras:
* Exponential backoff on failed rescues (a failed fallback burns up to
  solve_timeout of CPU; indoors/thick cloud would pay it every
  attempt), re-armed instantly when the SEP count jumps 1.5x -- a cloud
  gap opening on stars is never delayed.
* Shadow CSV (solver_shadow_detect): one row per attempt comparing both
  detectors, for offline A/B without affecting the solve.
* In-progress alignments are excluded from the rescue: alignment
  resolves through the production frame only (a follow-up PR maps the
  alignment coordinate through the SEP frame as well).

Both config keys default to off; with them off no SEP code runs and no
full frame is published.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant