Skip to content

Incremental steady-state fit driver (capstone of #419) - #422

Merged
matthewholman merged 4 commits into
mainfrom
feat/incremental-driver
Jul 6, 2026
Merged

Incremental steady-state fit driver (capstone of #419)#422
matthewholman merged 4 commits into
mainfrom
feat/incremental-driver

Conversation

@matthewholman

@matthewholman matthewholman commented Jul 5, 2026

Copy link
Copy Markdown
Collaborator

Summary

The capstone of incremental orbit determination (issue #419): incremental_orbitfit, a single steady-state maintenance pass over a batch of objects that ties the three levers together. For each object it routes:

  • skip — the observation set is unchanged since prior_catalog (matching fingerprint): carry the prior fit forward verbatim, no fit.
  • sequential update — observations were only appended (prior_obs given and every prior observation still present): update via sequential_update over the new observations only. Its nonlinearity gate falls back to a full refit when the update is too large.
  • full refit — observations removed or changed, or no per-object prior_obs: refit over all current obs, warm-started from the prior state.
  • cold — no prior for the object: cold IOD fit.

Returns the updated result catalog (same schema as orbitfit output, carrying the obs_hash/nobs_fit fingerprint so it seeds the next cycle) plus a routing tally.

Obs-level diffing (_obs_row_keys / _append_only_new_obs) reuses #420's per-row fingerprint strings, so a row's diff key equals its contribution to the object's fingerprint — append-only vs modified is detected exactly.

⚠️ Stacking — review order

This is one feature in three parts and stacks on #420 and #421:

This branch merges both, so the diff below includes them. Merge #420 and #421 first; this PR's net-new code is a single commit, 091f804 (incremental_orbitfit + tests/layup/test_incremental_driver.py) — review that commit directly, and I'll rebase once the two land so the diff reduces to just the driver.

Validation

  • 5 route tests in test_incremental_driver.py (skip byte-identity, sequential-matches-full-refit, modified-obs→full, cold, no-prior-obs→full). Full suite of Incremental fits: obs fingerprint + skip-unchanged (Phase A of #419) #420 + Sequential / information-filter orbit update (Lever 3 of #419) #421 + this + base fit tests: 31 passed.
  • Real MPC data (mpc_demo/cycle_rehearsal_seq.py, a steady-state cycle on numbered asteroids): the sequential route matches a full refit to state rel ~1e-8 / KEP a ~1e-9, at 6.5–13.7× less wall-clock on ~1000-obs arcs (only the new obs are integrated; the win approaches the integration-work ratio as the arc lengthens). The nonlinearity gate never tripped on well-observed objects with small nightly additions — its job is the pathological cases.

Combined with the skip route, the steady-state loop is: 95–99% of objects skip (zero cost), changed objects sequential-update at ~7–13× less work, and only non-append or nonlinear changes fall back to a full refit — the operational maintenance loop JPL/MPC run.

Follow-ups (unchanged from #421)

Outlier rejection on the new observations in the sequential path; non-grav priors; radar/occultation in the sequential path; differing-epoch priors. A CLI wrapper for incremental_orbitfit can follow once the API settles.

Part of #419.

Closes #419

matthewholman and others added 2 commits July 5, 2026 07:44
Ties the three levers into one operational maintenance pass over a batch of
objects. Per object, incremental_orbitfit routes:

  * skip        -- observation set unchanged since the prior catalog (matching
                   fingerprint): carry the prior fit forward verbatim, no fit.
  * sequential  -- observations only appended (prior_obs given and every prior
                   obs still present): sequential_update() over the new obs only
                   (integrating just the new observations). Its nonlinearity gate
                   falls back to a full refit when the update is too large.
  * full refit  -- observations removed or changed, or no per-object prior_obs:
                   refit over all current obs, warm-started from the prior state.
  * cold        -- no prior for the object: cold IOD fit.

Obs-level diffing (_obs_row_keys / _append_only_new_obs) reuses #420's per-row
fingerprint strings, so a row's diff key equals its contribution to the object's
fingerprint. Returns the updated result catalog (same schema as orbitfit output,
carrying obs_hash/nobs_fit so it seeds the next cycle) plus a routing tally.

Warm/cold full refits go through orbitfit in two calls (warm objects filtered to
their priors, cold with no guess) so _orbitfit's per-object initial-guess lookup
is satisfied.

Stacks on #420 (fingerprint + skip) and #421 (sequential update). Validated on
real MPC data (mpc_demo/cycle_rehearsal_seq.py): sequential route matches the full
refit to state rel ~1e-8 / KEP a ~1e-9, 6.5-13.7x faster on ~1000-obs arcs (only
new obs integrated). 5 tests in test_incremental_driver.py cover all routes.

Part of #419.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@matthewholman
matthewholman requested a review from kjnapier July 5, 2026 14:17

@kjnapier kjnapier left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@matthewholman
matthewholman merged commit 99fe622 into main Jul 6, 2026
7 checks passed
@matthewholman
matthewholman deleted the feat/incremental-driver branch July 6, 2026 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Incremental / steady-state catalog updates (skip-unchanged, warm-start, sequential update)

2 participants