Motivation
In steady state, most observations of an object are already in a previous fit and only a small fraction of objects receive new astrometry each cycle (a nightly/weekly MPC delta). Re-fitting the entire catalog from scratch every cycle is wasteful. layup should support incremental / steady-state catalog updates: fit the catalog once, then each cycle touch only what changed.
Three levers (in increasing effort / payoff)
1. Skip unchanged objects (exact; the big win).
Store an obs fingerprint in the fit result -- e.g. a hash of the observations used (count + hash of the sorted records). On a new cycle, an object whose fingerprint is unchanged has an identical fit -> carry the prior forward, don't refit.
- Needs: a fingerprint column in the orbitfit result schema + an incremental driver that loads a prior catalog and diffs by fingerprint. Python-only.
2. Warm-start the refit from the prior fit (exact; ~free).
For changed objects, seed the LM from the prior fit's state rather than from IOD/MPCORB. orbitfit already accepts a warm-start initial_guess, so this is mostly plumbing. Reduces LM/rejection iterations (and their IAS15 ramp-ups) -- marginal for well-behaved objects (they converge in ~1 iteration cold), but real for poorly-conditioned ones, and costs nothing.
3. True sequential / information-filter update (research extension; big per-object win).
For an object with thousands of old obs and a few new ones, (1)+(2) still re-integrate all old obs. A sequential update instead uses the prior covariance as a prior (information matrix Lambda = C_prev^-1) and processes only the new observations, never re-touching the old ones -- how production OD (JPL/MPC) actually runs.
- Needs: a C++ prior-information term in the LM core (add
Lambda to the normal equations; residuals/partials for new obs only), obs-level diffing (which obs are new), rejection with a prior, and validation against a full re-fit + a nonlinearity fallback gate (fall back to full refit when the update moves the state too far).
- Caveat: it's a linearized update -- valid for routine tracking, not for an orbit-changing recovery.
Prototype evidence
Levers 1+2 were prototyped in the MPC full-catalog-fit harness. A cycle rehearsal (400 numbered objects, 18% given new obs): skip 326 / refit 74, ~5x less compute (253 s -> 47.5 s of fit-loop). The compute saving scales as 1/change-fraction, so a realistic 1-5% nightly change rate is ~20-100x cheaper than a full re-fit. Carried-forward rows were byte-identical to the baseline and refits still matched MPCORB to ~1e-8. Warm-start (lever 2) bought only ~1.1x on these well-behaved objects.
Effort estimate
- Levers 1+2 as a layup feature: ~1-2 days (fingerprint in the result schema + warm-start plumbing + incremental driver + tests/docs; reuses existing pieces).
- Lever 3 (sequential update): ~1-2 weeks (C++ prior-information term + obs-level diff + validation harness + fallback gate).
Related: the MPC full-catalog fit demo (the operational model beyond the one-time cold fit).
Motivation
In steady state, most observations of an object are already in a previous fit and only a small fraction of objects receive new astrometry each cycle (a nightly/weekly MPC delta). Re-fitting the entire catalog from scratch every cycle is wasteful. layup should support incremental / steady-state catalog updates: fit the catalog once, then each cycle touch only what changed.
Three levers (in increasing effort / payoff)
1. Skip unchanged objects (exact; the big win).
Store an obs fingerprint in the fit result -- e.g. a hash of the observations used (count + hash of the sorted records). On a new cycle, an object whose fingerprint is unchanged has an identical fit -> carry the prior forward, don't refit.
2. Warm-start the refit from the prior fit (exact; ~free).
For changed objects, seed the LM from the prior fit's state rather than from IOD/MPCORB.
orbitfitalready accepts a warm-startinitial_guess, so this is mostly plumbing. Reduces LM/rejection iterations (and their IAS15 ramp-ups) -- marginal for well-behaved objects (they converge in ~1 iteration cold), but real for poorly-conditioned ones, and costs nothing.3. True sequential / information-filter update (research extension; big per-object win).
For an object with thousands of old obs and a few new ones, (1)+(2) still re-integrate all old obs. A sequential update instead uses the prior covariance as a prior (information matrix
Lambda = C_prev^-1) and processes only the new observations, never re-touching the old ones -- how production OD (JPL/MPC) actually runs.Lambdato the normal equations; residuals/partials for new obs only), obs-level diffing (which obs are new), rejection with a prior, and validation against a full re-fit + a nonlinearity fallback gate (fall back to full refit when the update moves the state too far).Prototype evidence
Levers 1+2 were prototyped in the MPC full-catalog-fit harness. A cycle rehearsal (400 numbered objects, 18% given new obs): skip 326 / refit 74, ~5x less compute (253 s -> 47.5 s of fit-loop). The compute saving scales as
1/change-fraction, so a realistic 1-5% nightly change rate is ~20-100x cheaper than a full re-fit. Carried-forward rows were byte-identical to the baseline and refits still matched MPCORB to ~1e-8. Warm-start (lever 2) bought only ~1.1x on these well-behaved objects.Effort estimate
Related: the MPC full-catalog fit demo (the operational model beyond the one-time cold fit).