Repository navigation
Lessons from #5063 for reducing reconciliation work without weakening recovery #5632
Unanswered
daveremodeling
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
We maintain a separate Python equity PAPER execution engine, not a Nautilus deployment. Issue #5063 and the targeted repeated-work fixes in commit 211ebe3 are particularly relevant to our investigation.
Our single owner serializes fills, protection, exits and reconciliation. In one retained case, an urgent exit waited roughly 14 seconds behind trade updates and a 6.5-second reconciliation unit. That unit included about 5 seconds of nested checkpoint work; those times overlap and should not be added. Other waits have been longer. Broker-side hard stops/targets, partial-fill accounting, operator controls and restart evidence must remain intact.
Each checkpoint rebuilds a growing logical state image, computes row changes, and commits/verifies a primary and mirror SQLite store before acknowledgement. The SQL writes are already incremental. Full-image serialization caches failed our performance screens. Moving exits ahead of fills or yielding arbitrary reconciliation stages could expose stale cross-position state, so we have not deployed those changes.
For engineers who worked on #5063: what profiling evidence distinguished repeated historical replay/copying from essential reconciliation work? Were the gains obtained with snapshot-local indexes/borrowing rather than cross-event caches, and which ambiguity/recovery regression tests were most valuable? We are looking for transferable measurement and equivalence techniques, not an assumption that your asynchronous persistence guarantees match ours. A public benchmark or call-graph example would help.
Reference: #5063
All reactions