Skip to content

[engine] Schema v20: persist quote_provenance and client_order_id on orders, at placement #715

Description

@eaitbrahim

Context (verified 2026-09-04, on main at the #714 merge)

#700 asked the Orders view to show two things per order that the engine does not record:

  • Quote provenanceBROKER QUOTE / SYNTHETIC ESTIMATE / UNPRICED / PREVIEW UNREADABLE. These strings exist today only as confirm-gate banners (keel/commands/confirm.py:33-36, NATIVE_PREVIEW_MARKER and its three siblings). They describe the preview a human was shown before placing, and nothing writes them down. The orders table has no provenance column.
  • client_order_id — the idempotency key. grep -n "client_order_id\|idempotency" keel/data/db.py returns nothing: the column does not exist.

Both were dropped from #700 rather than faked. Deriving either at read time would put a figure on the page that no record supports — the failure the evolution plan names as "a number invented for the UI would be a constitution violation", and the same rule _order_row_payload already follows when it refuses to state a fee rate.

The rest of #700 shipped: server-side ?status= filtering with honest empty states, and the rule that placed each order resolved to its name.

Scope

  1. Schema v20, following the standing pattern (v17/v18/v19 — DDL in _SCHEMA_STATEMENTS, a numbered, documented no-op step, SCHEMA_VERSION 19 → 20):
    • orders.quote_provenance TEXT — the preview's own verdict at placement, from a closed vocabulary that lives with the markers in confirm.py rather than being restated in the schema.
    • orders.client_order_id TEXT — the idempotency key as sent to the venue.
    • NULL means NOT RECORDED on both, never a default word: every row written before v20 was placed without either being captured, and stamping a value on them would manufacture exactly the claim the columns exist to make checkable. Same posture as orders.filled_quantity (NULL = "not observed") and the v17/v18/v19 no-backfill notes.
  2. Write them at placement, in the executor path that already writes submit_best_bid/submit_best_ask (keel stores nothing that measures spread, and at this deployment's clip size spread IS the cost #626) — the provenance is known at exactly the moment the preview is read, and capturing it later would be inference.
  3. Paper rows: paper has no venue preview at all, so quote_provenance stays NULL there for the same reason submit_best_bid does — a fabricated provenance sharing a column with a real one would poison the measurement the column exists for.

Then, and only then

Surfacing both in /api/orders is a two-line addition to _order_row_payload plus two columns in ordersView — genuinely trivial once the data is real. That follow-up closes the last of #700's original scope.

Refusals

  • No backfill, for the reason stated above.
  • No derivation of provenance from submit_best_bid/submit_best_ask: an observed book is evidence that a book was readable, not that the venue priced this order. They answer different questions and one must not stand in for the other.

Acceptance

  • Fresh DB stamps v20; a v19 DB migrates cleanly; re-running is idempotent.
  • A live order placed through the confirm gate records the provenance the gate displayed, and the client_order_id actually sent.
  • A paper order records NULL for provenance.
  • Tests: migration idempotence, NULL-means-unrecorded on pre-v20 rows, and the recorded provenance matching the banner the gate rendered for the same preview.

Split out of #700 (Sprint 1 of the honest-cockpit milestone), where both were found to be unbuildable as specified.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew capability (groups under Features)pythonPull requests that update python code

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions