Skip to content

v0.21.0 — the trade-bust release

Choose a tag to compare

@intrepidkarthi intrepidkarthi released this 10 Aug 10:10
· 89 commits to main since this release

The trade-bust release, and a reminder of why this project writes specs first: the
spec found a defect that had been shipping for four releases before a line of the
feature existed. An operator halt issued after the last checkpoint was never written
to the log, so a venue somebody had deliberately stopped came back open. Trade bust
needed a durable seam for control commands, went looking for one, and there wasn't
one.

The feature itself is mostly a list of things it refuses to do. A bust annuls a
print; it does not put the orders back, does not un-fire the stops the print
triggered, does not rewind the reference price, and does not amend the event that
reported the trade. Each of those is a test, because each looks like a bug until you
notice the book at bust time is not the book at trade time.

Added

  • Trade bust (Engine.Bust, TRADE-BUST.md) — annulling a
    print that has already been published. It is an appended EventBusted referring
    backwards by trade id, never a rewrite, because the tape a follower replays has to
    stay identical to the tape the primary produced. The surprising part is what it
    deliberately does not do, and each of the four has a test: the busted orders are
    not re-rested, the stops the print fired stay fired, LastTradePrice is not
    rewound, and the trade event is not amended. A bust arrives after the market has
    moved, and each of those undos would be a second wrong rather than a correction of
    the first.

    The registry lives in the snapshot and therefore in the digest — two engines that
    applied the same commands are equal only if they also agree on what settled. Drill
    D7 is why that matters: a follower that drops the bust has a byte-identical book
    and a different digest, which is the only reason the divergence is detectable at
    all. marketdata.Feed is the consumer, publishing UpdateBust alongside the trade
    id that UpdateTrade never carried.

    Validation is identity-only: the engine refuses ids it never issued and says
    nothing about price, size or counterparty, because it does not retain the trades it
    printed. Duplicate busts are refused rather than swallowed.

Fixed

  • Control commands were applied but never written down. Runner.logCommand
    ended with default: return // control commands carry no book state; the snapshot covers them. The snapshot covers them as of the snapshot — so a halt, resume,
    cancel-only or mark-price change issued after the last checkpoint was in no log,
    recovery did not replay it, and a venue an operator had deliberately halted came
    back Open, ready to trade, with nobody told. Shipped in every release since
    control commands existed.

    It is the same reasoning error as the durability comment corrected in v0.20.0: a
    guarantee stated against the wrong reference point. It surfaced because trade bust
    needed a durable seam for control commands and the seam turned out not to exist —
    CommandLog now carries AppendHalt/Resume/CancelOnly/SetMark/Bust, and
    TestControlCommandsSurviveRecovery fails against the old code. Breaking for
    anyone implementing matching.CommandLog outside this repository.

  • The threat model claimed a trade-bust path that did not exist, and named the
    wrong mechanism for it.
    THREAT-MODEL.md credited the WAL
    spine with "clean trade-bust / replay" while
    PRODUCTION-READINESS.md said there was no way to
    amend a published trade. Writing the spec settled it: nobody had built one, and
    replaying a log without the busted trade — the mechanism the row described —
    rewrites history and hands every downstream consumer a tape that never happened.

  • The published test count was 100 short. PRODUCTION-READINESS.md said 480 test
    functions for several releases after the suite passed it; it is 584, and the line
    now carries the command that produces the number so the next reader can check it
    instead of believing it. The event-conformance suite is 23 scenarios, not 22.

Changed

  • Wire protocol v2 → v3: a trade now has a name. Executed and MDTrade
    reported price, quantity and aggressor but no identifier, so no message could ever
    refer back to one specific print — which meant a venue with trade bust could annul
    a fill it had never named, and no client could be told which one. Both payloads
    gain TradeID (+8 bytes each), and two messages use it: Busted (U) on
    order entry, private to the two counterparties, and MDBust (u) on market
    data, public. Every other payload is byte-identical to v2 apart from the version
    field itself — the discipline a bump is supposed to carry, and what the regenerated
    golden vectors show.

    Breaking: internal/wire is not importable, but any client built against v2
    must be rebuilt. pkg/orderentry.Msg gains TradeID and KindBusted.

    Routing the bust turned out to be harder than encoding it, and for a reason that
    is the whole shape of this feature: by the time a bust arrives, both orders have
    usually left the book.
    orderentry.Registry forgets an order the moment it fills
    or cancels, so the obvious implementation — look the trade up among live orders —
    delivers a bust to nobody in the common case. The Registry now keeps a bounded
    memory of recent prints (SetFillMemory, default 65,536, about 26 seconds of tape
    at the SOAK.md rate) purely so a bust can be routed, and one older than that memory
    increments UnroutableBusts rather than vanishing — "we could not tell the client"
    is an operational fact somebody has to act on. Size it to your bust window; CME's
    is eight minutes.


Full changelog: v0.20.0...v0.21.0