Validate MatchResult invariants during decoding (#116) - #123
Conversation
…t panicking The parser advanced byte offsets one at a time and sliced with s[pos..] / s[i..]; a multibyte Unicode scalar before the next delimiter put an offset inside the scalar and the slice panicked on a non-char-boundary. FromStr accepts untrusted text and promises Result, so malformed input must never unwind. The scanner now operates on s.as_bytes(): all structural delimiters (=, ;, [, ]) and literal prefixes are ASCII, and every byte of a multibyte scalar is >= 0x80, so a byte comparison never mistakes an interior byte for a delimiter. &str slices are taken only at ASCII-delimiter positions or the string end -- always char boundaries. Every index is bounds-guarded. Malformed input returns the existing InvalidFormat / InvalidFieldValue / MissingField errors deterministically; canonical ASCII Display output round-trips unchanged. No unsafe, no new deps, no lossy normalization. Tests: multibyte scalars (2/3/4-byte) in field names, values, trade bracket data, and filled-order-id lists; unterminated multibyte brackets; Display round-trip; two proptest properties (1024 cases each) asserting from_str never panics on arbitrary and delimiter-dense UTF-8. The old scanner body fails the regression with the char-boundary panic. Closes #114
Codecov Report❌ Patch coverage is
... and 2 files with indirect coverage changes 🚀 New features to boost your workflow:
|
joaquinbejar
left a comment
There was a problem hiding this comment.
The validator closes several malformed decode paths, but explicit outcomes can still contradict completion and trades can still reference a different taker. The intended effective verdict is REQUEST_CHANGES. GitHub does not accept that event from the PR author, so this review is submitted as COMMENT.
joaquinbejar
left a comment
There was a problem hiding this comment.
The existing review covers outcome and taker identity mismatches. One additional filled maker uniqueness and ordering gap remains. The intended effective verdict is REQUEST_CHANGES, submitted as COMMENT because the active account owns this PR.
Every index and range slice in the scanner now goes through .get() with None mapped to InvalidFormat (or a loop restructure over bytes.get(i)), per the global rule banning unchecked [] indexing in production code -- bounds-guarded or not, a later offset change can no longer reintroduce a panic path.
Derived Deserialize wrote private fields directly and FromStr constructed without validation, so malformed input could claim is_complete with a positive remainder, attach trades to Rejected or Killed results, disagree about executed quantity, or list filled makers unrepresented by trades. Deserialize now goes through #[serde(try_from)] on a private permissive wire mirror (identical field names, outcome default preserved -- every previously-valid payload still decodes; Serialize untouched, wire format byte-identical), and FromStr routes through the same private validator: - is_complete iff remaining_quantity == 0 - checked trade-quantity sum, and sum + remaining must not overflow u64 - Killed / Rejected carry zero trades and zero filled ids (matching mark_killed / mark_rejected) - every filled_order_id appears as a trade maker (the exact engine contract in match_order: ids are recorded only on full consumption right after their trade) NotFilled/PartiallyFilled vs trades is deliberately not constrained: legacy payloads default outcome to NotFilled with trades present and accessors correct it; enforcing it would break backward compatibility. Also tightens the FromStr filled_order_ids bracket to reject trailing content after ']', symmetric with the trades branch (asymmetry noted in the #114 review). Tests: negative serde/text cases per invariant (structurally valid JSON that only the validator rejects), positive round-trips for filled / killed / rejected, and a 256-case proptest building arbitrary valid results via the public API asserting both encodings round-trip. Closes #116
ffa5d2e to
77d1342
Compare
… order - outcome decodes as an Option: absent (legacy) derives the benign classification from the fields; an explicit outcome is validated against remainder / trades / filled ids, so Filled with a positive remainder, PartiallyFilled without trades, NotFilled with trades, or a complete empty Killed/Rejected are all rejected - every trade's taker order id must equal the result's incoming order id, enforced by the validator and mirrored in add_trade - filled_order_ids must be a duplicate-free, order-preserving subsequence of the trade maker ids (match_order records each fully-consumed maker exactly once, in trade order)
|
Thanks for both passes. All three gaps are closed in 893d532, explicit outcomes, taker identity, and filled-id uniqueness and ordering are all validated now. |
joaquinbejar
left a comment
There was a problem hiding this comment.
The prior invariant findings are fixed, but explicit null outcomes are still normalized as legacy absence and the public filled id mutator can create values the decoder rejects. The intended effective verdict was REQUEST_CHANGES before this head merged. GitHub requires COMMENT because this is a self review.
| /// payload can no longer claim `Filled` with a positive remainder or | ||
| /// `PartiallyFilled` without trades. | ||
| #[serde(default)] | ||
| outcome: Option<MatchOutcome>, |
There was a problem hiding this comment.
🟠 Warning: Option with #[serde(default)] maps both a missing legacy key and an explicit "outcome": null to None, so the latter is silently normalized even though the old decoder rejected it. Use a missing-field sentinel or custom deserializer so only omission triggers legacy derivation, and add a null regression test.
| // order and filled-id order agree. The `any` on the shared iterator | ||
| // advances it past each match, which is exactly subsequence | ||
| // semantics (and implies plain membership). | ||
| if !self.filled_order_ids.is_empty() { |
There was a problem hiding this comment.
🟠 Warning: This validator rejects arbitrary, duplicate, or reordered filled IDs, but the public infallible add_filled_order_id still creates those states. Such a value serializes successfully and then fails to deserialize, so public API values no longer reliably round-trip. Enforce this invariant in the mutator or make it non-public, and cover API-built invalid sequences.
Summary
Derived
DeserializewroteMatchResult's private fields directly andFromStrconstructed without validation, so malformed input could claimis_completewith a positive remainder, attach trades toRejected/Killedresults, disagree about executed quantity, or list filled makersunrepresented by trades. Both decoders now route through one private
invariant validator; the wire format is byte-identical and every
previously-valid payload still decodes.
Stack
stack/2-114-utf8-safe-parsermainis cumulative; review against the base branch.Changes
Deserializereplaced with#[serde(try_from = "MatchResultWire")]— aprivate permissive mirror with identical field names and the
outcomedefault preserved;
Serializeuntouched.is_complete ⇔ remaining == 0; checkedtrade-quantity sum and
sum + remainingfitu64;Killed/Rejectedcarry zero trades and zero filled ids; every
filled_order_idappears as atrade maker (the exact
match_ordercontract — ids recorded only on fullconsumption right after their trade).
FromStrroutes through the same validator, and itsfilled_order_idsbracket now rejects trailing content after
](asymmetry flagged in Parse malformed UTF-8 in MatchResult::from_str without panicking (#114) #122'sreview).
Technical decisions
NotFilled/PartiallyFilledvs trades is deliberately notconstrained: legacy payloads default
outcometoNotFilledwith tradespresent and accessors correct it — enforcing it would break backward
compatibility. Only the pre-sweep
Killed/Rejectedoutcomes areconstrained.
InvalidOperation/InvalidFormaterror variants — no newvariant, no wire-format change, no snapshot impact.
Testing
validator rejects) + negative
FromStrcases + trailing-content tighteninground-trip through both encodings with no field drift
467 tests passed; 0 failed (448 lib + 9 + 1 + 9 doctests).
Checklist
rules/global_rules.mdandCLAUDE.md.unwrap()/.expect()/ unchecked indexing in production codesaturating_*/wrapping_*unsafe, no new dependenciesCloses #116