Skip to content

Parse malformed UTF-8 in MatchResult::from_str without panicking (#114) - #122

Merged
joaquinbejar merged 3 commits into
mainfrom
stack/2-114-utf8-safe-parser
Jul 14, 2026
Merged

Parse malformed UTF-8 in MatchResult::from_str without panicking (#114)#122
joaquinbejar merged 3 commits into
mainfrom
stack/2-114-utf8-safe-parser

Conversation

@joaquinbejar

@joaquinbejar joaquinbejar commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Summary

MatchResult::from_str advanced byte offsets one at a time and sliced with
s[pos..] / s[i..]; a multibyte Unicode scalar before the next delimiter
landed an offset inside the scalar and panicked on a non-char-boundary slice.
FromStr accepts untrusted text and promises Result — it must never unwind.
The scanner now works on s.as_bytes() and slices only at ASCII-delimiter
positions, which are always char boundaries.

Stack

Changes

  • Byte-safe scanning throughout the FromStr impl: find_next_field, the
    field-name scan, and both bracket scanners (trades, filled-order ids).
    Structural delimiters (=, ;, [, ]) and literal prefixes are ASCII,
    and every byte of a multibyte scalar is ≥ 0x80, so a byte comparison never
    mistakes an interior byte for a delimiter.
  • Every direct index is bounds-guarded (audited by architect review);
    malformed input returns the existing InvalidFormat / InvalidFieldValue /
    MissingField errors deterministically. No unsafe, no new deps, no lossy
    normalization, structural validation unchanged.
  • Canonical ASCII Display output round-trips unchanged.

Testing

  • Regression: 2/3/4-byte scalars in field names, values, trade bracket
    data, filled-order-id lists; unterminated multibyte brackets — all Err,
    no panic (old scanner body panics on these)
  • Two co-located proptest properties (1024 cases each): arbitrary UTF-8
    and delimiter-dense structural fuzz never panic
  • DisplayFromStr round-trip with populated trades + filled ids
  • Pre-push checks clean locally

454 tests passed; 0 failed (435 lib + 9 + 1 + 9 doctests).

Checklist

  • Code follows rules/global_rules.md and CLAUDE.md
  • No .unwrap() / .expect() / unchecked indexing in production code
  • Existing hand-written PriceLevelError variants reused — no new variants
  • No new production unsafe, no new dependencies (proptest already a dev-dep)
  • Module boundaries respected (execution untouched upward)
  • Public surface: no signature changes

Note: the filled_order_ids branch's tolerance of trailing content after ]
is pre-existing and out of scope here — it gets tightened in #116 (next in the
stack), which routes FromStr through the invariant validator.

Closes #114

with_reduced_quantity silently no-opped (`_ => self.clone()`) for
TrailingStop, PeggedOrder, and MarketToLimit, so a partial fill recorded
the executed quantity and decremented the level's visible counter while
the resting maker kept its ORIGINAL size. A later taker could execute
the same depth again, breaking quantity conservation and underflowing
the advisory counter; UpdateQuantity inherited the same silent no-op.

The helper is now exhaustive over all seven variants: the three
single-quantity types rewrite their quantity (preserving trail and peg
parameters -- a fill must never reprice), and ReserveOrder resizes its
visible tranche keeping hidden depth and replenish policy, mirroring
IcebergOrder (UpdateQuantity path only; its match path was already
correct). The match_against and refresh_iceberg fallback arms are also
explicit now, so a future variant is a compile error instead of a
silent double-execution.

Documents the visible-tranche semantics of UpdateQuantity on the public
OrderUpdate enum. Regression tests: two consecutive takers against each
fixed variant conserve the original quantity across the advisory
counter, the live queue, and a snapshot JSON round-trip; UpdateQuantity
decrease keeps FIFO position and increase demotes for each variant.

Closes #118
…t panicking

The parser advanced byte offsets one at a time and sliced with
s[pos..] / s[i..]; a multibyte Unicode scalar before the next delimiter
put an offset inside the scalar and the slice panicked on a
non-char-boundary. FromStr accepts untrusted text and promises Result,
so malformed input must never unwind.

The scanner now operates on s.as_bytes(): all structural delimiters
(=, ;, [, ]) and literal prefixes are ASCII, and every byte of a
multibyte scalar is >= 0x80, so a byte comparison never mistakes an
interior byte for a delimiter. &str slices are taken only at
ASCII-delimiter positions or the string end -- always char boundaries.
Every index is bounds-guarded. Malformed input returns the existing
InvalidFormat / InvalidFieldValue / MissingField errors
deterministically; canonical ASCII Display output round-trips
unchanged. No unsafe, no new deps, no lossy normalization.

Tests: multibyte scalars (2/3/4-byte) in field names, values, trade
bracket data, and filled-order-id lists; unterminated multibyte
brackets; Display round-trip; two proptest properties (1024 cases each)
asserting from_str never panics on arbitrary and delimiter-dense UTF-8.
The old scanner body fails the regression with the char-boundary panic.

Closes #114
@joaquinbejar joaquinbejar added bug Something isn't working price-level Engine: level/order_queue/snapshot/statistics + execution results labels Jul 14, 2026
@joaquinbejar joaquinbejar self-assigned this Jul 14, 2026
@joaquinbejar
joaquinbejar changed the base branch from stack/1-118-reduce-all-variants to main July 14, 2026 08:34

@joaquinbejar joaquinbejar left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The UTF 8 scanner now handles multibyte input without panicking and the focused parser suite passes. One binding safety rule is still violated by the new direct indexing, so the intended effective verdict is REQUEST_CHANGES. GitHub does not accept that event from the PR author, so this review is submitted as COMMENT.

Comment thread src/execution/match_result.rs Outdated
Every index and range slice in the scanner now goes through .get() with
None mapped to InvalidFormat (or a loop restructure over bytes.get(i)),
per the global rule banning unchecked [] indexing in production code --
bounds-guarded or not, a later offset change can no longer reintroduce
a panic path.
@joaquinbejar

Copy link
Copy Markdown
Owner Author

Thanks for the review. The indexing rule violation is fixed in 8b48449, the whole scanner is .get() based now.

@joaquinbejar
joaquinbejar merged commit 86b605d into main Jul 14, 2026
13 checks passed
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 58.64662% with 55 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/orders/order_type.rs 48.23% 44 Missing ⚠️
src/execution/match_result.rs 77.08% 11 Missing ⚠️

❌ Your patch status has failed because the patch coverage (58.64%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

Files with missing lines Coverage Δ
src/orders/update.rs 93.84% <ø> (ø)
src/price_level/level.rs 93.12% <ø> (ø)
src/execution/match_result.rs 89.61% <77.08%> (+9.79%) ⬆️
src/orders/order_type.rs 47.72% <48.23%> (-0.08%) ⬇️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@joaquinbejar joaquinbejar left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The UTF 8 scanner now uses guarded accesses throughout, the prior warning is resolved, focused parser validation and CI pass, and I found no new issues. The intended effective verdict is APPROVE. GitHub does not accept an approval from the PR author, so this review is submitted as COMMENT.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working price-level Engine: level/order_queue/snapshot/statistics + execution results

Projects

None yet

Development

Successfully merging this pull request may close these issues.

execution: Parse malformed UTF-8 without panicking

2 participants