feat(collector): accept Binance reference kinds on Polymarket market tape

[proerror77]

Change contract

Polymarket market tape carries Binance reference updates (spot_price/agg_trade/l2) end-to-end: recorder accepts them via include_kinds and the upload validator validates them field-level against the real producer shapes (zero-price spot sentinel accepted — emitted on every Binance WS reconnect; rejecting it would make any reconnect tape un-uploadable). PM coverage semantics (canonical, quote_coverage_complete) stay PM-scoped; a new binance_reference_counts manifest field carries per-family counts.

Out of scope

• l2_depth kind (filtered recorder-side already)
• Recovery-path validation (deliberately narrower; Binance kinds can never appear in reference tapes)
• Research-lane consumers (feat(research): accept tick-level + mixed-kind Polymarket tapes in research import and evidence validation #542)

Dependency / merge order

Stacked on the pm-upload-qcf PR (base codex/pm-upload-qcf). Textually conflicts with the pm-tick-tape PR in the toml — rebase after that lands.

Focused validation

• 4 new tests: mixed tape validates with PM coverage intact, 9 malformed sub-cases fail closed, zero-price sentinel accepted, PM coverage gaps not masked
• Field shapes verified against ploy-market-contracts/src/events.rs and binance_collectors.rs by adversarial review
• cargo test -p hft-collector --locked: 62 polymarket_upload tests green

Rollout / rollback impact

Validator + config; takes effect on the next recorder/uploader deploy. Rollback = revert config, older validator still accepts PM-only tapes.

Issue relationship

Closes #540

Summary by CodeRabbit

• New Features

  • Added support for recording Binance reference updates, including spot prices, aggregated trades, and order book updates.
  • Tape manifests now report counts for each Binance reference update type.

• Bug Fixes

  • Improved validation for malformed or invalid reference records.
  • Preserved existing Polymarket quote coverage and quality checks while supporting mixed tapes.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f10063e2-d7a7-4817-88cc-e7ad542a8ee8

📥 Commits

Reviewing files that changed from the base of the PR and between 91d7435 and a0f2b36.

📒 Files selected for processing (2)

• deployment/aliyun/polymarket-market-tape.toml
• rust_hft/tools/collector/src/polymarket_upload.rs

---

📝 Walkthrough

Walkthrough

The recorder now includes Binance spot_price, agg_trade, and l2 records. The upload validator checks their fields, excludes them from Polymarket coverage metrics, and adds independent per-kind counts to manifests.

Changes

Binance reference tape support

Layer / File(s)	Summary
Recording and accepted-kind contract deployment/aliyun/polymarket-market-tape.toml, rust_hft/tools/collector/src/polymarket_upload.rs	The recorder and tape validator accept spot_price, agg_trade, and l2 records.
Binance record validation rust_hft/tools/collector/src/polymarket_upload.rs	Dedicated validators enforce required fields and value constraints. Recovery validation remains restricted to Polymarket reference kinds.
Manifest counts and coverage tests rust_hft/tools/collector/src/polymarket_upload.rs	Manifests include independent Binance counts. Tests cover mixed tapes, zero-price spot records, malformed records, and Polymarket coverage failures.

Estimated code review effort: 4 (Complex) | ~45 minutes

Possibly related issues

• proerror77/monday issue 533 — Both changes modify tape-kind validation in polymarket_upload.rs.
• proerror77/monday issue 542 — Both changes support mixed Polymarket tapes with Binance reference kinds.

Possibly related PRs

• proerror77/monday#9 — Introduces the recording-policy and feed changes for the Binance RecordingKind values used here.
• proerror77/monday#34 — Introduces the tape validation and manifest logic extended by this change.
• proerror77/monday#104 — Adds related Binance market-tape validation for these record kinds.

Sequence Diagram(s)

sequenceDiagram
  participant Recorder
  participant TapeScanner
  participant BinanceValidators
  participant ManifestBuilder
  Recorder->>TapeScanner: write configured Binance reference records
  TapeScanner->>BinanceValidators: validate each record kind
  BinanceValidators-->>TapeScanner: accept or reject record
  TapeScanner->>ManifestBuilder: count accepted Binance records by kind
  ManifestBuilder-->>TapeScanner: emit manifest with reference counts

Loading

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies acceptance of Binance reference kinds on Polymarket market tapes.
Description check	✅ Passed	The description covers the change contract, issue, scope, dependencies, validation, and rollout details; the omitted scope exception is non-critical for this change size.
Linked Issues check	✅ Passed	The changes implement issue #540 by enabling, validating, and configuring the three Binance reference kinds while preserving Polymarket-only manifest semantics.
Out of Scope Changes check	✅ Passed	The configuration, validator, manifest, and tests are within issue #540, and the stated exclusions match the linked issue scope.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/pm-tape-binance-kinds

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}