Skip to content

feat(collector): accept Binance reference kinds on Polymarket market tape #540

Description

@proerror77

Change contract

Polymarket market tape may carry Binance reference updates (spot_price, agg_trade, l2 kinds) end-to-end: recorder accepts them via include_kinds, and the upload validator validates them with correct manifest semantics.

Depends on: upload validator quote_collection_failure fix (same file, land first).

Background

Recorder side is config-ready (RecordingKind includes SpotPrice/AggTrade/L2, recorded.rs:51-66; Binance feed already runs in-process). Blocker is the validator: ALLOWED_KINDS (polymarket_upload.rs:24-34), scan bail (:976-977), per-kind validation arms (:1094-1166), recovery re-check (:838-847), and PM-shaped quote_coverage_complete/canonical manifest assumptions.

Scope

  • Extend ALLOWED_KINDS + per-kind validation for spot_price/agg_trade/l2 (symbol/shape/timestamp checks)
  • Manifest semantics for mixed-kind tapes: coverage fields computed per family; canonical meaning documented and enforced
  • deployment/aliyun/polymarket-market-tape.toml: add kinds to include_kinds
  • Volume sizing note (~70 L2/s + trade bursts × 7 symbols)

Acceptance

  • Unit tests: mixed-kind tape validates; malformed Binance kinds rejected; PM-only invariants unaffected
  • Runtime evidence after deploy: OSS manifests reflect the new kinds with correct per-family counts

Out of scope

  • Replacing the standalone binance-lob-archiver datasets (cross-dataset alignment remains a research-side concern)

Metadata

Metadata

Assignees

Labels

enhancementNew feature or requestready-for-agentFully specified and safe for an autonomous agent

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions