Skip to content

feat(deploy): tick-level Polymarket tape recording + faster trade backfill - #548

Merged
proerror77 merged 1 commit into
mainfrom
codex/pm-tick-tape
Aug 1, 2026
Merged

feat(deploy): tick-level Polymarket tape recording + faster trade backfill#548
proerror77 merged 1 commit into
mainfrom
codex/pm-tick-tape

Conversation

@proerror77

@proerror77 proerror77 commented Jul 31, 2026

Copy link
Copy Markdown
Owner

Change contract

Polymarket market tape records tick-level full-depth books (quote_sample_ms = 0, upload service in lockstep so manifests declare temporal_updates_complete=true / upgraded replay_scope) and the reference collector raises its trade poll budget 112 → 200 (watchdog-safe: 180s cycle, 100ms spacing ⇒ 200 polls need ≥20s pacing).

Out of scope

Dependency / merge order

Merge together with the pm-gate-lockstep PR (either order, same deploy window). Textually conflicts with the pm-tape-binance-kinds PR in polymarket-market-tape.toml (adjacent lines) — whichever lands second rebases.

Focused validation

  • Key names cross-checked against producer/validator sources (recorded.rs:546, polymarket_upload.rs:1419-1440, polymarket-raw-ops.rs:60-61)
  • TOML parses; watchdog arithmetic documented in the unit file

Rollout / rollback impact

Volume risk: tick-level tape grows to est. hundreds MB/hr uncompressed within CPUQuota=30%/MemoryMax=512M and broadcast channel 8192 — canary observation required after deploy. Rollback = redeploy previous recorder bundle (config-only).

Issue relationship

Refs #539

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@proerror77, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 30 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: edae0dcb-03b3-4319-86d2-3d0e21cfdb7c

📥 Commits

Reviewing files that changed from the base of the PR and between d895506 and 5088916.

📒 Files selected for processing (4)
  • deployment/aliyun/README.md
  • deployment/aliyun/polymarket-market-tape-upload.service
  • deployment/aliyun/polymarket-market-tape.toml
  • deployment/aliyun/polymarket-reference-collector.service

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@proerror77
proerror77 enabled auto-merge (squash) August 1, 2026 02:08
Set record_market_updates_quote_sample_ms = 0 in polymarket-market-tape.toml
(0 disables sampling per recorded.rs) and --quote-sample-ms 0 in the market
tape upload unit so manifests record temporal_updates_complete=true and the
complete_full_depth_normalized_hour_segment replay scope
(polymarket_upload.rs).

Raise collect-reference --max-trade-polls-per-cycle from the default 112 to
200: the 180s cycle watchdog and 100ms minimum request spacing bound one
cycle to 1800 request starts, so 200 polls (>=20s of pacing) leave ~160s of
headroom for metadata and settlement work; concurrency stays at 4, so the
672MiB/768MiB memory calibration is unchanged.

Gate lockstep follow-ups required at rollout: polymarket-rust-health-policy.jq
pins trade_poll_budget == 112, and polymarket-raw-ops-cutover.sh /
polymarket-shadow-gate-policy.jq pin the old ExecStart/cmdline identities.

Refs #539
@proerror77
proerror77 merged commit c6e0563 into main Aug 1, 2026
41 checks passed
@proerror77
proerror77 deleted the codex/pm-tick-tape branch August 1, 2026 02:26
proerror77 pushed a commit that referenced this pull request Aug 1, 2026
…sampling

The scope-gated CI let #550/#548 merge without running
workspace_runtime_retirement; its pinned include_kinds and
quote_sample_ms went stale.

Refs #540
Refs #539
proerror77 added a commit that referenced this pull request Aug 1, 2026
…onfig pins (#562)

* fix(security): bump ruint past RUSTSEC-2026-0220 in prediction-markets lockfile

PR #522 updated only the main workspace lock; ploy-ci's Dependency
audit scans rust_hft/prediction-markets/Cargo.lock and now fails
repo-wide on lanes that trigger it.

Closes #561

* test(ploy): update tape config pins for Binance kinds and tick-level sampling

The scope-gated CI let #550/#548 merge without running
workspace_runtime_retirement; its pinned include_kinds and
quote_sample_ms went stale.

Refs #540
Refs #539

* fix(research): refresh checked-in Linux policy graph for ruint bump

The build.rs graph validator requires input:Cargo.lock to hash the
workspace lockfile exactly; the ruint bump changed it. ruint itself is
not in the Linux policy package set, so only the input line and the
canonical fingerprint need refreshing.

Refs #561

* test(research): re-pin mission templates to refreshed policy snapshot id

The policy snapshot id is derived from the dependency fingerprint;
the ruint lockfile refresh changed it, so the checked-in BTC/SOL
mission templates must be re-pinned.

Refs #561

---------

Co-authored-by: Sonic Shih <sonic.shih@mandonothing.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant