Skip to content

feat(cdp): run the rust hogvm synchronously and remove the shadow comparison - #70070

Merged
jose-sequeira merged 2 commits into
masterfrom
jose-sequeira/rust-hogvm-executesync
Jul 13, 2026
Merged

feat(cdp): run the rust hogvm synchronously and remove the shadow comparison#70070
jose-sequeira merged 2 commits into
masterfrom
jose-sequeira/rust-hogvm-executesync

Conversation

@jose-sequeira

Copy link
Copy Markdown
Contributor

Problem

Follow-up to #69605:

  • The executor ran through the async executeBatch entry point. The batch shape is only worth it when we parallelize multiple events per call; for one event at a time the libuv thread-hop isn't worth it, and executeSync matches the Node VM's synchronous exec while blocking the loop 6.7x less (168µs vs 1,127µs mean for the geoip template).
  • Shadow mode is no longer necessary: parity is validated and the Rust VM is the primary executor in every environment.

Changes

  • RustVmExecutor executes through executeSync on the calling thread; execute() is synchronous again.
  • Remove the shadow comparison end to end: RustVmShadow, its tests, the CDP_HOG_RUST_VM_SHADOW_SAMPLE_RATE config, the transformer's capture/flush hooks, and the napi executeBatch entry point plus its rayon machinery. Charts config removal: PostHog/charts#13069.
  • Re-introducing batching (one FFI crossing for many events) is the follow-up shape if per-pod transformation rate grows — the benchmark shows 2.3x throughput headroom there.
Benchmark: geoip template, release addon, Apple Silicon
Mode Per-invocation Throughput
Node execHog (previous prod path) 1,127µs mean (p99 1,865) 887/s
Rust executeSync (this PR) 168µs mean (p99 256) 5,955/s
Rust executeBatch(1) awaited 179µs 5,578/s
Rust executeBatch(1), 8 in flight 71µs effective 14,016/s
Rust executeBatch(128) per call ~122µs ~8,100/s

How did you test this code?

I'm an agent; checks I ran:

  • Rust: cargo test --features noop (19 pass, including the thread-local log carryover test for the back-to-back executeSync shape), cargo clippy -D warnings, cargo shear (rayon removal leaves no unused deps), release cargo build.
  • Jest: rust-vm-executor.test.ts (9 pass) reverted to the sync contract; transformer wiring tests updated to mock executeSync.
  • tsc -b clean.
  • The benchmark above was run locally against the real template bytecode and mmdb, then removed (not part of this diff).

Automatic notifications

  • Publish to changelog?
  • Alert Sales and Marketing teams?

Docs update

Internal executor change, no user-facing behavior difference.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

  • Authored with Claude Code (Claude Fable 5), directed by the assignee.
  • We first tried the async path, observed latency in dev, then benchmarked all entry points before deciding: sync wins sequential latency; async's throughput advantage only materializes with in-flight concurrency, which current volumes don't need. The numbers are in the PR body so the future batching follow-up doesn't have to rediscover them.

…ison

executeSync on the JS thread matches the Node VM's threading model; the
libuv thread-hop of the async path cost more than the sub-millisecond
executions it offloaded. Batching many events per FFI crossing is the
follow-up if that changes. The shadow comparison (RustVmShadow,
CDP_HOG_RUST_VM_SHADOW_SAMPLE_RATE, executeBatch + rayon) served its
purpose validating parity and is removed now that the rust vm is the
primary executor.
@trunk-io

trunk-io Bot commented Jul 10, 2026

Copy link
Copy Markdown

Static BadgeStatic BadgeStatic BadgeStatic Badge

View Full Report ↗︎Docs

…hogvm-executesync

# Conflicts:
#	nodejs/src/cdp/hog-transformations/rust-vm-executor.test.ts
#	nodejs/src/cdp/hog-transformations/rust-vm-executor.ts
@jose-sequeira
jose-sequeira marked this pull request as ready for review July 13, 2026 08:11
@pr-assigner-resolver-posthog
pr-assigner-resolver-posthog Bot requested a review from a team July 13, 2026 08:11
@greptile-apps

greptile-apps Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "Merge remote-tracking branch 'origin/mas..." | Re-trigger Greptile

Comment thread nodejs/src/cdp/hog-transformations/rust-vm-executor.ts
@veria-ai

veria-ai Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

@jose-sequeira
jose-sequeira merged commit 7711346 into master Jul 13, 2026
273 checks passed
@jose-sequeira
jose-sequeira deleted the jose-sequeira/rust-hogvm-executesync branch July 13, 2026 09:20
@deployment-status-posthog

deployment-status-posthog Bot commented Jul 13, 2026

Copy link
Copy Markdown

Deploy status

Environment Status Deployed At Workflow
dev ✅ Deployed 2026-07-13 11:21 UTC Run
prod-us ✅ Deployed 2026-07-13 11:34 UTC Run
prod-eu ✅ Deployed 2026-07-13 11:36 UTC Run

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants