chore(batch-imports): e2e test harness with misbehaving mock export APIs

[nickbest-ph]

Problem

The batch import worker had no test coverage for how the Mixpanel/Amplitude export APIs actually behave across multiple downloads of the same date range. Their output is not byte-stable between calls (event ordering shifts, late data arrives), and our resume mechanism stores decompressed-byte offsets that are only meaningful against the exact byte stream they were committed against. When a pod replacement mid-part forces a re-download, the saved offset can land mid-line in the new stream and the job pauses with a confusing JSON parse error at byte 0 of the chunk. We hit exactly this in production.

All existing tests mock exports as byte-stable, so this entire failure class was invisible to CI.

Changes

• tests/common/mock_export.rs: a stateful axum mock of the Mixpanel (gzip JSONL) and Amplitude (zip of .json.gz members) export APIs. Unlike httpmock it can vary its response per download attempt of the same day: Reorder (the incident behavior), LateData, RateLimit, NotFound, EmptyBody, TruncatedGzip. Bodies are derived deterministically from (seed, day, attempt) so tests compute exact ground truth, and generated lines have variable lengths so a reordered stream always misaligns byte offsets.
• tests/common/harness.rs: drives the real pipeline in-process. Real DateRangeExportSource with the production gzip/zip extractors, the production JSONL parser and per-provider transforms (hand-built TransformContext with the existing mock caches), through the DB-free select_and_fetch_next_chunk loop. A restart() helper drops and rebuilds the source while keeping the committed JobState, which is exactly the shape of a deploy landing mid-part.
• tests/mixpanel_amplitude_e2e_test.rs: seven scenarios, all asserting exactly-once output against generated ground truth. No external services, ~2s total.
• src/job/mod.rs: select_and_fetch_next_chunk goes pub(crate) -> pub. Its doc comment already describes it as the DB-free testable core; integration tests compile as external crates so they need the export.

How did you test this code?

Automated only, no manual testing:

• New e2e suite: 7 tests pass in ~2s (cargo test -p batch-import-worker --test mixpanel_amplitude_e2e_test).
• Full crate suite: 371 unit tests plus all existing integration targets pass; cargo fmt and cargo clippy --tests clean.
• Bug-injection check: temporarily switching the incident test's mock from Reorder to Stable makes the test fail, proving it detects the reorder misalignment specifically and not something incidental.

Regressions each test group catches that nothing caught before:

• Reordered re-download after restart: the production incident. No existing test re-downloads with different bytes; all existing mocks are byte-stable.
• Stable re-download after restart: resume-from-offset exists at the source level (date_range_export_streaming_test.rs) but not through the job loop with exactly-once output assertions.
• Happy paths / chunk-size sweep: pins newline_delim's consumed-offset arithmetic end to end across chunk boundary shapes; unit tests cover single buffers only.
• Rate limit / empty days: pins that a 429 surfaces as a rate-limited error for job-level backoff and that 404/zero-byte days complete instead of stalling, both through the full loop.

Automatic notifications

• Publish to changelog?
• Alert Sales and Marketing teams?

Docs update

Not needed, test-only change plus one visibility widening.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

• Authored with Claude Code (Fable 5). The work started from a production support investigation that root-caused the offset-resume failure via Loki logs, then turned into this test plan and implementation. Skills invoked: /writing-tests.
• Notable decision: the harness collects parsed events directly from the fetch loop rather than going through an Emitter, since the emit path is unit-covered and adds no discriminating power here.
• Notable finding while writing assertions: Amplitude $identify generation races benignly under rayon (check-then-mark against the identify cache), so the test asserts every unique user/device pair gets identified rather than an exact event count, matching the real guarantee.

[trunk-io]

     

View Full Report ↗︎ ⋅ Docs

[greptile-apps]

_{Reviews (1): Last reviewed commit: "Merge remote-tracking branch 'origin/mas..." | Re-trigger Greptile}

[eli-r-ph]

😍

[deployment-status-posthog]

Deploy status

Environment	Status	Deployed At	Workflow
dev	✅ Deployed	2026-07-11 00:20 UTC	Run
prod-us	✅ Deployed	2026-07-11 00:32 UTC	Run
prod-eu	✅ Deployed	2026-07-11 00:33 UTC	Run