test(etl): Wave 4E — real-data e2e + per-route latency regression

CHANGELOG.md

@@ -13,6 +13,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Wave 4B — backfill actually populates `usage_events`.** `stackunderflow etl backfill` now reads every message from the `messages` table, runs the matching provider normalizer (Wave 2A), and inserts into `usage_events`; `--force` rebuilds from scratch. Idempotent via `uniq_events_msg` UNIQUE index. The ingest writer (`stackunderflow/ingest/writer.py`) gets a normalize+insert hook so newly-ingested messages auto-create events without needing a backfill pass. Marts auto-refresh via `refresh_all_marts()` after each batch.
 - **Wave 4C — `/api/etl/status` + `stackunderflow etl status`.** Single endpoint surfaces watcher health, mart watermarks vs max event id, per-provider event counts, and a `health` enum (live/syncing/stale/error) so the dashboard can show a status badge and the CLI a one-line health check. <50ms response — all counts are indexed COUNT(*).
 - **Wave 4F — ETL status badge in the dashboard header + Settings backfill button.** New `EtlStatusBadge` polls `/api/etl/status` every 10s and shows live/syncing/stale/error health with a click-through popover detailing per-mart watermarks, per-provider event counts, and watcher state. Settings page gains an "ETL pipeline" section with a "Backfill now" button — POSTs to `/api/etl/backfill` when available, else shows the equivalent CLI command.
+- **Wave 4E — real-data ETL pipeline + per-route latency regression suite.** New `tests/stackunderflow/integration/` package with two slow-marker test files: `test_etl_pipeline_e2e.py` builds a 10K-message synthetic store across 5 providers, runs backfill, validates every mart sums correctly, then hits every dashboard route asserting 200 + non-empty + <500ms. `test_route_perf_regression.py` parametrises every dashboard route against a 100K-row synthetic marts fixture with explicit per-route latency budgets — fails CI if any route regresses. Run with `pytest -m slow`. New `[tool.pytest.ini_options]` section in `pyproject.toml` registers the `slow` marker and adds `addopts = "-m 'not slow'"` so the default `pytest tests/ -q` run keeps the fast feedback loop (slow tests are opt-in).
 - **Wave 3A — hot-path routes migrate to mart reads.** `/api/projects?include_stats=true`, `/api/dashboard-data`, and `/api/cost-data` (totals/by_day/by_model blocks) now read from `project_mart` + `daily_mart` instead of running per-request aggregator passes against raw `messages`. Same JSON contract; ~50× faster on the user's 28K-message project (cold 2.5–2.8s → 50ms warm). Per-session / per-command / per-tool detail blocks stay on the aggregator path until lower-grain marts ship in Wave 4.
 - **ETL foundation: usage_events fact table + 5 marts + watermarks + backfill orchestrator (Wave 1).** Lays the schema and base classes; Waves 2 (normalizers + mart builders + watcher) and 3 (route migrations) fill in the bodies. Migration v006 (the spec called it v004, but v004/v005 were taken by the synthetic-models cleanup and cursor-workspace redistribute — the migration file is renumbered to v006 and the spec doc is updated to match) adds 7 tables (`usage_events`, `daily_mart`, `session_mart`, `project_mart`, `provider_day_mart`, `model_day_mart`, `mart_watermark`) plus indexes (`idx_events_day`, `idx_events_project`, `idx_events_provider`, `idx_events_session`, `idx_events_model`, `uniq_events_msg` UNIQUE on `source_message_fk`, `idx_daily_mart_project`, `idx_session_mart_project`, `idx_session_mart_first`, `idx_provider_day_mart_day`). New `stackunderflow.etl` package: `normalize/base.py` (`Normalizer` ABC) + `normalize/__init__.py` (last-wins `register/get/all` registry), `marts/base.py` (`MartBuilder` ABC with abstract `refresh(conn, since_event_id) -> int` and concrete no-op `rebuild_from_scratch`) + `marts/__init__.py` (last-wins registry), `watermark.py` (`get_watermark` returns 0 on missing, `set_watermark` upserts with UTC ISO8601 `last_refresh_ts`, `refresh_all_marts` iterates the marts registry and persists each mart's new watermark), and `backfill.py` (`BackfillReport` dataclass with `events_inserted`, `events_skipped_duplicate`, `marts_refreshed: dict[str, int]`, `duration_seconds`; `backfill(conn, *, force=False)` orchestrator skeleton — empty-registry no-op until Wave 2 lands, `force=True` empties events + marts + watermarks). New CLI: `stackunderflow etl backfill [--force]` (no-op until normalizers register in Wave 2; reports zero counts). Migration is **additive** — does not touch existing `messages`/`sessions`/`projects` tables, all existing routes keep working unchanged. 39 new tests across `tests/stackunderflow/store/test_migration_v006.py` (12: tables exist, columns/PKs per table, indexes present, UNIQUE on `uniq_events_msg`, idempotent re-apply), `tests/stackunderflow/etl/test_registries.py` (7: register/get/all, copy semantics, last-wins overwrite for both registries), `tests/stackunderflow/etl/test_watermark.py` (9: missing→0, set/get round-trip, overwrite, ts stamping, per-mart independence, empty-registry refresh, advance + idempotent + pickup-from-existing-watermark), `tests/stackunderflow/etl/test_backfill.py` (7: empty-store report shape, idempotent re-run, `force=True` drops events + marts + watermarks, `force=True` idempotent, mart refresh runs even with empty normalizers, BackfillReport field-set is locked). Spec at `docs/specs/etl-architecture.md`.
 - **Wave 2A — 4 default-on provider normalizers (`stackunderflow/etl/normalize/`).** Per-provider transforms from raw `messages` rows into canonical `usage_events`. Codex token normalization (subtract cached, fold reasoning) moves out of the pricer into `CodexNormalizer` — single source of truth. Cursor v3 no-per-message-tokens path estimates from `len(text)//4` with `cost_source='estimated'` flag. cost_usd computed once per event during normalization, stored on the row, never recomputed downstream.

pyproject.toml

@@ -117,6 +117,17 @@ ignore = [
 "stackunderflow/infra/discovery.py" = ["UP038"]
 
 
+[tool.pytest.ini_options]
+# Custom markers. ``slow`` is opt-in: the default ``pytest tests/ -q`` run skips
+# anything marked ``slow`` (see the ``-m "not slow"`` filter below) so CI keeps
+# the fast feedback loop. The Wave 4E real-data integration + per-route
+# regression suite under ``tests/stackunderflow/integration/`` is gated on this
+# marker — run those explicitly with ``pytest -m slow``.
+markers = [
+    "slow: long-running real-data integration / latency regression tests (skipped by default; run with `pytest -m slow`)",
+]
+addopts = "-m 'not slow'"
+
 [tool.mypy]
 python_version = "3.11"
 warn_return_any = true

tests/stackunderflow/integration/__init__.py

@@ -0,0 +1,22 @@
+"""Wave 4E — real-data integration + per-route latency regression tests.
+
+Two slow-marker test files live here:
+
+* ``test_etl_pipeline_e2e.py`` — builds a 10K-message synthetic store across
+  five providers, runs the normalize → marts → routes path end-to-end, and
+  asserts every dashboard route returns 200 + non-empty data within its
+  per-route latency budget.
+
+* ``test_route_perf_regression.py`` — parametrises every dashboard route
+  against a pre-populated mart fixture (~100K daily, ~50K session, ~1K
+  project, ~2K provider_day, ~5K model_day rows) and pins each route's
+  cold + warm latency budget. Fails CI if any route regresses.
+
+Both files are gated on the ``slow`` pytest marker (registered in
+``pyproject.toml``) so the default ``pytest tests/ -q`` run skips them. Run
+with ``pytest -m slow tests/stackunderflow/integration -q`` to exercise.
+
+Synthetic stores are always built in ``tmp_path`` (the test never touches
+the user's real ``~/.stackunderflow/store.db``) and adapter normalization
+runs against in-process objects, never against real provider source files.
+"""