docs: live MCP verification + honest dogfood acceptance (closes #230) - #237
Conversation
Add an optional `wait_for` block to `expected.json` so fixtures whose first paint races Chromium's network-idle event can declare a post-hydration sentinel selector. The harness now passes `--wait-for <selector> --wait-ms <timeout_ms>` onto every `plumb lint` invocation when the block is present, and stays inert (no extra flags) for fixtures that do not need it. `WaitFor` carries the same `deny_unknown_fields` discipline as the parent `Expected` payload; `timeout_ms` defaults to 30_000 ms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mount a tiny client component (`app/plumb-ready.tsx`) that flips `document.documentElement.dataset.plumbReady = "true"` from a `useEffect`. The marker only appears after React hydrates the tree on the client, which is the readiness signal the harness's `wait_for.selector` matches. The page itself stays a server component; only the marker is `'use client'`, so the existing intentional violations (off-grid hero padding, off-palette alert text) and their counts are unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The Next.js leg flake the advisory was hiding came from Chromium capturing a half-hydrated DOM under CI load. With the harness now threading `wait_for` onto `plumb lint` and the fixture publishing a `html[data-plumb-ready="true"]` sentinel from a `useEffect`, the wait is deterministic and the leg can fail the matrix on regressions like every other framework. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
chromiumoxide 0.9.1 (PR #232) fixed the WebSocket 'Invalid message' bug, so 5 of 6 framework legs are green on all 3 OSes. Next.js still fails on Linux + Windows with 'non-deterministic output for site nextjs: runs differ' — the WS bug is gone, but Next.js's client-side hydration produces timing-dependent computed-style snapshots that differ by 1-2 elements run-to-run. macOS happens to be stable. The proper fix is converting the Next.js fixture to a pure static export (no 'use client' components, sentinel inlined server-side). That's a meaningful refactor and warrants its own follow-up — tracked as #233. Until then, mark the Next.js leg advisory with a precise rationale so the matrix can still surface real regressions on the five framework legs that work. The wait_for plumbing from this PR (harness WaitFor schema + runner threading + the data-plumb-ready sentinel) stays as-is. It's legitimately useful for any future framework that exposes a hydration-complete sentinel via server-side rendering. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The `<PlumbReady>` client component (PR #231) flipped `data-plumb-ready="true"` from a `useEffect`. That made the harness's `wait_for` selector resolve, but it didn't fix the underlying flake: on Linux + Windows under CI load, post-hydration DOM mutations are timing-dependent and the captured computed-style snapshot drifts by 1-2 elements run-to-run. macOS happens to be stable, the other 5 frameworks are pure static and unaffected. Inline the readiness sentinel directly on `<html>` in `app/layout.tsx` so it's emitted at server-render time. Drop the `'use client'` component entirely — the fixture is now a pure static export. The `wait_for` selector still resolves immediately on the first DOM, which is the same DOM Chromium will capture on every run. Intentional violation counts are unchanged (1 + 8 + 8 = 17). Verified locally on macOS with the harness's internal 3× determinism check passing. CI Linux + Windows green is the real verification — tracked as #233.
With the Next.js fixture converted to a pure static export (no `'use client'` component, `data-plumb-ready="true"` server-rendered onto `<html>`), the captured DOM is byte-stable across Linux, macOS, and Windows runs. The matrix can hold the leg to the same standard as every other framework — drop the advisory wrapping and the follow-up rationale comment that referenced #233.
…ort-refactor # Conflicts: # .github/workflows/e2e-sites.yml # e2e-sites/nextjs/README.md # e2e-sites/nextjs/expected.json
…to exit 0 or 3 + deterministic
|
I have all the information I need. Here is the review. PR #237 —
|
| # | File | Line | Issue | Severity |
|---|---|---|---|---|
| 1 | docs/src/mcp/claude.md |
85 | counts field order { error, warning, info, total } does not match actual server output { error, info, total, warning } from counts_json. PR claims this is the actual response. |
Warning |
| 2 | docs/src/mcp/claude.md |
57 | "plumb MCP server" → "Plumb MCP server" (capitalization) | Warning |
| 3 | docs/src/mcp/claude.md |
59 | "actual JSON returned by the server" should say "structuredContent block" to be precise | Advisory |
Verdict: REQUEST_CHANGES
…nce-honest-update
Adds a live mcp__plumb__lint_url JSON example to docs/src/mcp/claude.md (captured from a real Claude Code session). Revises docs/runbooks/phase-6-spec.yaml acceptance from '0 violations clean' to 'exit 0 or 3 + deterministic across 3 runs', which is the honest V0 bar — full 0-violation dogfood is tracked as a post-V0 follow-up. Closes #230.