feat(check): opt-in --layout proseCoverageFloor - #2834
Conversation
Keep the default prose coverage floor at 0.15 for all callers, and allow stricter agents (e.g. Zephyr) to lower it via --layout "proseCoverageFloor=0.05" without changing other layout gates. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
miguel-heygen
left a comment
There was a problem hiding this comment.
The plumbing is clean and the default-off shape is good: omitting --layout leaves the browser audit on its existing 0.15 default, while the explicit option is threaded through one typed LayoutOptions envelope.
Two changes before I can stamp this:
-
[important]
packages/cli/src/commands/check.ts:253-257— reject trailing junk instead of partially parsing it.Number.parseFloataccepts prefixes, so values such asproseCoverageFloor=0.05garbage,0.1%, and1e-1oopssilently become valid floors. That contradicts the command's advertised fraction grammar and the test name claiming malformed specs are rejected. Parse the whole token (for example withNumber(raw)after the existing trim) and pin at least one trailing-junk case. -
[important]
packages/cli/src/commands/layout-audit.browser.test.ts:1781-1795— pin the actual opt-in behavior in the browser audit. The added tests stop at parser →runPipelineoptions. None executes the changedauditLayout(options)branch, so the core contract in this PR can regress while every new test stays green. Extend the existing ~0.07 prose fixture to prove: default options still produce notext_occluded;proseCoverageFloor: 0.05produces the finding; and atomic-label behavior remains unchanged. Ideally also retain one driver-level assertion thatcollectLayoutforwards the value into the page evaluation payload.
CI is otherwise green except the still-running Windows test lane at this exact head.
— Magi
Verdict: REQUEST CHANGES
Reasoning: The implementation direction is right, but the new CLI parser accepts malformed values and the browser-side feature itself is not regression-tested.
Reject trailing-garbage fractions that Number.parseFloat would accept, and pin the existing ~0.07 coverage fixture for default vs floor=0.05 (atomic labels unchanged) plus a collectLayout forwarding assertion. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Addressed in 46579aa:
|
james-russo-rames-d-jusso
left a comment
There was a problem hiding this comment.
Reviewed at 46579aa5fdbd2effec70a4ddbbb531ea4f26fcd1 — Miguel's two [important] asks are addressed in the fix(check): strict proseCoverageFloor parse + pin 0.07 floor tests commit. Layering with the AI-authored / cross-repo lens.
Miguel's asks — closed
- Strict parse (
packages/cli/src/commands/check.ts:255-257):Number()instead ofNumber.parseFloat, plus an empty-string guard. The new test cases atcheck.test.ts:1159-1161pin"0.05garbage","0.1%", and empty string — all reject cleanly. Good. - Browser-side opt-in behavior test (
packages/cli/src/commands/layout-audit.browser.test.ts:1786-1806): two new tests exercise the changedauditLayout(options)branch — prose fires atcoveredFraction=0.07when the floor is lowered to0.05; atomic-label behavior unchanged. Plus the "ideal also" driver-level assertion Miguel called out lands atcheck.test.ts:1186-1192—it("forwards layout options into driver.collectLayout")pins theproseCoverageFloorvalue all the way to the driver-mock's arg. Cleanly closed.
Fresh concerns
🟡 packages/cli/src/commands/check.ts:216 — sibling parser still has the parseFloat prefix-parse bug. parseFrameCheckField for tol reads const tol = Number.parseFloat(raw), so --frame-check "tol=2px" or tol=4garbage parses to 2 / 4 silently — same class of bug your fix here just corrected for proseCoverageFloor. If the pattern is worth codifying (I'd argue yes), extracting a parseFraction(raw, min, max) / parseNumberStrict(raw) helper and reusing it across parseFrameCheck and parseProseCoverageFloor would carry the fix everywhere in one hop. Not blocking this PR — the immediate goal is scene-6 unblock — but the follow-up is worth naming.
🟡 Cross-repo wire contract. The PR body's "After merge" plan says the CLI will be released as 0.7.77, then EF bumps HYPERFRAMES_VERSION and Zephyr updates CHECK_OPTIONS. That makes the exact --layout grammar a wire contract for at least those two consumers. Two suggestions:
- Document the canonical string Zephyr will emit in the PR body (e.g.
--layout "proseCoverageFloor=0.05"— quoted-single-token, no semicolons yet). Future flag additions (proseCoverageFloor=0.05;anotherField=X) should stay backwards-compatible with the single-field form Zephyr will hardcode. - Consider whether EF and Zephyr should pin CLI version in a way that fails loud if the CLI upgrades past the wire contract. Not this PR's concern, but worth naming so the multi-repo coordination doesn't become silent.
Per [[wire-contract-summary-for-coordinated-multi-repo]] — since this is the CLI slice of an FE↔BE↔CLI series, leading with the DTO summary (the exact quoted flag string) in the PR body makes downstream slices' review much easier.
Nits
packages/cli/src/commands/layout-audit.browser.js:1420-1423— the browser-side clampMath.min(1, Math.max(0, options.proseCoverageFloor))is now redundant with the CLI validation (parseProseCoverageFloorrejects<0 || >1). Not a bug — defensive against non-CLI callers of__hyperframesLayoutAudit— but the comment could mention that the audit is also invoked from browser dev-tools / scratch runs so the clamp isn't dead code.- Grammar shape: this uses the semicolon-key=value grammar mirroring
--caption-zone, but--layoutcurrently has only one field. Fine — future-oriented — but if that stays true for a while, the single-field cost is a minor UX friction (users typing--layout proseCoverageFloor=0.05vs a bare--prose-coverage-floor=0.05).
What I didn't verify
- Full scene-6 dump on a real project — trusted the PR body's manual verification.
- Zephyr's actual
CHECK_OPTIONSshape — I don't have visibility into that repo from here. - The CLI-side
parseCaptionZonesibling — sameparseFloatpattern? Skipped for scope; worth a follow-up sweep.
LGTM from my side once Miguel's stamps this — leaving as a comment. Approval's Miguel's.
miguel-heygen
left a comment
There was a problem hiding this comment.
R2 at 46579aa5f closes both prior findings cleanly.
packages/cli/src/commands/check.ts:253-259now parses the entire token withNumber(), explicitly rejects the empty string, and pins the trailing-junk cases that previously slipped through.packages/cli/src/commands/layout-audit.browser.test.ts:1781-1806now locks the actual browser contract: ~0.07 prose stays quiet at the 0.15 default, becomestext_occludedat a 0.05 floor, and atomic-label behavior remains unchanged.packages/cli/src/commands/check.test.ts:1184-1192additionally pins the pipeline → driver forwarding seam.
Local verification at this exact head: full workspace build green; targeted browser-audit suite 78/78; targeted check suite 44/44. The rewritten-head CI matrix is still running, with no failures at review time.
— Magi
Verdict: APPROVE
Reasoning: Both requested correctness/test gaps are fixed and pinned at the exact head; no remaining code blocker found.
Sweep the sibling --frame-check tol parser (and caption fractions) onto the same strict Number() helper so trailing garbage cannot prefix-parse. Co-authored-by: Cursor <cursoragent@cursor.com>
miguel-heygen
left a comment
There was a problem hiding this comment.
R3 at a0d068ab3 closes the follow-up parser nit cleanly.
packages/cli/src/commands/check.ts:216-218now routes--frame-check tolthrough the same strict whole-token parser asproseCoverageFloor, sotol=4pxandtol=2garbageno longer prefix-parse.packages/cli/src/commands/check.ts:242-247centralizes the invariant inparseNumberStrict; caption fractions reuse it without changing their already-strict behavior.packages/cli/src/commands/check.test.ts:1150-1151positively pins both malformed tolerance cases.
Local exact-head verification: check.test.ts 45/45; formatting and lint clean. CI has no failures; the two Windows jobs and CLI smoke are still running at review time.
— Magi
Verdict: APPROVE
Reasoning: The requested sibling-parser cleanup is minimal, shared, and regression-pinned; no new blocker found.
Summary
--layout "proseCoverageFloor=<0–1>"(semicolon key=value, same grammar as--caption-zone) so callers can lower the prosetext_occludedcoverage floor without changing OSS defaults.--layoutis unchanged for everyone else.layout.proseCoverageFloorthrough check parse → browser audit (auditLayout).proseCoverageFloor,--frame-check tol, caption fractions) use a sharedparseNumberStrict(Number(), notparseFloat) so trailing garbage is rejected.Zephyr / EF wire contract
Post-release, experiment-framework pins
HYPERFRAMES_VERSIONand ZephyrCHECK_OPTIONSconsume this grammar as a cross-repo contract. The exact argv Zephyr emits today:Built by EF
_layout_check_specasproseCoverageFloor={float:g}fromlayout_options(prose_coverage_floor=0.05)→CHECK_OPTIONS["layout"]. Future--layoutkeys should stay semicolonkey=valueand keep this string parseable (additive fields only; do not renameproseCoverageFloor).Why
Long titles with padded image overlays can sit at ~0.11 coveredFraction and get discarded under the default floor. Zephyr wants a stricter opt-in (0.05) for scene6-class occlusion without making HyperFrames noisier for other agents.
Test plan
bun run test src/commands/check.test.ts(strict parse + CLI/driver threading)bun run test src/commands/layout-audit.browser.test.ts(~0.07 fixture: default silent, floor 0.05 fires, atomic unchanged)text_occluded;--layout proseCoverageFloor=0.05→text_occludedwithcoveredFraction: 0.11HYPERFRAMES_VERSION+ land ZephyrCHECK_OPTIONS(EF #43493)