v1.18.0 — measurement harness + input forensics v2

Two workstreams: a measurement harness for the detection layer, and a set of accessibility and scoring corrections it surfaced.

The benchmark harness (bench/)

FCaptcha's false-positive behaviour — keyboard-only users, privacy extensions, touch users — is now measured rather than asserted.

bench/ drives a real Chromium through real input events, intercepts the /api/verify body the widget produces, and replays that labeled corpus against a running server — reporting false-positive rate per persona, true-positive rate per agent class, and a per-signal FP budget that gates CI. 14 human personas (keyboard-only, screen-reader, touch, trackpad, tremor, slow/corrected, elderly, throttled, DevTools-open, privacy-extension, typing, pasting, scrolling) and 8 agent personas.

Read bench/README.md before quoting any number from it. A 0% FPR there means "no sample in this corpus crossed the threshold", not "0% of real users would". The corpus is scripted personas from one machine, and the panel's environment is partly reconstructed because an automated browser announces itself. Both limits are documented rather than buried.

Accessibility and false-positive fixes

The first three affect accessibility populations directly:

1. Users with no mouse were flagged for an "unnaturally direct mouse path." The client reports directness 1 when there is no path to measure, so it fired on every keyboard-only, screen-reader and touch visitor — precisely the populations the neighbouring checks exempt.
2. The touch exemption required 3 touch events; a plain tap produces 1. A mobile user who taps without scrolling collected three agent detections, including "Zero mouse, touch, or keyboard events recorded" at confidence 0.9.
3. Slow users were read as automated. "Event rate abnormally low" and "velocity too consistent" fired on the elderly and motor-impaired personas. Both now stand down when the movement independently looks like a hand.
4. Forwarding headers were suspicious even from a trusted proxy — a permanent bot detection on every visitor to every proxied deployment. It fired on 100% of the human panel and 100% of the agent corpus.
5. Pasting was treated as a bot behaviour. People paste addresses, error messages and one-time codes constantly.
6. uvicorn resolved client IPs behind FCaptcha's back, so the Python server checked the visitor's address against the trusted-proxy list instead of the proxy's. Same class as chi's middleware.RealIP (removed in v1.16.0) and Express's trust proxy.
7. /api/score skipped the JA4 trust gate /api/verify applies, letting an untrusted client present its own TLS fingerprint on that endpoint.

Measured effect: touch 0.385 → 0.040, keyboard-only 0.206 → 0.040, screen-reader 0.190 → 0.035, elderly 0.299 → 0.097.

Scoring: evidence now accumulates

Within-category aggregation moves from a confidence-weighted mean to noisy-OR, so corroborating signals now strengthen a verdict rather than dilute it. Under the previous scheme a browser reporting navigator.webdriver = true alone scored 0.95, while the same browser with five additional automation indicators scored 0.686.

Replaced with noisy-OR. Candidates were compared on identical evidence (bench/tools/compare-aggregation.js), not by argument.

Self-declared automation now sets a 0.9 floor. A weighted sum across eleven categories means no single fact can consume much of the budget, so a blatant local agent landed near 0.5 — "challenge", not "block". The bar for the dispositive mark is that a browser cannot produce the signal without being automated: navigator.webdriver and ChromeDriver/Puppeteer globals. The DevTools console-attach probe is excluded — the human panel proves it fires on a developer.

Agent TPR at the 0.8 threshold: 0% → 97.33%, with every captured class at 100%.

Input forensics v2

Five checks on how input arrived, with every threshold derived from the corpus on real hardware rather than copied from published figures:

signal	human	scripted agent
inter-key interval	median 226.9ms	median 7.9ms
interval variance	4549	8
key hold	median 82.0ms	median 7.8ms
scroll max step	109px	704px

• Typing cadence floors — gated behind ten keystrokes and no paste, because a paste is two keydowns a millisecond apart and is arithmetically identical to a fast agent
• Paste/platform contradiction — Ctrl+V from a client claiming macOS, with the paste actually landing. No threshold: the client has disagreed with itself
• Programmatic fill — synthetic input event with no keystrokes. Deliberately the weakest, since autofill looks like this
• Scroll morphology — the page moved further in one event than a hand can move it. Stands down for keyboard users, since PageDown jumps a viewport
• Font/platform coherence — scores the contradiction, never the enumeration

The keystroke-overlap check was not built: an earlier design attributed it to FP-Agent, the paper does not say it, and there is no measurement for it here either.

Known gaps

• Source-patched browsers remain an open area of work.
• Declared agents are scored as a policy decision for the operator rather than a block, by design.
• The benchmark corpus is one machine. Captured traces from real browsers driven by real people are the most useful contribution anyone could make.

Compatibility

No API or configuration changes. Scores shift downward for legitimate visitors and upward for unhidden automation; if you have tuned custom thresholds against the old composition, re-check them.

Tests: Node 93/93 E2E + 39 unit; Go 89/93 E2E + unit; Python 80/93 E2E + 33 unit. Go and Python residuals are pre-existing documented divergences.