Flag lookalike link domains for the moderator, never for the door (#41)

[mayankguptadotcom]

Advances #41. It does not close it — see "What this does not catch" and "Where the signal goes today".

Deferred out of #4, which decided the renderer links https://аpple.example/ with a Cyrillic а because it is a genuine https URL with nothing executable in it. That decision is untouched here. A lookalike domain is a judgement about intent, so it goes to the human gate instead, and layer 5 now names it.

The detection, and its sources

Mixed-script within a single label, not "contains non-Latin script".

• Unicode UTS Optional page-key canonicalisation: path case folding, index and extension folding #39 §5.1 Mixed_Script_Detection — a string is mixed-script when the resolved script set, "the intersection of the augmented script sets over all characters", is empty. That is the algorithm implemented, using Script_Extensions rather than Script, as the standard specifies.
• UTS Optional page-key canonicalisation: path case folding, index and extension folding #39 §5.2 Restriction_Level_Detection is why the tracked set is exactly Latin, Cyrillic, Greek. "Moderately Restrictive" permits a string "covered by Latin and any one other Recommended script, except Cyrillic, Greek" — those two are carved out by name because their letters have Latin twins. One level up, "Highly Restrictive" explicitly permits Latin + Han + Hiragana + Katakana, because that is simply Japanese. Tracking three scripts rather than all of them is not a shortcut; it is the standard's own carve-out.
• MDN, Unicode character class escapes — \p{Script_Extensions=…} with the u flag, ES2018, baseline since 2020. No new dependency and no table in this repo: the script data is the engine's own, so there is nothing here to go stale. workerd is V8 and the tests run in workerd, so support is asserted rather than assumed.
• WHATWG URL, domain to ASCII — why the host is parsed as text and never with new URL(): the parser runs the host through IDNA ToASCII, so аpple.example comes back xn--pple-43d.example and the Unicode form, which is the entire subject of the check, is gone. Checked 2026-07-25: new URL('https://аpple.example/').hostname → xn--pple-43d.example.
• WHATWG URL, forbidden domain code point — cited in the file for an honest downgrade, below.

Percent-decoding is load-bearing and verified rather than assumed: new URL('https://%D0%B0pple.example/').hostname → xn--pple-43d.example, checked 2026-07-25. A browser resolves that to the lookalike host; a reader cannot decode it; undecoded it is pure ASCII and would have sailed past.

Why review and never reject

Argued in the file, not just here. пример.рф is a Russian domain the way example.uk is a British one, and no property of the string separates "IDN belonging to someone who does not write in Latin" from "IDN chosen to impersonate a Latin brand" — only intent does, and intent is what a moderator has and a regular expression does not. A reject would therefore fall entirely on non-Latin-script readers, silently, in the name of protecting them. That is a discrimination bug wearing a spam control's clothes.

The same reasoning produces the per-label rule. Nearly every IDN sits under an ASCII TLD, so пример.com mixes scripts as a host while no label of it mixes anything. Judging the host would flag the entire IDN namespace.

What this does not catch

Stated in the file as well, so it is visible to whoever picks it up:

• Whole-script confusables. аррӏе.example is Cyrillic end to end and reads as "apple". Nothing is mixed, so no mixed-script measure can see it. UTS Optional page-key canonicalisation: path case folding, index and extension folding #39 answers this with a separate mechanism (§4, Confusable Detection) needing the confusables mapping — thousands of entries — plus a list of names worth impersonating, because the alternative rule, "flag every wholly non-Latin label", is the discrimination bug with extra steps. There is a test asserting this gap rather than a comment implying it is covered.
• Punycode. An xn-- label is ASCII, so it is never mixed here. Deciding it needs a real decoder, not a prefix match: xn-- is exactly how a legitimate Cyrillic domain looks when copied out of the address bar. Named in Flag lookalike link domains in the moderation queue #41's body; still open there.
• Link text disagreeing with its href. A different attack with a different answer.

Both of the first two remain on #41, which is why this is "Advances".

Where the signal goes today — please read before assuming this is finished

The verdict is content: lookalike-domain, and today it reaches the Worker log only. src/submit/pipeline.ts reads the verdict's action and drops its reason, which is #70 (owned by another session). So a moderator opening the queue sees the held comment but not why.

Rather than imply otherwise, there is an end-to-end test that drives the real Worker and asserts the log line — the whole of the feature's output today:

{"event":"spam_verdict","action":"review","layer":"content","reason":"lookalike-domain","pageKey":"/notes/leaving"}

Once #70 carries the reason through, the signal lands in the queue with no change needed here.

Kill-shots (card rule 6)

Every guard disabled, test run, restored. Real output, pnpm vitest run test/worker/spam/lookalike.test.ts.

#	Guard disabled	Result
1	isMixedScript returns false	Tests 9 failed | 14 passed — × Cyrillic а, × Greek ο, × userinfo host, × percent-encoded @, × percent-encoding, × holds for review, × is never a reject, × www-prefixed, × layer holds
2	judge the whole host, not each label	Tests 1 failed | 22 passed — × judges each label on its own, so a Cyrillic name under a Latin TLD is fine
3	add Han to the tracked scripts	Tests 1 failed | 22 passed — × does not flag Latin mixed with a script that is not confusable with it
4	stop percent-decoding the host	Tests 2 failed | 21 passed — × percent-encoded @, × sees through percent-encoding
5	keep the userinfo in the host	Tests 2 failed | 21 passed — × reads the host after any userinfo, × percent-encoded @
6	do not stop the host at punctuation	Tests 1 failed | 22 passed — × stops the host at punctuation, so a link written mid-sentence is not flagged
7	lookalikeOutcome returns reject	Tests 3 failed | 19 passed — × holds for review and names why, × is never a reject, × layer holds a comment whose only fault is a lookalike
8	let the lookalike review outrank a reject (in content.ts)	Tests 1 failed | 37 passed (with content.test.ts) — × still rejects a link flood that happens to contain a lookalike
9	make the renderer refuse non-ASCII URLs (in markdown.ts, reverted)	Tests 1 failed | 21 passed — × renders a lookalike link exactly as it did before this signal existed

Two guards were removed rather than kept because they could not be shot:

• An "ignores a port" test passed with the punctuation rule disabled — a colon and digits belong to no script, so рф:8443 resolves to Cyrillic either way. It was coverage theatre; it is gone, and the file says why there is no port case.
• The false-positive claim in the header originally named Devanagari, Arabic, Han and Thai as "enforced by" the test file, where only Han was tested. Devanagari and Arabic cases are now actually there.

Both of those came out of the review pass, along with a real false positive found before the tests existed: https://пример.рф,and — the link pattern stops at whitespace, so a comma written after a URL came along with it, and рф,and is Cyrillic beside Latin. A comma has done nothing wrong. Fixed by cutting the host at the first character that cannot be in a domain name, and kill-shot 6 is that guard.

pnpm check

Test Files  48 passed (48)
     Tests  848 passed (848)
[ok] embed: src/embed/index.ts is 6791 B gzipped of 10240 B allowed
[ok] pnpm is the only lockfile here, its version is pinned to a major the Cloudflare build image runs (10.x), and its format is one that image reads (9.0)
[ok] every package licence is within policy
[ok] every action reference is pinned to a full commit SHA
No known vulnerabilities found

Skills used: security-and-hardening, source-driven-development (every Unicode and URL claim above is a fetched citation, none from memory), test-driven-development (failing test first — the first run failed on the missing module), improve over the branch, code-review-and-quality including a fresh-context adversarial reviewer, whose three findings are all addressed above, superpowers:verification-before-completion.

Scope: src/spam/lookalike.ts (new), src/spam/content.ts, test/worker/spam/lookalike.test.ts. No dependency added; package.json and pnpm-lock.yaml untouched. Nothing in migrations/, src/db/, src/render/, src/embed/ or the dashboard.

🤖 Generated with Claude Code

[mayankguptadotcom]

Coordinator verification

The discrimination risk is genuinely handled — I probed it directly

This was the concern in the brief: a heuristic that flags non-Latin script would silently penalise readers whose domains are legitimately not in Latin. I wrote my own probe against nine real internationalised domains rather than reading the tests:

https://пример.рф/        wholly Cyrillic, Cyrillic TLD
https://пример.com/       wholly Cyrillic label, ASCII TLD
https://例え.jp/           Japanese
https://中国移动.中国/      Han
https://بيت.شبكة/         Arabic
https://नमस्ते.भारत/        Devanagari
https://sony製品.jp/       Latin + Han
https://example.com/
https://sub.domain.example.co.uk/path?q=1

→ flagged: []            ✓ none

And both attacks still caught:

https://аpple.example/    Cyrillic а + Latin
https://gοogle.example/   Greek ο + Latin

→ missed: []             ✓ none

Judging per label rather than per host is the decision that makes this work. Nearly every IDN sits under an ASCII TLD, so пример.com mixes scripts as a host and nothing as a label — judging the host would have flagged the entire IDN namespace. That is the difference between a useful signal and a bug that falls on exactly the people who did nothing wrong.

Restricting to Latin/Cyrillic/Greek is the standard's carve-out, not a shortcut

I had assumed this would be an approximation. It is not: UTS #39 §5.2 "Moderately Restrictive" permits Latin beside any one other Recommended script except Cyrillic and Greek, and "Highly Restrictive" explicitly permits Latin + Han + Hiragana + Katakana. So sony製品 being clean and аpple being flagged is the specification's own line, not a convenient one.

The two findings I would not have anticipated

new URL() destroys the thing being checked. The WHATWG parser IDNA-encodes the host, so new URL('https://аpple.example/').hostname is xn--pple-43d.example — the Unicode form the whole check is about is gone before it can be inspected. Parsing the host as text is not a shortcut here; it is the only way the check can work at all. Verified rather than asserted.

A real false positive found before the tests existed: https://пример.рф,and — a comma written after a URL is captured by the link pattern, and рф,and is Cyrillic beside Latin. That is ordinary prose producing a flag. Found by driving it, fixed by cutting the host at the first non-domain character.

Two guards deleted rather than kept

A port test that passed with its guard disabled, and a header claim naming four scripts as test-enforced when only Han actually was. Deleting a test that cannot fail, and correcting a comment that overstated its coverage, is the right response — both were the exact defect CLAUDE.md describes, where the artefact sits where a reader goes to verify the property.

Correctly titled "Advances", and the limits are asserted rather than described

Whole-script confusables (аррӏе.example, Cyrillic end to end) are not caught, and there is a test asserting the gap rather than a comment implying coverage. Punycode is not decoded. Both are named on the issue.

The signal reaches the Worker log today and no further, because src/submit/pipeline.ts drops the verdict's reason — that is #70, owned by the other session. A moderator sees the held comment but not why. Saying so plainly, rather than closing the issue, is right.