Skip to content

CQT 2.17 (2026-08-24 revision)

Choose a tag to compare

@dhh1128 dhh1128 released this 24 Aug 17:00
· 8 commits to main since this release
cqt2.17
092a1cb

Canonical Quoted Text 2.17 canonicalizes human-readable text so that two chunks which a literate reader would call the same produce identical bytes, ready to be hashed or signed. It is for text that travels by chat, copy/paste, social media and email rather than as a file — the channels that quietly change a quote mark, collapse a space, or turn two hyphens into an em dash.

This release freezes cqt2.17. The identifier denotes one exact function, and the bytes it produces do not change from here. Neither number is a compatibility signal: a change to the logic would be published as cqt3.17, and adopting a later Unicode as cqt2.18, and both would break existing signatures. A defect found in this version is repaired by publishing a new number, leaving this one intact, because signatures already made depend on exactly these bytes.

The specification may still be revised where the output does not move — corrected prose, behavior that was always required but never written down, vectors that pin what was already mandated. Revisions are dated. This is the 2026-08-24 revision.

What is in it

The algorithm is a single pass and a total function. Every input produces output; there are no error conditions. Human text has spelling errors and grammar errors, but it does not have syntax errors, and an algorithm that canonicalizes it has no business complaining about it.

Text that must survive exactly — a command, a hash, a snippet of code — can be marked as a fenced block, an inline code span, or an HTTP(S) URL, and is copied through byte for byte.

Every Unicode-dependent operation is pinned to Unicode 17.0.0, and every character set the algorithm reads is enumerated in the specification, so only normalization depends on the character database a runtime happens to ship.

Conformance

goldens/cqt2.17.json is attached to this release and is normative. 98 vectors; an implementation conforms exactly insofar as it passes all of them.

Six implementations ship in impl/, each with a harness that runs the vectors, and all six run in CI on every push:

language Unicode strategy
python unicodedata2==17.0.0
javascript Node 24+, ICU 78 / Unicode 17 natively
rust unicode-normalization 0.1.25, Unicode 17
go toolchain floor go1.27 for x/text's Unicode 17 tables, plus its own recomposition
java Unicode 16 JDK, corrected by proxy substitution
swift Foundation is Unicode 15.1 with a broken composition pass, so the port supplies its own canonical ordering and composition

That column is the hard part of porting this, and the specification has a section on it. A runtime whose normalizer is not Unicode 17 can often be corrected, but only from the input side — no fix-up applied to the normalizer's output can repair a combining-class difference.

A note for implementers

Test your normalizer on sequences, not just on single characters. While this release was being prepared, golang.org/x/text was found to truncate a supplementary-plane starter to 16 bits before its composite lookup, so U+10041 followed by U+0301 produced U+00C1, and the same clipping in reverse turned Hebrew U+05D2 U+0307 into a Todhri letter. Per-scalar checking cannot find that class of defect. Two vectors here exist to catch it.