Skip to content

v3.5.2: number-grammar digits are ASCII only (errata)

Choose a tag to compare

@blackwell-systems blackwell-systems released this 11 Aug 04:17
· 116 commits to main since this release

Spec errata: number-grammar digits are ASCII only (Sections 2.3, 2.4)

The number grammar's DIGIT rules are ASCII 0-9 per RFC 5234; non-ASCII Unicode decimal digits (Arabic-Indic U+0660-U+0669, fullwidth U+FF10-U+FF19, etc.) are not digits. A token such as 1.٥ (ASCII 1, ., then U+0665) does not match the number grammar and decodes as the string "1.٥", never the number 1.5.

This is not a grammar change (ABNF DIGIT was already ASCII); the note makes the intent explicit and forbids a Unicode-mode regex \d (which matches \p{Nd}), plus tightens the numeric-like classification (Section 2.4) to the exact implemented rule.

Why it mattered

Several SDKs (gcf-python, gcf-rust, gcf-dotnet, gcf-swift) used a Unicode-mode \d, so they disagreed with the ASCII SDKs (gcf-go, gcf-typescript, gcf-kotlin) on whether to quote a value like "1.٥". A bare 1.٥ emitted by an ASCII encoder then decoded back as the number 1.5 under gcf-python (whose float() folds Unicode digits): a silent cross-SDK losslessness violation on mixed pipelines. Pinning \d to ASCII [0-9] fixes both the quoting divergence and the corruption.

Conformance

Four new fixtures (scalar/029, scalar/030, scalar/031, decode/007) lock the ASCII-only behavior: "1.٥" encodes bare, and a bare 1.٥ decodes to the string. Verified across all seven SDKs (269 conformance fixtures) and roughly 5,500 cross-SDK differential-fuzz iterations with non-ASCII-digit inputs, zero divergences.

The four affected SDKs are patched in their accompanying releases; gcf-go, gcf-typescript, and gcf-kotlin were already compliant.

Full changelog: CHANGELOG.md