A trailing period stops a CJK honorific being recognized:
parse("김민준, 씨") # family 김민준, suffix 씨 ← recognized
parse("김민준, 씨.") # family 김민준, given '씨.' ← not
parse("田中, 様.") # family 田中, given '様.'Why
_vocab.is_initial is ^(\w\.|[A-Z])$, ported verbatim from v1's regexes.py. Python's \w is Unicode-aware, so it matches any word character plus a period — including CJK:
is_initial('A.') True is_initial('씨.') True
is_initial('j.') True is_initial('様.') True
is_initial('V') True is_initial('李.') True
is_suffix_strict applies that as a veto ("V. in John V. Smith is a middle initial, not roman five"), so a period-written CJK honorific is vetoed out of the suffix vocabulary.
The veto is correct for Latin and doing real work there — Smith, I. gives given I. rather than reading it as roman numeral one, which is the right call because a middle initial genuinely is the likelier reading. It is simply the wrong question to ask of 씨.: "an initial" means a single letter standing in for a name, and no CJK character is one.
v1 never had to care, because its suffix vocabulary was Latin-only. 2.1 added twenty CJK entries (#307, #308) into a predicate whose veto was written for Latin.
Measured scope
Over the whole 652-entry default suffix vocabulary, is_suffix_lenient and is_suffix_strict disagree on zero bare lowercase forms and exactly ten written forms:
2. i. v. ← Latin: the veto is correct here
様. 殿. 氏. 군. 님. 씨. 양. ← CJK: the veto is wrong here
So seven of the ten disagreements are this bug, and scoping the regex to Latin would collapse them.
Proposed fix
Narrow the character class — ^([A-Za-z]\.|[A-Z])$ or equivalent — so the veto applies to Latin initials only.
Care needed:
- It is v1-parity-breaking. The differential harness will show it; the diffs want classifying rather than allowlisting.
- Check
is_initial's other callers, not just is_suffix_strict — it is used for middle-initial detection elsewhere, and CJK text reaching those paths may be relying on the current over-match in ways worth measuring before changing.
2. stays vetoed either way (\d is not [A-Za-z]), which is probably right — a bare 2. is not obviously the suffix 2.
Why this is worth doing before 2.1.0
The affected vocabulary is new in 2.1 and unreleased, so fixing it now means the bug never ships. Milestoned v2.1 on that reasoning — move it if you'd rather take the parity break in 2.2 alongside the other vocabulary work.
Related
Surfaced while reviewing whether is_suffix_lenient / is_suffix_strict should be renamed after they produced two composition bugs on #318. The conclusion there was that renaming addresses the wrong problem — the predicates differ in exactly this veto, and the veto is what is actually wrong. #319 covers the structural half (sharing the run-level predicate so consumers stop re-deriving suffix-ness token by token).
A trailing period stops a CJK honorific being recognized:
Why
_vocab.is_initialis^(\w\.|[A-Z])$, ported verbatim from v1'sregexes.py. Python's\wis Unicode-aware, so it matches any word character plus a period — including CJK:is_suffix_strictapplies that as a veto ("V.inJohn V. Smithis a middle initial, not roman five"), so a period-written CJK honorific is vetoed out of the suffix vocabulary.The veto is correct for Latin and doing real work there —
Smith, I.gives givenI.rather than reading it as roman numeral one, which is the right call because a middle initial genuinely is the likelier reading. It is simply the wrong question to ask of씨.: "an initial" means a single letter standing in for a name, and no CJK character is one.v1 never had to care, because its suffix vocabulary was Latin-only. 2.1 added twenty CJK entries (#307, #308) into a predicate whose veto was written for Latin.
Measured scope
Over the whole 652-entry default suffix vocabulary,
is_suffix_lenientandis_suffix_strictdisagree on zero bare lowercase forms and exactly ten written forms:So seven of the ten disagreements are this bug, and scoping the regex to Latin would collapse them.
Proposed fix
Narrow the character class —
^([A-Za-z]\.|[A-Z])$or equivalent — so the veto applies to Latin initials only.Care needed:
is_initial's other callers, not justis_suffix_strict— it is used for middle-initial detection elsewhere, and CJK text reaching those paths may be relying on the current over-match in ways worth measuring before changing.2.stays vetoed either way (\dis not[A-Za-z]), which is probably right — a bare2.is not obviously the suffix2.Why this is worth doing before 2.1.0
The affected vocabulary is new in 2.1 and unreleased, so fixing it now means the bug never ships. Milestoned v2.1 on that reasoning — move it if you'd rather take the parity break in 2.2 alongside the other vocabulary work.
Related
Surfaced while reviewing whether
is_suffix_lenient/is_suffix_strictshould be renamed after they produced two composition bugs on #318. The conclusion there was that renaming addresses the wrong problem — the predicates differ in exactly this veto, and the veto is what is actually wrong. #319 covers the structural half (sharing the run-level predicate so consumers stop re-deriving suffix-ness token by token).