Scope leading-particle behaviour to the default name order, and fix the ambiguity detail (#355) - #358
Merged
Merged
Conversation
The PARTICLE_OR_GIVEN emitter in `_assign` hardcoded "read as a given
name" into its detail text. Roles are assigned twenty lines above it,
from `_effective_order`, so under `Policy(name_order=FAMILY_FIRST)` the
head piece is Role.FAMILY and the report described the reading the
parse did not take:
Parser(policy=Policy(name_order=FAMILY_FIRST)).parse("Van Johnson")
-> family='Van', given='Johnson'
detail="leading 'Van' may be a family-name particle;
read as a given name"
`detail` is user-facing -- it is read off `ParsedName.ambiguities` and
printed -- so this was a false statement about the parse in hand, not
an internal inaccuracy.
The fix is the one the same function already applies twelve lines
above: the SUFFIX_OR_NAME block computes its "read as X rather than Y"
from `token.role.value`, and `AmbiguityKind.SUFFIX_OR_NAME`'s docstring
states the principle -- which name part was declined depends on
position and `name_order`, so `detail` names it rather than the kind.
The particle emitter now reads the role off the same token.
The `kind` is deliberately unchanged. The fork genuinely is "particle
or given" -- that is what the two readings of "Van Johnson" are, in
either order -- and the enum member is public API. Only the
human-readable text moved; default-order output is byte-identical.
Test lands in tests/v2/pipeline/test_assign.py rather than
tests/v2/test_policy.py: the string is the assign stage's output, that
file owns the emitter's module and already holds the sibling
PARTICLE_OR_GIVEN test, and test_policy.py never runs the pipeline --
it exercises Policy/PolicyPatch construction and validation only.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Six places described a leading particle by naming the FIELD the piece
ends up in. Each was written before `Policy(name_order=FAMILY_FIRST)`
shipped in 2.1, and each is falsified by it. Measured:
default FAMILY_FIRST
Van Johnson given='Van' family='Johnson' family='Van' given='Johnson'
van Gogh given='van' family='Gogh' family='van' given='Gogh'
de la Vega given='' family='de la Vega' family='de' given='la Vega'
The rule itself runs in the grouping stage, before roles are assigned,
and never consults `policy.name_order` -- so every MECHANISM claim at
these sites was already right and is kept verbatim where it stood: a
leading particle has no surname to attach to yet and chains nothing,
and whether it can double as a given name is what decides the branch.
Only the destinations needed scoping, and they are scoped to the
default order rather than restated per order.
Follows the pattern PR #354 landed in `config/particles.py`: mechanism
first, then where the piece lands under the default given-first order,
with the order-dependence stated instead of implied. That docstring
carries the long version for both orders, so these are deliberately
economical -- the two `.rst` prose sites and `concepts.rst` take a
clause, and only `_lexicon.py` and `_types.py` spell out what
membership decides under EITHER order (the ambiguity report, which is
measured identical under both).
`_types.py`'s `PARTICLE_OR_GIVEN` now closes the way its sibling
`SUFFIX_OR_NAME` three entries up already did -- the declined part
depends on `name_order`, so `detail` names it rather than the kind --
which the previous commit made true of the emitter.
The sixth site is the paragraph one below the fifth in customize.rst,
outside the issue's enumeration but the same defect: "take it out of
the ambiguous set and leading `van` becomes part of the surname" is a
destination stated unconditionally. It gets the section's one existing
caveat rather than a second copy of it, and leads with the effect that
does hold under every order. That effect is the ambiguity, not a
field -- under FAMILY_FIRST the knob moves nothing else:
van LISTED default given='van' family='Gogh' amb=[particle-or-given]
van LISTED FAMILY_FIRST given='Gogh' family='van' amb=[particle-or-given]
van REMOVED default given='' family='van Gogh' amb=[]
van REMOVED FAMILY_FIRST given='Gogh' family='van' amb=[]
No doctests were added or changed. The existing ones in usage.rst and
customize.rst all use the default `parse()` and remain true;
`sphinx -b doctest` is 223 tests, 0 failures, and a fresh
`sphinx -b html` build emits no warnings. Per the lean-docs rule in
AGENTS.md, the non-default order gets prose and the unit test added in
the previous commit, not a demonstration block.
Deliberately untouched: `post_rules` rule 1b keys on roles, so the
never-given fold that makes "de Mesnil" wholly a surname does not fire
under FAMILY_FIRST -- which is also why the bottom two rows above are
identical. That asymmetry is a behavior question filed separately, and
nothing here asserts what a non-default order does to a never-given
particle.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #358 +/- ##
=======================================
Coverage 98.50% 98.50%
=======================================
Files 44 44
Lines 2881 2883 +2
=======================================
+ Hits 2838 2840 +2
Misses 43 43 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
The previous commit stopped hardcoding "given" in the PARTICLE_OR_GIVEN detail, but the test it shipped with only used inputs where the assigned role and `policy.name_order[0]` agree. So an implementation reading `state.policy.name_order[0].value` -- which is the wrong answer, and the exact falsehood #355 was filed about -- passed the whole suite and all 223 doctests. Measured, before this commit, with that substitution applied to a copy of the tree: 3110 passed, 20 skipped, 11 xfailed The two implementations come apart on the script_orders path (#271), which `_effective_order` resolves ahead of `name_order` and which therefore flips the effective order without touching `name_order` at all. With a Han ambiguous particle: parse('毛 泽东') policy.name_order[0] given role actually assigned family `test_leading_particle_detail_follows_the_effective_order` is that case, and it is the only test the substitution now breaks: FAILED tests/v2/pipeline/test_assign.py:: test_leading_particle_detail_follows_the_effective_order 1 failed, 3116 passed, 20 skipped, 11 xfailed `_assign`'s new comment named the same wrong mechanism -- it said the role "follows name_order" nine lines under `order = _effective_order(...)`, writing the wrong implementation into the comment that justifies the right one. It now says where `order` actually comes from. The `_group` half of this kind was unpinned in the other direction: replacing its whole detail string with `f"{tokens[i].text!r} MUTATED DETAIL TEXT "` also passed everything. That emitter's order-invariance is load-bearing -- it is why the leading-particle docs can scope DESTINATIONS to the default order without qualifying this text -- so `test_chained_particle_detail_is_order_invariant` asserts the exact string under all three orders. "Dr. Van Johnson" takes the chained branch under every one of them (family 'Van Johnson', identical detail), so the assertion is the same string three times, which is the point. Under the mutation, 3 failed. `test_leading_particle_detail_names_the_role_it_took` is parametrized over FAMILY_FIRST_GIVEN_LAST as well, and the facade-level twin sits beside its exact sibling `test_ambiguous_acronym_detail_names_the_role _it_got` -- `detail` is public output, and nothing checked that the role it names survives assembly into `ParsedName`. Tuple-unpacking (`(amb,) = ...`) replaces `.ambiguities[0]` throughout, matching test_parser.py's existing style and pinning the count once. No behavior change: tests and one comment only. pytest 3117 passed, 20 skipped, 11 xfailed mypy Success: no issues found in 103 source files ruff All checks passed! sphinx -b html build succeeded sphinx -b doctest 223 tests, 0 failures Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous sweep replaced a field-destination error with a MECHANISM
error at two of its six sites, and missed two more instances of its
own defect class.
`_types.py`'s `PARTICLE_OR_GIVEN` documents a KIND, and this kind has
two emitters: `_assign`'s lone leading particle and `_group`'s
prefix chain, when a title shifts the particle off index 0. All three
of the rewrite's claims were false for the second. Measured:
Dr. Van Johnson default family='Van Johnson'
Dr. Van Johnson FAMILY_FIRST family='Van Johnson'
Dr. Van Johnson FF_GIVEN_LAST family='Van Johnson'
detail (all three, identical):
'Van' was chained onto the following name piece; it is also a
given name in other names
It WAS chained; it lands in `family` under every order, not just the
default; and its detail names no field, so "detail names it rather
than the kind" does not describe it. Worse, the rewrite added an
explicit denial of chaining ("rather than as a particle chaining onto
what follows") to a kind whose other emitter fires only when chaining
happened. The docstring now covers both shapes and says what
distinguishes them -- one was left standing alone and carries the role
assignment gave it, the other was claimed by the chain and names no
field because grouping runs before roles exist.
`_lexicon.py`'s `particles_ambiguous` said a member "stays a name
piece of its own instead of chaining onto what follows", which implies
a non-member chains. None does. `_group.py:204` is `if k == 0 or not
prefix(k): continue`, so the prefix chain skips index 0 before
membership is ever consulted, and both group into two pieces:
de Mesnil -> [['de'], ['Mesnil']] (never-given)
van Gogh -> [['van'], ['Gogh']] (ambiguous)
What produces family='de Mesnil' under the default is `post_rules`
rule 1b, a role fold AFTER assignment -- the rule #359 is about. That
claim also contradicted `config/particles.py:85-87`, the site the
issue named as the pattern to follow, leaving two rendered API
docstrings disagreeing; and it is the same claim `c647381` says it
measured false and deleted from `customize.rst`. Removed there,
planted here. It now names what membership actually decides: the
ambiguity report under either order, and the default order's fold for
a non-member (with the degenerate bare "de" that rule 1b's
middle/family guard leaves alone).
`docs/usage.rst`'s "where the pieces land follows that order instead"
covered both branches of the preceding sentence, including the
never-given one, so it asserted exactly what the PR said it did not.
It is wrong in substance too -- "de Mesnil" under FAMILY_FIRST is not
those pieces relocated, it is a fold that never fires. It now says
only that the destinations shown are the default order's and that
`name_order` is what decides them.
`docs/concepts.rst`: an inserted em-dash left "which is the right call
for the actor Van Johnson" binding to "the start of a surname" rather
than to the reading actually taken -- i.e. backwards. The chosen
reading and the right-call/wrong-one pair are adjacent again, and the
order-dependence moved to its own parenthesis instead of splitting
them.
Two sites the sweep missed, both its own defect class:
_group.py:213 "Van Johnson" -> given nine lines above the
emitter it examined
AGENTS.md:179 "the particle stayed the GIVEN name"
Dr. Van Jr. default given='Van'
Dr. Van Jr. FAMILY_FIRST family='Van'
#354 fixed the sibling at AGENTS.md:136 and this one was missed.
`docs/release_log.rst` said `SUFFIX_OR_NAME` "has always named the
part it declined". It names both -- `read as a family name rather
than a post-nominal` -- and the new particle detail names only the
part it took. The bullet now says which is which.
Not touched, and not this change's business: `customize.rst`'s "no
given name at all" after a family comma and `usage.rst:746-749`'s
"more likely reading" are pre-existing; the 2.2.0 preamble is
finalized at release time; and what FAMILY_FIRST should mean for a
never-given Latin particle is #359's open question, so nothing here
asserts an answer to it.
pytest 3117 passed, 20 skipped, 11 xfailed
mypy Success: no issues found in 103 source files
ruff All checks passed!
sphinx -b html build succeeded, 0 warnings
sphinx -b doctest 223 tests, 0 failures
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The PARTICLES docstring said a leading particle "chains nothing, since it may be a given name instead". The behavior is right and the reason is not: the chain loop skips index 0 unconditionally (_group.py, `if k == 0 or not prefix(k)`), so a never-given particle chains nothing either. Measured -- "de Mesnil" and "van Gogh" both group into two separate pieces, though only one of them could be a given name. The `since` clause offered a contingent rationale for unconditional behavior, which is the shape that gets read as a rule and then relied on. This is the docstring #355 named as the pattern for its sweep, and the sweep's own rewrites now say membership decides nothing about chaining. Leaving the model site giving a membership-flavored reason for it would have left the two disagreeing about why, one commit after they were made to agree about what. Narrower than it first looks, and the narrowing matters: the claim is about the PREFIX chain, not about grouping as a whole. The bound given-name rule does chain a leading piece, and three entries are in both vocabularies -- `parse("Abu Bakr Ahmed")` groups as [Abu Bakr] [Ahmed], a leading ambiguous particle that was chained. It keys on `vocab:bound-given` and never on this set, so the statement holds as written. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #355.
Policy(name_order=FAMILY_FIRST)shipped in 2.1. Six places described leading-particle behaviour by naming the field the piece lands in, which is only true under the defaultGIVEN_FIRST— and one of them was not documentation at all.The runtime string (
342cac2)ParsedName.ambiguitiesreported the wrong answer under a non-default order:The emitter hardcoded "given name". Twelve lines above it, the
SUFFIX_OR_NAMEblock already computes the role fromtoken.role.value, and_types.pystates the principle — "Which name part was declined depends on position andname_order, sodetailnames it rather than the kind." The particle emitter just didn't follow its own file's precedent. It now readsread as a family nameunderFAMILY_FIRST.AmbiguityKind.PARTICLE_OR_GIVENis unchanged — the fork really is particle-or-given, and the member is public API. Only the human-readable detail moved.The docs (
c647381)Six sites scoped to the default order:
_lexicon.py'sparticles_ambiguous,_types.py'sPARTICLE_OR_GIVEN,docs/usage.rst,docs/concepts.rst, and two paragraphs indocs/customize.rst. The mechanism claims are preserved throughout — a leading particle chains nothing, and whether it can double as a given name decides which branch is taken; only the destinations are scoped.No doctests were added or changed. The existing ones use the default
parse()and remain true, and perAGENTS.mdedge-case behaviour gets a prose sentence rather than a doctest block.The sixth site was found during the sweep rather than listed in the issue, and its first drafted wording ("leading
vanthen chains onto what follows") turned out to be false underFAMILY_FIRST— measured, it does not chain, it stays its own piece. The shipped wording leads with the one effect that holds under every order: the ambiguity stops being recorded.Deliberately not fixed
post_rulesrule 1b keys on roles rather than position, so the never-given fold never fires underFAMILY_FIRST. Filed separately — it is a decision about what family-first ordering should mean for a Latin particle, and documenting the current output here would have blessed behaviour that is probably a bug. Nothing in this PR asserts what a non-default order does to a never-given particle.Review round
A three-agent review (comments, tests, general code) found that two of the six rewrites had replaced the field-destination error with a mechanism error, and that the new test pinned the string rather than the invariant. Three follow-up commits:
policy.name_order[0].valueinstead oftoken.role.valuepasses all 3110 tests and 223 doctests — and is wrong on thescript_orderspath, which derives the effective order without touchingname_order. Measured: a Han ambiguous particle assignsfamilywhilepolicy.name_order[0]is stillgiven. The new test fails under exactly that mutation, and the code comment that namedname_ordernow names the effective order._types.pydocumented one of the kind's two emitters.PARTICLE_OR_GIVENis also raised from_groupwhen a title shifts the particle off the front, where it was chained, lands infamilyunder every order, anddetailnames no field. The rewrite had added an explicit denial of chaining to a kind whose other emitter fires only when chaining happened._lexicon.pysaid membership decides chaining. It decides nothing about chaining: the prefix chain skips index 0 unconditionally. That contradictedconfig/particles.py, the site this sweep was told to follow — and was the same claim commitc647381had measured false and removed fromcustomize.rst.Also:
usage.rstno longer asserts what a non-default order does to a never-given particle (the thing this PR claims to avoid and #359 relies on), a dangling relative clause inconcepts.rstthat inverted the sentence is fixed, two missed sites (_group.py's comment,AGENTS.md:179) are scoped, andconfig/particles.py's own "since it may be a given name instead" is corrected — the behaviour is unconditional, so the reason was contingent for something that isn't.One recommendation was declined: a
strict=Truexfail assertingde Mesnil→family='de Mesnil'underFAMILY_FIRST. #359 deliberately leaves open what family-first should mean for a Latin particle, and an xfail asserting one answer decides it by the back door.Verification
🤖 Generated with Claude Code