Skip to content

fix(da): restore lowercase dotted abbreviation boundaries - #95

Merged
yisding merged 1 commit into
mainfrom
codex/fix-danish-initialism-sentence-boundaries
Jul 11, 2026
Merged

fix(da): restore lowercase dotted abbreviation boundaries#95
yisding merged 1 commit into
mainfrom
codex/fix-danish-initialism-sentence-boundaries

Conversation

@yisding

@yisding yisding commented Jun 29, 2026

Copy link
Copy Markdown
Owner

Motivation

  • A recent split-mode change only treated two-letter initialisms as split candidates when both letters were uppercase and removed the Danish-specific boundary-abbreviation override, causing lowercase/mixed-case Danish dotted abbreviations like s.u./s.U. to be under-segmented before capitalized followers.
  • The change is a correctness regression for Danish sentence-splitting and should be fixed narrowly without changing global two-letter initialism heuristics.

Description

  • Add a per-replacer opt-in SENTENCE_BOUNDARY_ABBREVIATIONS attribute to AbbreviationReplacer so languages can list multi-period abbreviations that should be treated as sentence-boundary candidates.
  • In replace_multi_period_abbreviations() compute abbreviation_key = matched[:-1].lower() and use listed_sentence_boundary to influence the final-period restoration decision (treat listed entries as split candidates before likely sentence starts).
  • Configure Danish AbbreviationReplacer to include SENTENCE_BOUNDARY_ABBREVIATIONS = frozenset({"s.u"}) so s.u. and s.U. split correctly before a capitalized follower.
  • Add golden-rule tests for the Danish cases and update the regression snapshot to reflect the restored behavior.

Testing

  • Ran targeted tests with uv run pytest tests/regression/test_segment_snapshot.py tests/unit/test_abbreviation_replacer.py tests/regression/test_greek_abbreviations.py tests/lang/test_danish.py and all collected tests passed.
  • Updated the regression baseline with uv run python -m tests.regression.segment_snapshot --update and verified the snapshot-based regression test tests/regression/test_segment_snapshot.py passed after the update.
  • Ran the full suite with coverage via uv run pytest --cov=sentencesplit tests/ --color yes and observed the test suite completed with the expected pass/skip/xfail summary and coverage threshold met.

Codex Task

@charliecreates charliecreates Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the current diff and test updates, and I don’t have actionable code feedback to address. CI checks for this head commit are still running, so merge readiness still depends on those results.

@yisding
yisding merged commit 4b77ccd into main Jul 11, 2026
9 checks passed
@yisding
yisding deleted the codex/fix-danish-initialism-sentence-boundaries branch July 11, 2026 06:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant