Skip to content

refactor(text): char_boundary_*/ellipsize helpers + pin critic transcript (R D) - #357

Merged
yogthos merged 1 commit into
mainfrom
refactor/text-helpers-d
Jun 2, 2026
Merged

refactor(text): char_boundary_*/ellipsize helpers + pin critic transcript (R D)#357
yogthos merged 1 commit into
mainfrom
refactor/text-helpers-d

Conversation

@yogthos

@yogthos yogthos commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

Round D of the duplication consolidation (epic dirge-9l12, dirge-ep1f), scoped honestly after a rigorous feasibility pass. The headline finding: most of Round D's proposed merges are intentional per-consumer variation over load-bearing, thinly-tested formats — consolidating them would risk silently changing model/critic-facing output. So this PR does the genuinely-safe subset and declines the rest with reasoning.

Done (pure, zero-output-change)

  • fix: token counter uses full_input after multi-line rename #8 boundary helpers — promoted char_boundary_at_or_before/after to crate::text; compression.rs drops its private copies and summarize.rs::head_tail_truncate replaces its two inline boundary-walk loops. No marker/ratio/format touched.
  • Live token counter in input bar #6 byte-budget clustertext::ellipsize(s, max_bytes) absorbs skills::truncate + memory_store::truncate_for_error (display/error text, not model-facing).

Declined (with evidence)

  • The 3 head/tail truncators — different ratios (40/60 vs ~90/10 vs 70/30) + model-facing markers ("call the tool with a narrower scope…"). Merging needs 4+ knobs = over-engineering, and the markers have only .contains()/zero test coverage.
  • The other 6 ellipsis sites — char-unit vs byte-unit, whitespace-collapse, and per-site markers (... vs vs …[N more]) are genuinely different.
  • Line-numbered read output + environment preamble #7 transcript builders — 3 different input types (SessionMessage/LoopMessage/serde_json::Value), 3 intentional formats; a trait+style abstraction adds more complexity than it removes.

Plus the highest-value action the audit surfaced

A byte-exact pinning test for build_critic_transcript — the load-bearing critic format (which just had a stale-summary bug fixed) was only .contains()-tested. Now a refactor can't silently shift its USER:/ASSISTANT: labels or tool-call rendering. + ellipsize/char_boundary unit tests.

2457 default / 2551 all-features tests pass. This completes the consolidation epic.

… critic transcript [dirge-ep1f]

Round D, scoped HONESTLY after a rigorous feasibility pass: most of the
proposed text-helper merges are intentional per-consumer variation with
load-bearing, thinly-tested formats — consolidating them would risk silently
changing model/critic-facing output. So:

#8 (boundary helpers, the one pure win): promote char_boundary_at_or_before/
after to crate::text; compression.rs drops its private copies, summarize.rs's
head_tail_truncate replaces its two inline boundary loops. Zero output change,
no marker/ratio touched. The 3 head/tail TRUNCATORS themselves are LEFT —
different ratios + model-facing markers, intentionally per-consumer.

#6 (byte-budget … cluster): text::ellipsize(s, max_bytes) absorbs
skills::truncate + memory_store::truncate_for_error (display/error text, not
model-facing). The other 6 ellipsis sites are LEFT — their char-unit /
whitespace-collapse / marker variations are genuinely per-site.

#7 (transcript builders): NOT consolidated — 3 different input types
(SessionMessage/LoopMessage/serde_json::Value), 3 intentional formats, and a
trait+style abstraction would add more complexity than it removes.

PLUS the highest-value action the audit surfaced: a byte-EXACT pinning test
for build_critic_transcript (load-bearing critic format, previously only
.contains()-tested) so a future refactor can't silently shift its labels.
+ ellipsize/char_boundary unit tests. 2457 default / 2551 all-features pass.
@yogthos
yogthos merged commit 5238f60 into main Jun 2, 2026
10 checks passed
@yogthos
yogthos deleted the refactor/text-helpers-d branch June 2, 2026 22:27
allen-munsch pushed a commit to allen-munsch/dirge that referenced this pull request Jun 3, 2026
… critic transcript [dirge-ep1f] (dirge-code#357)

Round D, scoped HONESTLY after a rigorous feasibility pass: most of the
proposed text-helper merges are intentional per-consumer variation with
load-bearing, thinly-tested formats — consolidating them would risk silently
changing model/critic-facing output. So:

dirge-code#8 (boundary helpers, the one pure win): promote char_boundary_at_or_before/
after to crate::text; compression.rs drops its private copies, summarize.rs's
head_tail_truncate replaces its two inline boundary loops. Zero output change,
no marker/ratio touched. The 3 head/tail TRUNCATORS themselves are LEFT —
different ratios + model-facing markers, intentionally per-consumer.

dirge-code#6 (byte-budget … cluster): text::ellipsize(s, max_bytes) absorbs
skills::truncate + memory_store::truncate_for_error (display/error text, not
model-facing). The other 6 ellipsis sites are LEFT — their char-unit /
whitespace-collapse / marker variations are genuinely per-site.

dirge-code#7 (transcript builders): NOT consolidated — 3 different input types
(SessionMessage/LoopMessage/serde_json::Value), 3 intentional formats, and a
trait+style abstraction would add more complexity than it removes.

PLUS the highest-value action the audit surfaced: a byte-EXACT pinning test
for build_critic_transcript (load-bearing critic format, previously only
.contains()-tested) so a future refactor can't silently shift its labels.
+ ellipsize/char_boundary unit tests. 2457 default / 2551 all-features pass.

Co-authored-by: Yogthos <yogthos@gmail.com>
allen-munsch pushed a commit to allen-munsch/dirge that referenced this pull request Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant