Skip to content

feat(fs): auto-compress cat'd markdown via compress_md pipeline#88

Merged
claudioemmanuel merged 1 commit intomainfrom
feat/viewer-markdown-auto-compress
Apr 22, 2026
Merged

feat(fs): auto-compress cat'd markdown via compress_md pipeline#88
claudioemmanuel merged 1 commit intomainfrom
feat/viewer-markdown-auto-compress

Conversation

@claudioemmanuel
Copy link
Copy Markdown
Owner

Summary

  • Added extract_md_viewer_target() helper in src/commands/fs.rs that detects viewer commands (cat/head/tail/less/more/bat) targeting .md files
  • New md-mode branch in FsHandler::compress routes .md files through compress_md::compress_text_with_locale when auto_compress_md=true (the default), bypassing smart_filter/dedup/truncation entirely
  • Emits [squeez: md-mode — compressed N→M tokens from path] header; falls back to raw text if integrity check fails
  • Adds 3 tests to tests/test_fs.rs covering the happy path, disabled flag, and non-.md files
  • Adds bench/fixtures/cat_claude_md.txt (486 lines) picked up by bench/run.sh

Motivation

Large plan files like plan.md at 1,754 lines were being read repeatedly with no compression — the existing FsHandler only head-truncated to 50 lines. The markdown compressor was already proven on memory files but unused for ad-hoc cat invocations. This change closes that gap with zero new dependencies.

Benchmark impact

FIXTURE                               BEFORE    AFTER  REDUCTION  LATENCY STATUS
cat_claude_md.txt                     18406tk     437tk        98%      12ms  ✅

98% token reduction on a 486-line representative CLAUDE.md fixture.

Test plan

  • cargo test --test test_fs — 5 tests pass (2 existing + 3 new)
  • cargo test — full suite green
  • bash bench/run.sh — new fixture shows 98% reduction
  • cargo build — zero warnings

When auto_compress_md=true (default), viewer commands (cat/head/tail/less/more/bat)
targeting .md files now route through the markdown compressor instead of the generic
smart_filter+truncation path. Emits a [squeez: md-mode — compressed N→M tokens from path]
marker header. Falls back to the original output if the integrity check fails.

Adds bench/fixtures/cat_claude_md.txt (486 lines, 98% token reduction in bench/run.sh).
@claudioemmanuel claudioemmanuel force-pushed the feat/viewer-markdown-auto-compress branch from aebc461 to 380d16a Compare April 22, 2026 00:47
@claudioemmanuel claudioemmanuel merged commit ee418bc into main Apr 22, 2026
3 of 5 checks passed
@claudioemmanuel claudioemmanuel deleted the feat/viewer-markdown-auto-compress branch April 22, 2026 00:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant