Update to 2.1.1 by ppgranger · Pull Request #29 · ppgranger/token-saver

ppgranger · 2026-03-17T19:33:04Z

What

Why

How

Checklist

ruff check . passes
ruff format --check . passes
python3 -m pytest tests/ -v passes (all 207+ tests)
New code has tests (if applicable)
No secrets or credentials in the diff

…ures Add a visual hero section to the README with CI, coverage, Python, license, and savings badges. Include a before/after compression table with real numbers derived from audit_compression.py. Add generate_demo.py script and example fixtures for git diff, pytest, terraform, npm install, and kubectl.

Create docs/comparison.md with a factual comparison of Token-Saver against cc_token_saver_mcp, token-optimizer-mcp, and Claude Context Mode. Add a "How It Compares" section in README linking to it.

Add support for user-defined processors loaded from ~/.token-saver/processors/. User processors inherit from Processor base class and are auto-discovered alongside built-in ones. Broken processors are skipped gracefully. Add user_processors_dir config key, example ansible processor, and 7 tests.

Add push trigger on main, Python 3.11 to matrix, pytest-cov for coverage reporting, codecov upload on 3.12, mypy type checking step. Add dev optional dependencies group to pyproject.toml.

Add py.typed marker for PEP 561 compliance. Create mypy.ini with permissive settings. Add type annotations to engine.py and config.py public method signatures. mypy step is included in CI workflow.

Add 'token-saver benchmark <command>' to measure compression on real commands. Supports --format json for scripting and --dry-run to show processor match without executing. Add 4 tests for the new subcommand.

Support project-level config files that override global settings. Token-Saver walks up parent directories to find .token-saver.json (like .gitignore). Track config sources for debug mode. Add 5 tests for project config.

Create demo.py that processes all fixtures through the compression engine and displays before/after stats. Add examples/README.md documenting the fixtures and how to run the demo.

Fix ruff warnings: unused imports, noqa directives, dict.fromkeys, TYPE_CHECKING import, and apply ruff formatter to all new files.

Feat/open source quality upgrade

Replace three near-identical JSON summarization implementations (network._summarize_json, file_content._summarize_json_value, cloud_cli._compress_json_value) with calls to the shared utils.compress_json_value() function.

Unify the head+tail+error-extraction-with-context pattern from file_content._compress_log, docker._process_logs, and kubectl._process_logs into a single compress_log_lines() utility.

Pre-scan git diff output for lockfile paths (package-lock.json, yarn.lock, etc.) and replace their entire diff section with a one-line summary, preventing hundreds of useless hunks from being sent through compress_diff.

When git diff --stat has more than 20 files, group them by directory with aggregate change counts instead of listing each file individually.

Detect coverage tables in pytest output and compress them to show only the TOTAL line and files below 80% coverage.

Track parameterized test names during pytest output processing and report them grouped with failed parameter names when failures occur.

Add minified file detection before source code pass-through check. Files like .min.js, .bundle.js and files with very long average line length are summarized instead of passed through unchanged.

Add .env variant detection that redacts sensitive values in deployed env files while preserving .env and .env.example pass-through behavior.

Route tsc --noEmit through a dedicated type-check handler that groups errors by TS error code with occurrence counts and examples.

Detect compose log format (service | message) and group output by service, showing error lines with context and per-service summaries.

When search results span more than 30 files, group them by directory with top files per directory and match counts.

Short API responses (500-1500 chars) are often more useful uncompressed for the model to read.

Three new specialized processors for DevOps/infrastructure output: - AnsibleProcessor (priority 40): compresses ok/skipped tasks, preserves errors and PLAY RECAP - HelmProcessor (priority 41): summarizes template manifests, omits NOTES, truncates lists - SyslogProcessor (priority 42): head/tail compression for journalctl and dmesg output Updates test suite with 43 new tests (565 total).

Split rsync from the ssh/scp exclusion — local rsync (no host:path) is now compressible via the file_listing processor, while remote rsync (with host:path like user@server:/path/) remains excluded.

Update test_expected_priority_order with ansible/helm/syslog entries. Add ansible, helm, journalctl, dmesg commands to hook pattern coverage test.

Update processor count from 18 to 21 throughout README. Add ansible, helm, syslog to processor table and project structure. Add rsync to file_listing commands. Create per-processor docs (ansible.md, helm.md, syslog.md). Update test counts (567 tests). Fix ruff lint issues (SIM103, SIM102, PIE810, RUF005). Add PLR0913 to global ignores for shared utility signatures.

Bump version across all manifests (src/__init__.py, pyproject.toml, plugin.json, marketplace.json). README improvements: - Stronger value proposition in header - Added docker compose and helm template to Before & After table - Precision guarantees: source code passthrough, .env redaction - Updated processor descriptions in plugin manifests Fix helm NOTES omission bug: NOTES section is now fully stripped instead of leaking non-indented content lines.

Feat/processor improvements

ppgranger added 30 commits March 17, 2026 08:00

feat: add comparison table vs alternatives

043e509

Create docs/comparison.md with a factual comparison of Token-Saver against cc_token_saver_mcp, token-optimizer-mcp, and Claude Context Mode. Add a "How It Compares" section in README linking to it.

feat: enhance CI with coverage, Python 3.11, mypy, and dev deps

da33c48

Add push trigger on main, Python 3.11 to matrix, pytest-cov for coverage reporting, codecov upload on 3.12, mypy type checking step. Add dev optional dependencies group to pyproject.toml.

feat: add type hints, mypy configuration, and PEP 561 marker

eb70a48

Add py.typed marker for PEP 561 compliance. Create mypy.ini with permissive settings. Add type annotations to engine.py and config.py public method signatures. mypy step is included in CI workflow.

feat: add benchmark CLI subcommand

273e4b8

Add 'token-saver benchmark <command>' to measure compression on real commands. Supports --format json for scripting and --dry-run to show processor match without executing. Add 4 tests for the new subcommand.

feat: add per-project configuration via .token-saver.json

67d6ca0

Support project-level config files that override global settings. Token-Saver walks up parent directories to find .token-saver.json (like .gitignore). Track config sources for debug mode. Add 5 tests for project config.

feat: add examples directory with demo script and documentation

0770d9a

Create demo.py that processes all fixtures through the compression engine and displays before/after stats. Add examples/README.md documenting the fixtures and how to run the demo.

style: apply ruff lint and format fixes across new files

162fea0

Fix ruff warnings: unused imports, noqa directives, dict.fromkeys, TYPE_CHECKING import, and apply ruff formatter to all new files.

fix: add version field to pyproject.toml for editable install

d1e3547

fix: resolve all mypy type errors across 7 files

f00a993

fix: resolve remaining mypy errors in tracker.py

b85e221

Merge pull request #27 from ppgranger/feat/open-source-quality-upgrade

fe4ccdc

Feat/open source quality upgrade

refactor: extract shared log compression into utils.compress_log_lines()

c623a54

Unify the head+tail+error-extraction-with-context pattern from file_content._compress_log, docker._process_logs, and kubectl._process_logs into a single compress_log_lines() utility.

feat: detect and summarize lockfile diffs in git processor

b4ef611

Pre-scan git diff output for lockfile paths (package-lock.json, yarn.lock, etc.) and replace their entire diff section with a one-line summary, preventing hundreds of useless hunks from being sent through compress_diff.

feat: group git diff --stat output by directory for large refactors

52842f0

When git diff --stat has more than 20 files, group them by directory with aggregate change counts instead of listing each file individually.

feat: compress pytest coverage reports in test output processor

84b963e

Detect coverage tables in pytest output and compress them to show only the TOTAL line and files below 80% coverage.

feat: group parameterized pytest results with failure detail

e814619

Track parameterized test names during pytest output processing and report them grouped with failed parameter names when failures occur.

feat: detect and compress minified files in file_content processor

e7c3d14

Add minified file detection before source code pass-through check. Files like .min.js, .bundle.js and files with very long average line length are summarized instead of passed through unchanged.

feat: redact secrets in .env variant files (.env.production, .env.local)

8e76e9f

Add .env variant detection that redacts sensitive values in deployed env files while preserving .env and .env.example pass-through behavior.

feat: group tsc --noEmit errors by TypeScript error code

3053150

Route tsc --noEmit through a dedicated type-check handler that groups errors by TS error code with occurrence counts and examples.

feat: group docker compose logs by service with error extraction

dc7547c

Detect compose log format (service | message) and group output by service, showing error lines with context and per-service summaries.

feat: group search results by directory for large result sets

d78cad8

When search results span more than 30 files, group them by directory with top files per directory and match counts.

fix: raise JSON compression threshold from 500 to 1500 chars

c6a1c7b

Short API responses (500-1500 chars) are often more useful uncompressed for the model to read.

feat: allow local rsync through hook, exclude only remote rsync

1b0e108

Split rsync from the ssh/scp exclusion — local rsync (no host:path) is now compressible via the file_listing processor, while remote rsync (with host:path like user@server:/path/) remains excluded.

test: add priority assertions and hook patterns for new processors

3491f5a

Update test_expected_priority_order with ansible/helm/syslog entries. Add ansible, helm, journalctl, dmesg commands to hook pattern coverage test.

Merge pull request #28 from ppgranger/feat/processor-improvements

61c660f

Feat/processor improvements

ppgranger merged commit f47a347 into production Mar 17, 2026
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update to 2.1.1#29

Update to 2.1.1#29
ppgranger merged 31 commits intoproductionfrom
main

ppgranger commented Mar 17, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

ppgranger commented Mar 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

How

Checklist

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

ppgranger commented Mar 17, 2026 •

edited

Loading