Skip to content

docs: remove remaining caveman references from README #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Patel230 merged 1 commit into from
Jun 2, 2026
Merged

docs: remove remaining caveman references from README #44

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,9 +130,9 @@ Reproduce: `go build -o tok ./cmd/tok && TOK=./tok evals/bench.sh --no-rtk`

## Recent additions

Session 2026-06-01 closed the second round of gaps vs rtk 0.37.1 and caveman:
Session 2026-06-01 closed the second round of gaps vs rtk 0.37.1 and prompt-compression:

- **`tok.CompressCaveman(text, intensity)`** — caveman-style prompt compression (Lite / Full / Ultra) as a public Go API. ~150 phrase substitutions, drop-lists for articles / filler / pleasantries, and auto-clarity (security/destructive segments pass through verbatim). `intensity` is monotonic: `len(ultra) <= len(full) <= len(lite)`.
- **`tok.PromptCompress(text, intensity)`** — prompt compression (Lite / Full / Ultra) as a public Go API. ~150 phrase substitutions, drop-lists for articles / filler / pleasantries, and auto-clarity (security/destructive segments pass through verbatim). `intensity` is monotonic: `len(ultra) <= len(full) <= len(lite)`.
- **`tok.IsSensitiveFilename(path)`** — 3-layer filename detection (exact basename, sensitive directory, name token). Companion to the content-based `SecretDetector`. Catches `.env`, `id_rsa`, `~/.ssh/...`, `test_credentials.json`, etc. Returns the category that fired.
- **`tok.SmartTruncate(content, maxLines, lang)`** — code truncation that preserves function signatures and **always reports the exact drop count** in the marker (`kept + dropped == total`, rtk invariant).
- **`tok.ExtractJSON(text)` / `tok.ExtractJSONArray(text)` / `tok.ExtractAllJSON(text)`** — brace-balanced JSON extraction that handles LLM output with surrounding prose, markdown code fences, and unterminated objects. Apostrophes in English prose are not confused with string delimiters.
Expand All @@ -143,7 +143,7 @@ Session 2026-06-01 closed the second round of gaps vs rtk 0.37.1 and caveman:

### Earlier (2026-04-20)

Session 2026-04-20 closed the last gaps vs rtk 0.37.1 and caveman:
Session 2026-04-20 closed the last gaps vs rtk 0.37.1 and prompt-compression:

- **`tok commit-msg`** — read staged diff, emit Conventional Commits subject. Rule-based, no LLM.
- **`tok review-diff`** — scan diff, emit one-line review comments (`🔴 bug / 🟡 risk / 🔵 nit`). Rule-based, no LLM.
Expand Down
Loading