Sift 0.4.0

Sift 0.4.0 is the first release under the Sift name. The repository, executable, configuration, installers, integrations, formula, and release assets now use sift.

Sift reduces the text sent to coding agents through reversible encodings and conservative command-aware views. Enable original-output retention when full native bytes may need recalling later. This release adds faster embedded token counting, more lossless text representations, command-aware Git and Cargo views, explicit read/JSON/summary/error/test views, and broader automatic integration and migration support.

The final Linux x64 artifact is faster than RTK 0.49 on every row in the release's ten-workload synthetic benchmark. On the seven substantive workloads, median Sift time ranges from 2.3–12.8 ms, compared with RTK's 3.5–16.2 ms. Sift used 1,518 tokens versus 2,662 raw tokens across those rows, a 43% reduction. RTK used fewer tokens on several narrow command-specific rows, so the benchmark reports both measurements rather than claiming a universal token win.

An optional recoverable semantic selector can further reduce large Pi built-in grep results. It is off by default and requires an explicit project allowlist plus TYPESAFE_API_KEY. Eligible passages are selected with TypeSafe Jev; the complete original remains local and can be restored with sift recall. Timeouts, malformed responses, missing configuration, or insufficient savings keep the deterministic local output. Ordinary Sift compaction makes no model calls.

Install on Linux or macOS:

curl -fsSL https://raw.githubusercontent.com/ctxrs/sift/main/install.sh | sh

Install on Windows PowerShell:

irm https://raw.githubusercontent.com/ctxrs/sift/main/install.ps1 | iex

To replace recognized RTK integrations during installation, add --replace-rtk on Unix or -ReplaceRtk on Windows. Existing installations can run sift init --replace-rtk.

Release assets include standalone Linux x64/arm64, macOS x64/arm64, and Windows x64 binaries, per-binary CycloneDX SBOMs and third-party notices, and SHA256SUMS. The release factory notarized and Developer ID signed the macOS binaries and Authenticode signed the Windows binary. Native macOS and Windows execution/trust checks were skipped for this release at the maintainer's direction; those checks are not claimed as passing.

Retok 0.3.0

Retok 0.3.0 adds automatic integration, safer RTK migration, and practical command workflows.

• Install supported integrations with retok init --replace-rtk; custom or unsupported RTK hooks remain preserved with an explanation.
• Use retok run --capture -- COMMAND for finite output that should remain eligible for compaction beyond the normal progress window.
• Select output explicitly with read, json, summary, err, and test; default compaction remains reversible.
• Filter local gain reports, inspect retained output, discover eligible historical commands without executing them, and import an existing usage export without fetching another package.
• Faster tokenizer startup with identical ordinary o200k counts and selection behavior; the precomputed vocabulary makes binaries larger.

RTK remains faster on the small synthetic command fixtures. Token reduction, retained information, execution behavior, and elapsed time are reported separately in the benchmark charts.

Integrations have host-specific limits. Gemini/VS Code rewrites are withheld after approval-regression checks; Cursor/Droid retain RTK pending qualification. OpenClaw may ask again for a previously approved exact command. PowerShell automatic rewriting is unavailable; Windows completion hooks and explicit commands remain supported.

macOS binaries use Developer ID signing and accepted notarization; Windows uses timestamped Authenticode signing. Each binary ships with an SBOM, license notices and checksums. See INTEGRATIONS.md and COMPATIBILITY.md for supported routes and deliberate differences.

Qualification includes native Linux x64, Windows x64 and macOS x64 execution checks. Linux ARM64 core behavior was exercised under QEMU; macOS ARM64 received native signature verification on an x64 Mac, without ARM64 runtime execution. Host-integration checks have the version and scope limits documented in INTEGRATIONS.md.

Retok v0.2.0

Retok 0.2.0 adds command execution and agent setup.

• Run arbitrary executables with retok COMMAND ...; arguments, stdin, output streams and exit status are preserved. Interactive and ongoing output passes through.
• Install native output adapters for Claude Code, Copilot CLI, Pi / Oh My Pi, OpenCode and current Kilo. Other supported hosts receive instructions; see the integration matrix.
• Switch recognized stock RTK setups with retok init --replace-rtk, with backups and custom settings preserved.
• Inspect measured local savings with gain, optionally retain originals for recall, and replay saved output with discover.

Install and switch on Linux/macOS:

curl -fsSL https://raw.githubusercontent.com/ctxrs/retok/main/install.sh | sh -s -- --replace-rtk

Windows PowerShell:

& ([scriptblock]::Create((irm https://raw.githubusercontent.com/ctxrs/retok/main/install.ps1))) -ReplaceRtk

The synthetic benchmark reports tokens, retained information and timing together. Retok was slower on these small workloads; this release makes no speed or model-billing claim. RTK workflow differences are documented.

Five binaries are supplied below: static Linux x64/ARM64, notarized Developer ID macOS x64/ARM64, and timestamped Authenticode Windows x64. Each binary has a CycloneDX SBOM and third-party notices; SHA256SUMS covers all fifteen assets.

Retok v0.1.1

Retok v0.1.1 signs the native desktop artifacts while keeping the v0.1 compaction behavior unchanged.

• macOS x64 and arm64 binaries are signed with the ctx Apple Developer ID and accepted by Apple notarization.
• Windows x64 is Authenticode-signed with the ctx Azure Artifact Signing identity and an RFC 3161 timestamp.
• Release staging now refuses macOS or Windows artifacts until the exact signed bytes pass native signature, identity, timestamp, and version checks.
• Linux binaries remain byte-identical after the signing stage.

Install

Linux and macOS:

curl -fsSL https://raw.githubusercontent.com/ctxrs/retok/main/install.sh | sh

Windows PowerShell:

irm https://raw.githubusercontent.com/ctxrs/retok/main/install.ps1 | iex

The installers select the host architecture, verify the binary and license notices against SHA256SUMS, and install into a user directory without editing PATH or shell profiles.

Assets

• Linux x64 and arm64 static musl executables
• Apple-notarized macOS 13+ x64 and arm64 executables
• Authenticode-signed Windows x64 executable
• CycloneDX SBOM and complete third-party/runtime notices for every executable
• SHA256SUMS covering the five binaries and ten sidecars

Source: 434b4f3be6882f8fa301d6bd5e87a5e6e32e6347

Retok v0.1.0

Retok 0.1.0 is the first public release of a local, lossless tool-output compactor for coding agents.

It chooses among reversible JSON and text representations using exact o200k_base token counts, and retains the original whenever no candidate is smaller. This release includes the persistent JSONL protocol, explicit restoration, bounded decoders, invalid-UTF-8 passthrough, and no network calls or telemetry.

Install

Linux and macOS:

curl -fsSL https://raw.githubusercontent.com/ctxrs/retok/main/install.sh | sh

Windows PowerShell:

irm https://raw.githubusercontent.com/ctxrs/retok/main/install.ps1 | iex

The installers verify the binary and license notices against SHA256SUMS. They install into a user directory and do not edit PATH, shell profiles, or agent settings.

Assets

• Linux x64 and arm64: static musl executables
• macOS 13 or newer: x64 and arm64
• Windows: x64
• CycloneDX SBOM and complete third-party/runtime notices beside every executable
• SHA256SUMS for the complete 15-file asset inventory

The macOS and Windows artifacts are not Developer ID/AuthentiCode signed, and the macOS artifacts are not notarized. Gatekeeper or SmartScreen may warn. Linux x64 and arm64 passed full functional artifact smoke tests. The macOS and Windows artifacts were cross-built and passed executable-structure, dependency, SBOM, and checksum validation; managed native smoke runners were unavailable because their shared disk-health admission gate remained closed.

Source: 32c9cc634c85e7b28e8c3e1e1f49739d548239c1