Add RTK Directly Into Codex CLI to Reduce Token Usage 60–90% by Filtering Shell Command Output

[klondikemarlen]

What version of Codex CLI is running?

codex-cli 0.122.0

What subscription do you have?

Plus

Which model were you using?

gpt-4.5 medium

What platform is your computer?

Linux 6.17.0-22-generic x86_64 unknown

What terminal emulator and version are you using (if applicable)?

Terminal → https://gitlab.gnome.org/chergert/ptyxis

What issue are you seeing?

Token usage grows rapidly during normal coding sessions because raw shell command output is passed directly into the LLM context. Commands like git status, git diff, ls, cargo test, and docker ps produce verbose, repetitive output — often 60–90% noise from the model's perspective — yet every token counts toward the context limit and API cost. Context fills fast, triggering expensive compaction or hitting limits mid-task.

What steps can reproduce the bug?

1. Start a Codex CLI session on a mid-sized project.
2. Run a sequence of typical coding commands: git status, git diff, ls, a test run.
3. Observe token count (uploaded thread: 019db5ee-d016-74d0-9228-76e5b75fbcc0).
4. Note how quickly context fills relative to actual useful information exchanged.

What is the expected behavior?

Codex CLI should filter and compress shell command output before adding it to the model context — the same way RTK (Rust Token Killer) works as a CLI proxy today. RTK intercepts output, strips noise, and passes a compact semantic equivalent to the model, saving 60–90% tokens on common operations.

Integration could mean: automatically piping shell tool output through RTK if it is installed, or shipping an equivalent built-in output filter so no separate install is needed.

Additional information

I use RTK today with Codex, Claude Code, and Windsurf — it works well, but it requires a separate manual install and per-tool configuration for each CLI. This is friction that shouldn't exist for something so universally useful. Output compression should be a first-class feature of the CLI itself, not an external addon users have to discover and wire up themselves.

[github-actions]

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• Feature request: make request compression explicit instead of conditional behind enable_request_compression #18677
• Fresh-thread token usage is massively inflated by enabled plugins/skills/apps in Codex Desktop #18498
• First message in fresh session consumes ~100k input tokens immediatly, no matter how trivial the input #18526
• Confirmation screen after submitting /feedback #17251
• Inefficient context compression #18318

Powered by Codex Action

[klondikemarlen]

It isn't a duplicate, this feature request proposes adding a specific library to gain pre-LLM compression for common terminal commands.

[etraut-openai]

Can you say more about the functionality you'd like to see here? What is the problem definition or use case? Assume that other community members are not familiar with RTK. We generally prioritize feature request by community upvote, so it's important for the title and description to be clear if you want to attract upvotes.

[klondikemarlen]

@etraut-openai Is that better?

[HeMuling]

Hi @etraut-openai , I would like to provide more context on this issue.

Description

RTK, Rust Token Killer, is a local CLI tool that reduces noisy command output before it reaches an LLM context. It wraps common development commands such as git status, git diff, git log, cargo test, pytest, npm test, rg, ls, curl, gh, and similar tools, then returns a compact command-aware summary instead of the full raw output.

The goal is not to change what the user asks Codex to run. The user or agent can still issue normal commands:

git status
cargo test
rg "some pattern"

The goal is to let Codex receive a smaller, semantically useful result when the raw command output is verbose, repetitive, or mostly operational noise.

Current workaround

Because Codex currently does not expose a native pre-command hook surface comparable to Claude Code or OpenCode hooks, I implemented experimental Codex support in RTK through a PATH shim adapter.

rtk init -g --codex injects a Codex-only shim directory into subprocess PATH and sets RTK_HOST=codex. The shim entrypoints, such as git, cargo, or python, point back to the RTK binary. RTK detects argv[0], rewrites eligible commands through its shared rewrite registry, and then runs the compressed equivalent.

For example:

git status

can be routed to:

rtk git status

This works, but it is a workaround. A native Codex hook would be cleaner, safer, and more complete.

Measured impact

I can share measured RTK savings from my own local usage.

Using RTK's own tracking database, rtk gain, over the most recent 30-day window from 2026-03-25 to 2026-04-24, RTK recorded:

• 10,387 tracked commands
• 32.96M input tokens before filtering
• 11.02M output tokens after filtering
• 21.95M tokens saved
• 66.6% average savings

The largest measured savings came from normal development workflows, not synthetic benchmarks:

Category	Commands	Input tokens	Output tokens	Saved tokens	Savings
Data/network output	454	12.56M	60K	12.50M	99.5%
Git	6,914	17.08M	10.75M	6.34M	37.1%
Cargo/build/test output	1,931	2.09M	131K	1.96M	93.7%
Generic command filters	332	1.03M	53K	975K	94.9%
File/search commands	23	150K	13K	137K	91.7%
GitHub CLI	33	42K	6K	35K	84.7%

This is actual measured reduction from deterministic command-aware filtering. It does not count commands that never reached RTK, so it underestimates the full opportunity in Codex. In particular, Codex currently needs a workaround based on PATH shims, and that workaround intentionally avoids some command families for safety.

Concrete examples

Example 1: git status

Raw output:

On branch pr-642
nothing to commit, working tree clean

RTK output:

📌 pr-642

For a dirty working tree, raw Git may include many sections and hint lines. RTK turns that into a bounded summary:

📌 main
✅ Staged: 3 files
📝 Modified: 4 files
❓ Untracked: 2 files

Example 2: git diff

Raw output can be hundreds or thousands of patch lines:

diff --git a/src/foo.rs b/src/foo.rs
index ...
@@ -10,7 +10,12 @@
- old implementation...
+ new implementation...
...

RTK keeps the useful structure while bounding the output:

src/foo.rs | 18 +++++++++---
src/bar.rs |  7 +++--

--- Changes ---
📄 src/foo.rs
  @@ -10,7 +10,12 @@
  + key changed lines...
  ... (truncated)
  +14 -5

Example 3: test output

Raw cargo test or pytest output often includes compilation logs, passing test progress, repeated stack frames, and long traces. RTK compresses it to the actionable failure:

FAILED tests/test_parser.py::test_invalid_input
AssertionError: expected "x", got "y"

Summary: 1 failed, 42 passed

This is usually what the model needs to decide the next edit.

Problems with the PATH shim workaround

The current workaround has some practical issues.

1. It can affect command semantics. For example, git status --porcelain=v1 uses empty output to mean a clean working tree. A filtered human-friendly output such as ok ✓ is useful for reading, but it is not equivalent for machine-readable workflows.

2. It can make Codex bypass RTK. In my session history, Codex sometimes re-ran commands through /usr/bin/git to avoid the shim when it needed authoritative raw output.

3. It adds process overhead. On my machine:

/usr/bin/git status:             ~9.7 ms median
rtk git status, clean PATH:      ~24.3 ms median
Codex shim route git status:     ~31.0 ms median
git --version raw:               ~7.9 ms median
git --version through shim:      ~21.5 ms median

This overhead is acceptable for large outputs and slow commands, but it is not ideal for very frequent small commands. A native hook could avoid much of the extra process routing.

Requested functionality

I would like Codex to expose a native pre-command and/or pre-LLM output filtering hook surface that allows trusted local tools to:

1. Inspect a command before execution (which supported).
2. Optionally rewrite it (which not supported), for example:
   • git status → rtk git status
   • cargo test → rtk cargo test
3. Preserve approval semantics, so Codex and the user still understand the original command.
4. Return filtered model-facing output while preserving raw output when exact verification is needed.
5. Provide bypass rules for machine-readable commands, such as:
   • git status --porcelain
   • gh ... --json
   • commands using --format json
6. Clearly separate:
   • original command
   • rewritten command
   • raw stdout/stderr
   • filtered/model-facing output
   • exit status

The general feature is not RTK-specific. RTK is one implementation of a broader need: local deterministic command-output reduction before shell output consumes model context.

Why this matters

Long coding sessions spend a large amount of context on routine command output. A native hook would let Codex integrate with tools that compress noisy outputs while preserving correctness and user intent. It would reduce context waste, improve long-session reliability, and avoid fragile PATH-level interception.

[artile]

I can confirm the gap from a side-by-side setup where RTK is installed globally and integrated with both Claude Code and Codex CLI.

In Claude Code, RTK can be integrated deterministically through a native PreToolUse Bash hook:

{
  "hooks": {
    "PreToolUse": [
      {
        "matcher": "Bash",
        "hooks": [{ "type": "command", "command": "rtk hook claude" }]
      }
    ]
  }
}

That gives a clean control point before command execution. For example, the hook receives the original shell command and can return an updated tool input:

git status

becomes:

rtk git status

This is deterministic, centrally configured, and does not depend on the model remembering an instruction.

In Codex CLI today, the best available integration is prompt-level guidance via AGENTS.md / RTK.md, for example telling the agent to prefer:

rtk git status
rtk cargo test
rtk npm run build
rtk pytest -q

That works only as an instruction-following convention. It is not equivalent to Claude Code's hook integration because Codex has no native pre-shell-tool interception point where a trusted local tool can inspect or rewrite a command before execution.

The missing primitive is not RTK-specific. Codex needs a deterministic shell tool hook / middleware interface that can support local command adapters such as RTK without fragile PATH shims or prompt-only rules.

A useful hook contract would expose at least:

1. The original command string and working directory before execution.
2. A way to return an updated command, or explicitly pass through unchanged.
3. A way to preserve user approval semantics around the original command.
4. A way to distinguish original command, rewritten command, raw stdout/stderr, filtered model-facing output, and exit code.
5. A safe bypass mechanism for machine-readable workflows such as git status --porcelain, gh --json, jq, or commands that explicitly need raw output.
6. A config surface in ~/.codex/config.toml, not just prompt instructions, so users can install this once globally.

A concrete config shape could be something like:

[[shell_hooks.pre_execute]]
command = ["rtk", "hook", "codex"]
timeout_ms = 200
on_error = "pass_through"

For output filtering, a second hook could be useful:

[[shell_hooks.post_execute]]
command = ["rtk", "filter", "codex"]
timeout_ms = 500
on_error = "use_raw_output"

The core requirement is that the model should not be the enforcement mechanism. Prompt-level AGENTS.md guidance is helpful, but it cannot provide deterministic adoption or safety boundaries for command rewriting. A native hook surface would let Codex support RTK and similar tools cleanly while preserving correctness and user intent.