llm-legs

Battle-tested subscription-only vendor legs for local multi-model orchestration: call OpenAI GPT, Google Gemini, and Anthropic Claude headlessly through their subscription CLIs — no API keys, no per-call billing — with tier guards, quota signaling, and an audit trail.

Shared by local "model-symbiosis" projects (e.g. multi-model offer research, portfolio decision-support) as a git submodule. One fix here propagates to every consumer via a pin bump.

The legs

Script	Vendor	Transport	Default model
ask_codex.sh	OpenAI	codex exec (read-only sandbox)	CLI flagship (auto-tracked, no pin)
ask_gemini.sh	Google	Antigravity agy --print	the Pro label geminib families prints, (High) → in-family fallback (Low)
ask_claude.sh	Anthropic	claude -p --output-format json	opus alias (cost ceiling: Opus — Mythos-class refused)

Contract (what consumers may rely on)

• Input: prompt as $1 or stdin. Output: the model's text answer on stdout.
• Exit codes: 0 ok · 1 transport/leg failure · 3 weak-tier model refused (a weak model must never sit in a judgment seat) · 5 subscription quota exhausted (callers should drop the leg for the rest of the run instead of retrying after exhaustion).
• Audit: every call appends {ts, leg, requested, served, weak_tier} to $LLM_LEGS_DATA_DIR/served-models.jsonl (default: $PWD/data — orchestrators invoke legs with cwd = project root; set LLM_LEGS_DATA_DIR explicitly for cron/launchd).
• Probes: every leg supports --probe (health check; gemini's spends no quota).

Guard rails baked in

• Tier guard, not version pin: served models are extracted and classified; weak tiers (mini/nano/lite/flash/haiku/...) fail the call rather than silently degrade judgment. Overrides: CODEX_ALLOW_WEAK=1, GEMINI_ALLOW_WEAK=1, CLAUDE_ALLOW_WEAK=1.
• Billing traps disarmed: stray OPENAI_API_KEY / ANTHROPIC_API_KEY env vars are unset so calls never silently flip from subscription to API billing.
• Quota → exit 5: "you've hit your usage limit" (codex) and quota signals (gemini) are mapped to a distinct exit code; agy's silent-quota behavior (rc 0 with empty output, the real RESOURCE_EXHAUSTED only in its log) is documented in the wrapper headers.
• Read-only: codex runs in a read-only sandbox; claude disallows mutating tools; gemini (agy) is print-mode only.
• Receipts where possible: claude's served model is verified from .modelUsage (auxiliary haiku entries do not poison the guard); codex's from the CLI banner. agy does NOT report the served model — its audit rows record the pin, marked unverified.
• English between models: bin/cyrillic-share reads text on stdin and prints the share of Cyrillic letters outside code and closed «…» quotes of up to 6 words (a trigger phrase), rounded up; worker-run refuses a brief with any, or one it cannot measure, prepends an AUDIENCE line telling the worker its reader is a model, and stamps a Russian result LANG: cyrillic N%; claude-resume-timer -m refuses Russian too.

Knobs

CODEX_MODEL, CODEX_EFFORT (minimal..xhigh) · AGY_MODEL, AGY_MODEL_FALLBACK, AGY_PRINT_TIMEOUT · CLAUDE_MODEL (sonnet/opus tiers) · LLM_LEGS_DATA_DIR.

Consuming as a submodule

git submodule add https://github.com/LoyEgor/llm-legs lib/legs
# call: lib/legs/ask_codex.sh "prompt"   (invoke with cwd = your project root)
# update later: git -C lib/legs pull && git add lib/legs && git commit -m "bump llm-legs"

Tests: python3 -m unittest discover tests.

Daily self-check

llm-selfcheck runs a ~10s live smoke (Hammerspoon alive, menu builds), then the hermetic limits, claudeb, codexb, and geminib suites every day at 10:30 local time. llm-selfcheck run --e2e adds the full tests/e2e_surfaces.sh on top and skips the daily debounce — the change-time invocation, not a scheduled one. Each run is recorded in ~/.claude-profiles/.claudeb/selfcheck.log; failures also raise a Hammerspoon alert and a macOS notification. Before any suite, the path-links step fails naming every bin/X → ~/.local/bin/X link listed in this README that is missing or points elsewhere.

Subscription limit collector

llm-limits.sh normally reads cached local CLI state without invoking a vendor CLI, making an external network request, or spending tokens. It prints schema-1 JSON and atomically refreshes ~/.llm-limits.json by default. When no output flag is given and stdout is a terminal, it renders the --table view instead; piped or redirected output keeps the JSON default, and an explicit --json, --plain, or --table always wins. The stable top level is {schema, fetched_at, vendors}. Claude includes current_account, an accounts array in render order (each vendor's pinned primaries first, then oldest profile directory first, accounts without one last by name — every surface renders that order as-is, and is_current alone marks the current account), and the current account's five_hour, optional weekly, as_of, stale_seconds, and age_alarm hoisted at vendor level for compatibility. Each account has its own windows and freshness; account and vendor age use the oldest as_of among windows with a numeric used_pct, ignoring absent and null-valued windows. age_alarm is true when that age is a day or more old, or when there is no dated window at all; --table, --plain, and the menubar paint such an age red (never when it is missing entirely), while claudeb status/accounts render the same age uncoloured. Each account also has an enabled flag reflecting worker-selection membership (absent means enabled). Use --plain for a human-readable line per account or --no-write to leave the cache untouched. --table renders the same model as aligned columns: 5h, weekly, and Fable percentages and local reset times; account age; Claude worker-pool state; Codex reset credits; and vendor status. Fable is - for rows without that bucket. ~ and ! suffix percentage values whose snapshots are stale or whose reset has passed, without rewriting the last known used_pct. ROT is off for a disabled Claude account, limit-5h/limit-weekly/fb:limit-fable when a local effective percentage reaches 100, or - otherwise. Transient upstream 429s are handled by the Claude CLI's own retry behavior. CR renders Codex reset credits as ↻N. The current account is marked *; plan tags and Gemini quota-group labels are omitted. Percent columns are colorized only when stdout is a TTY; piped output stays plain text. --sort 5h|weekly|reset reorders the table by that column (descending for percentages, ascending by the nearest of the 5h, weekly, and Fable resets). For a PATH entry point, symlink the script — it resolves its own symlink, so helper discovery keeps working:

ln -s /Volumes/Work/Projects/llm-legs/llm-limits.sh ~/.local/bin/llm-limits
llm-limits --table --sort 5h
llm-limits --plain

--refresh is reserved for the manual Get Data & Refresh action. It always performs Claude's free usage-endpoint poll through claudeb accounts --no-spend and fetches each Gemini profile's quota through agy's authenticated localhost Connect RPC. The Codex leg is a zero-spend usage query too: codex-quota.py asks the local codex app-server for account/rateLimits/read and the response is cached in ~/.llm-limits-codex.json; rollout session files remain the passive fallback when they are newer or the helper is unavailable. --refresh --start-windows is the only paid path: for each vendor whose five-hour window has already reset it issues one minimal model call (claudeb's own --refresh --start-windows, a one-word codex exec, a one-word agy --print) to start a fresh window, then re-reads the free usage endpoints. Vendors that cannot be started are reported on stderr, never skipped silently. The Gemini request is /usage itself in print mode; it consumes no model tokens. The main profile keeps the legacy cache ~/.llm-limits-gemini.json; named profiles use ~/.llm-limits-gemini/<name>.json. --gemini-remove is the menubar's spelling of geminib remove main: both write ~/.llm-limits-gemini.json.removed, which takes main out of every roster until that file is deleted. --refresh-account gemini/<name> refreshes only that profile; a bare vendor name (--refresh-account claude|codex|gemini) refreshes every account of that vendor and touches no other, always free — --start-windows stays a single-account request. Without --refresh, collection remains token-free and external-network-free.

Machine contract

Consumers read ~/.llm-limits.json. A bare llm-limits run is a passive read with zero external network access. llm-limits --refresh performs a free live refresh. Scripts must never invoke --start-windows; it spends money. Read each bucket's effective_pct and each vendor's usable_now. Never make availability decisions from raw used_pct. vendors.codex.accounts is always a non-empty array when Codex is available; legacy snapshots synthesize main. vendors.codex.current_account names the account whose buckets remain hoisted at vendor level. Each vendors.codex.accounts[] may expose reset_credits and auth_needed; auth-needed accounts have no usage buckets and are never usable. vendors.gemini.accounts contains per-profile buckets, auth-needed state, refresh causes, and removed markers; the vendor-level buckets hoist from the lowest-spending usable account, and a main removed through its marker (geminib remove main) appears nowhere. Raw usage values persist for provenance after a window expires, while effective_pct becomes 0. Claude usable_now considers enabled, authenticated accounts and their general 5h/weekly limits; the model-specific fable bucket does not block other Claude work. Codex and Gemini additionally require available == true. Respect bucket and vendor stale flags when freshness matters. Claude reads the $CLAUDEB_DIR/limits/*.json accounts (CLAUDEB_DIR defaults to ~/.claude-profiles/.claudeb) and uses .claudeb-state to select the current account. main (the default plain-claude login, not a real claudeb token account) is excluded from every output — JSON, cache, --plain, and --table — so only real claudeb token accounts are reported. The terminal formats expose the same per-account windows, age, rotation, credits, and status fields. If the store is absent, it falls back to the freshest Claude status-line snapshot as a single main account.

claudeb disable <name> removes an account from worker selection and claudeb enable <name> puts it back; the set lives in $CLAUDEB_DIR/disabled (one name per line, missing file = all enabled). An explicit claudeb profile <name> launch bypasses membership. The last enabled account cannot be disabled, and each JSON account carries the resulting enabled flag.

Set LLM_LIMITS_WALLS_LOG to a served-models.jsonl audit log to include the most recent exit-5 wall timestamp for each vendor. LLM_LIMITS_CACHE overrides the cache path.

For Hammerspoon, copy or symlink hammerspoon/llm-limits.lua into ~/.hammerspoon. Requiring the module has no side effects. Use its menu items inside an existing menu:

local limits = require("llm-limits")
local submenu = { title = "LLM Limits", menu = limits.menuItems() }

The LLM doctor: … row opens ONE submenu: LLM doctor's four blocks, then the Gemini pin:

LLM doctor: 6 problems
  Reviewers: 1 problem
    owner: Review-bench improvements phase 4 · 212 legs in 3 d
    failed · pool empty   3  ↑   █   █  58m  new                 V2  grok47
    ───
    failed · bad output  17     ▁▃▅▄▃▆   5h  fixed 0d · 0 since  R1  flash37, opus
    cap · watchdog       40     ▃▃▅▂▁    9m  weather                 flash38
    top: cap · watchdog
    ───
    By model
    Copy brief for the owner
  Review machinery: 2 problems
    closure_pending: 23 · open  M2
      <age>  <project>  <chat>  <label>
    debt_line: 2 · new
    Rescan now
  Workers: ok
  Light: 3 problems
  Image: no legs
  window: 3 d → 3 h / 6 h / 12 h / 24 h / 3 d / 7 d
  Refresh blocks
  Gemini
    review flash: 3.8

A block is one kind of LLM leg — Reviewers (review-bench cells, chunks, judges and panels), Workers (worker-run, prelaunch refusals included), Light (the light research and edit legs) and Image (the image and video wrappers) — and every block classifies its legs the same way. A bug is a failed leg whose origin is ours or unknown, or an escaped one; everything else — walled, cap, stalled, failed · theirs, slow, and a leg a retry superseded — is weather, which is counted but never a bug. Each bug row carries its state against share/doctor-ledger.json: new (no ledger row), open, regressed (legs after the row's fixed_at) or fixed. A row opens its last 15 incidents and Copy for an LLM; Copy brief for the owner hands the whole block to the chat that owns it. Review machinery is review-bench doctor's snapshot, a row of its own right after Reviewers, each class held against the same ledger. Every row counts problems — one cause each, never legs or weather — and the rows sum to the title. The window selection lasts for this module session and forces a collection. The doctor row appends · rescanning during rescans; action rows are disabled and read rescanning… or refreshing… while their tasks run.

Vendor	Limit freshness
Claude	Per-account claudeb file mtime; status-line snapshot fallback
Codex	Live app-server rate-limits RPC on refresh; last rollout event otherwise
Gemini	Per-profile last successful manual Get Data & Refresh through agy's print-mode /usage

Gemini refresh runs agy -p /usage --output-format json under each profile's HOME, which answers the read-only usage command without a model turn, quota spend or conversation. agy ignores BROWSER and opens its OAuth page with open off PATH, so the no-op share/no-browser/open goes first on PATH and a logged-out profile cannot open a tab; the Authentication required line on stderr ends the run as login needed. Set AGY_WORKDIR to the folder agy should run in. Overrides for tests or alternate installs: AGY_BIN, LLM_LIMITS_GEMINI_CMD, LLM_LIMITS_GEMINI_CACHE, GEMINIB_PROFILES_DIR, and LLM_LIMITS_GEMINI_ACCOUNTS_DIR.

claudeb multi-account suite (bin/)

Canonical sources for the multi-account Claude Code tooling; installed via symlinks:

• bin/claudeb → ~/.local/bin/claudeb — account prober/launcher (OAuth auto-refresh, --start-windows, headless passthrough for worker agents).
• bin/statusline.sh → ~/.claude/statusline.sh — Claude Code statusline; also writes per-account limit snapshots on real usage.

Snapshot store and schema live in ~/.claude-profiles/ (documented in its README).

• bin/chat-find → ~/.local/bin/chat-find — find a past chat by what was SAID in it; --recent lists every chat instead of searching. Both rank by the last real message, never by file mtime: opening a chat rewrites its transcript, so mtime says "just now" about a chat nobody spoke in. Headless runs (workers, benches) are left out unless --agents; the listing caches per (size, mtime) in ~/.claude-profiles/.claudeb/chat-recent-cache-hot.json (chats spoken in within HOT_DAYS, the only file a week-wide listing reads or writes) and chat-recent-cache-cold.json (the rest, loaded and reconciled on wider listings only).
• bin/chats → ~/.local/bin/chats — full-screen picker over that listing: ↑↓ chat, ←→ claudeb account (default: worker-pick --account claudeb --role chat, else the account in use as llm-limits reports it, else .claudeb-state), typing filters, Enter execs `claudeb profile

  --resume --permission-mode bypassPermissions` in the target chat's directory. The context column names the account instead of a size while that account's prompt cache still holds the chat (the statusline's verdict, shared-invariants row bj), and landing on such a chat moves the account to it; one with no live cache returns to that default, and ←→ holds only until the cursor moves on. It opens on the last week and loads more history when he scrolls past the end; `--days N` / `--all` pin one window instead, `--print` shows the launch line rather than running it. Needs a terminal of its own.

• bin/chat-name → ~/.local/bin/chat-name — <session-id>... prints <original title> (<short id>) off the one resolver (share/chat_names.py), and exits 1 with nothing where the chat has no name it may surface by. Several ids in one call are a line each, which is how a caller naming a turn's chats pays the corpus walk behind an 8-hex prefix once instead of once per id. The symlink is the whole install: the shell surfaces that hold no Python find it on PATH (../claude-setup/hooks/commit-report.sh) and fall silently back to a bare uuid when they cannot, so an unlinked entry point is a feature that never once runs.
• bin/chat-pin → ~/.local/bin/chat-pin — [claudeb|claude|codex|gemini|grok|<account>|auto] pins THIS chat's workers in ${CHAT_PINS_DIR:-~/.cache/claude-chat-pins}/<session_id>, replacing the global pins there: a vendor means any of its pool accounts (*), auto drops the pin, no argument prints it. Inside a session it runs only on the grant Egor's own words wrote.
• bin/claude-resume-timer → ~/.local/bin/claude-resume-timer — [terminal|all] [extra-minutes] reads the chat's account's 5h window from ~/.llm-limits.json and arms the Hammerspoon ClaudeContinue.startTimerFor terminal slot of the calling chat's tty (all: every live chat) for that reset + extra minutes (default +10), falling back to +15 minutes if the window is expired or unknown. It arms nothing for a worker run or a caller outside every Claude chat; the app and Kimi slots are the Hammerspoon menu's alone.

Scripts the agents run by name

The claude-setup agent definitions call these bare, so each needs its PATH link (tests/test_consistency.sh holds every name they run to this list):

• bin/worker-run → ~/.local/bin/worker-run — relay launcher every worker agent drives (start, wait, report).
• bin/worker-pick → ~/.local/bin/worker-pick — account and vendor pick per docs/routing-contract.md.
• bin/grokb → ~/.local/bin/grokb — SuperGrok multi-account launcher.
• bin/media-run → ~/.local/bin/media-run — the one door to every media script: picks the vendor's script from share/image-caps/*.json, passes the script's own args byte for byte, fans several vendors, --takes or --jobs out through image-fanout, and drops the job pointer the statusline's media line reads.
• bin/codex-image → ~/.local/bin/codex-image — image generation and editing on Codex (media-run's default image vendor).
• bin/gemini-image → ~/.local/bin/gemini-image — image generation and editing on Gemini.
• bin/grok-image → ~/.local/bin/grok-image — image generation and editing on Grok.
• bin/grok-video → ~/.local/bin/grok-video — video clips on Grok Imagine.
• bin/image-fanout → ~/.local/bin/image-fanout — one request across every vendor and account.
• bin/image-cutout → ~/.local/bin/image-cutout — exact-pixel background removal with macOS Vision (no generation).
• bin/gemini-video → ~/.local/bin/gemini-video — Veo/Omni clips through Google Flow.
• bin/video-chain → ~/.local/bin/video-chain — frame extraction for chaining clips.
• bin/gemini-music → ~/.local/bin/gemini-music — music through the Gemini app's Lyria tool.
• bin/gemini-sfx → ~/.local/bin/gemini-sfx — sound effects from a Flow Omni render.
• bin/gemini-listen → ~/.local/bin/gemini-listen — a Gemini model's judgement of video or audio.
• bin/gemini-web → ~/.local/bin/gemini-web — the hidden Chrome behind the Gemini media scripts (login <profile>, which geminib web <profile> runs for a roster account).
• bin/chatgpt-web → ~/.local/bin/chatgpt-web — the same hidden Chrome on chatgpt.com, behind codex-image --route web (login <codex profile>, which codexb web <codex profile> runs for a roster account).
• bin/llm-doctor → ~/.local/bin/llm-doctor — LLM doctor; claude-setup's night-sweep skill runs it and the six below by name.
• bin/harness-doctor → ~/.local/bin/harness-doctor — Harness doctor.
• bin/updater-doctor → ~/.local/bin/updater-doctor — Updater doctor.
• bin/code-doctor → ~/.local/bin/code-doctor — Code doctor.
• bin/doctor-fix → ~/.local/bin/doctor-fix — doctor fixer briefs and their close gate.
• bin/vendor-fingerprint → ~/.local/bin/vendor-fingerprint — vendor CLI fingerprint requests.
• bin/night-run → ~/.local/bin/night-run — the night run's jobs, base and report.
• bin/pkill → ~/.local/bin/pkill — shadows /usr/bin/pkill for every model and vendor: refuses an option after the first pattern (macOS stops option parsing there, and a stray -P pattern matches every Claude Bash shell's pwd -P), else execs the real tool.
• bin/pgrep → ~/.local/bin/pgrep — the same guard for /usr/bin/pgrep (a link to bin/pkill).

codexb multi-account suite

Same idea for OpenAI Codex CLI, without a proxy daemon: each extra account lives in its own CODEX_HOME under ~/.codex-profiles/<name> (auth, sessions, history are per-account; config, AGENTS.md, skills, plugins, rules are symlinked to ~/.codex). The default ~/.codex account is always available as main and is never modified.

• bin/codexb → ~/.local/bin/codexb — profile <name> [args...] (create the profile if missing and exec codex under it, so a bare launch prompts login; aliases p, run), add <name> (create a profile without launching + print the login command), list, status (per-account quota via the zero-spend app-server RPC), pick (freest usable account).

Adding an account: codexb profile work creates it and launches codex, which prompts login in one step (mirrors claudeb profile <name>); codexb profile work login runs the browser OAuth flow the menu uses. On a tty a first codexb profile work then offers codexb web work, the hidden-Chrome sign-in the ChatGPT web image route needs. Afterward codexb status should show both accounts. Worker agents pick the freest account automatically (codexb pick) unless pinned via codex_profile= in the worker toggle file; the menubar shows per-account rows once more than one account exists.

geminib multi-account suite

The existing Antigravity login is main and continues to use the real home directory unchanged. Named accounts live under ~/.gemini-profiles/<name> and launch with HOME=~/.gemini-profiles/<name>; shared non-auth Gemini settings are symlinked from ~/.gemini, and Playwright driver/browser caches are shared from the base HOME. Antigravity exposes no narrower supported profile flag or environment variable. Each profile also gets its own Library/Keychains/login.keychain-db, created on first use with a random password kept in the profile's .keychain-password. agy stores its OAuth token in the login keychain macOS resolves from $HOME and prefers it over the profile's token file, so a profile sharing the real keychain silently authenticates as the base account — a profile without any keychain instead raises the modal "A keychain cannot be found to store antigravity" dialog on every refresh. A profile still holding the old symlink is converted on the next geminib run and has to sign in once more. macOS refuses security unlock-keychain on anything named login.keychain-db, so a profile keychain that locks is signed in again rather than unlocked.

• bin/geminib → ~/.local/bin/geminib — profile|p|run <name> [args...] creates a missing profile and launches agy for its one-time login, add <name> creates without launching, remove <name> forgets any named profile but never main, and list/status report every profile.
• bin/gemini-probe → ~/.local/bin/gemini-probe — by-hand active check of Gemini serving per model family (a short and a long request each, capacity fallback off); docs/DIAGNOSTICS.md has the reading.
• bin/light-research → ~/.local/bin/light-research — read-only multi-repository research on the light_research row's vendor and model (default the table's first gemini row) with account selection through role research, automatic rotation after a quota wall, and a per-vendor read-only guard. Its terminal exits are 3 for usage limit, 4 for unavailable, 5 for a read-only violation, and 124 for timeout.

Adding an account: geminib profile work opens an isolated, logged-out Antigravity profile and prompts for Google login; on a tty it then offers geminib web work, the one hidden-Chrome sign-in the media routes (Flow video, Flow Music, Gemini app music) need. Then run geminib status and llm-limits --refresh-account gemini/work --table. Worker routing follows docs/routing-contract.md (pool toggle, lowest spending bucket, walls only at 100%/dead auth), prints ACCOUNT: work, and ranks main last on a tie unless gemini_profile= pins it.

OpenCode Go review models

bin/opencode-go is a read-only client for the OpenCode Go subscription: an OpenAI-compatible gateway whose base URL is pinned, so pay-as-you-go Zen models can never be billed. The key comes from the macOS Keychain and never reaches curl's argv; OPENCODE_GO_PROFILE=<name> selects the service opencode-go-<name> when more than one subscription key exists. There is no usage endpoint — a spent window is only visible as an HTTP 429 naming limitName, so a wall means stop and come back later, never retry.

review-bench reads key priority from ~/.config/opencode-go/profiles, one profile name per line. A line containing only - selects the default unnamed key; blank lines and lines beginning with # are ignored. If the file is missing, only the default key is used.

• opencode-go key stores a key, models lists the plan, run <model> … reviews a prompt file, raw <path> posts an arbitrary body. --effort and --no-reasoning are mutually exclusive: the reasoning-off knob silently overwrites an effort, and which knob a model accepts varies between requests, so the client negotiates per request rather than pinning a strategy.

These models are review raters only, never workers: the client has no agentic loop. Their measured capability, the composition that survived strict adjudication, and the plan models that failed screening are printed by review-bench oc-models; the bench runs them raw as review-bench run <sha> --leg, and a review runs them under a tier, where the finding verifier lives.