skillstate

Explicit execution state for long-horizon LLM agent skills.

A minimal, dependency-free implementation of the SKILL.state architecture (arXiv:2608.26263) — built to experiment, measure, and extend. An independent project; not affiliated with the paper's authors.

The problem

Most agent runtimes execute skills by appending every action, observation and reasoning trace to an ever-growing conversation. Long horizons mean:

• prompt size grows with execution length → latency and cost grow with it,
• obsolete reasoning stays embedded in context → context poisoning,
• correctness depends on reconstructing state from textual history.

The idea

Replace append-only history with an explicit, mutable execution state. At every step the model receives exactly:

A_t = (P, Σ_t, O_t)

• P — the immutable skill specification
• Σ_t — the current structured execution state (JSON, schema-validated)
• O_t — the latest environment observation

The model returns one decision: an action, its args, and the complete updated state. The state update is validated against the skill's schema; invalid updates are rejected and retried. The reasoning trace is discarded immediately — it never crosses a step boundary. The prompt stays O(1) no matter how long the skill runs.

   ┌─────────────┐  ┌─────────────┐  ┌──────────────────┐
   │ skill P     │  │ state Σ_t   │  │ observation O_t  │
   │ (immutable) │  │ (validated) │  │ (latest only)    │
   └──────┬──────┘  └──────┬──────┘  └────────┬─────────┘
          └───────────────┼───────────────────┘
                          │  prompt (bounded, O(1))
                   ┌──────▼──────┐
                   │     LLM     │  any OpenAI-compatible endpoint
                   └──────┬──────┘
              {action, args, state', reasoning}
                          │
           validate state' vs schema ── invalid → error obs, retry
           state = state'; reasoning discarded
                          │
                   execute action ──► next observation ──► loop

The repo also ships a naive baseline (skillstate/baseline.py): the classic append-only loop, same environments, same token accounting — so the comparison isolates the execution model itself.

Quickstart

Zero dependencies — Python 3.10+ stdlib only. Any OpenAI-compatible endpoint works (OpenAI, OpenRouter, Ollama, vLLM, llama.cpp, ...):

export OPENAI_BASE_URL="https://your-endpoint/v1"
export OPENAI_API_KEY="sk-..."

# small demo: 3 sources x 2 batches, both execution modes
python3 examples/run_demo.py --model your-model --sources 3 --batches 2

# scale the horizon up and watch the baseline prompt grow
python3 examples/run_demo.py --model your-model --sources 8 --batches 4

Each run prints success, step count, token usage, and the largest prompt seen — the bounded-vs-growing signature is visible in that last number.

First results

First reproducible run (2026-08-31, deepseek-v4-flash via an OpenAI-compatible endpoint; pipeline env 4 sources × 2 batches, 1 run, seed run1):

mode	success	steps	prompt tokens	completion tokens	largest prompt (chars)
skillstate	✅	31	23,317	26,162	2,161 (flat)
baseline (append-only)	✅	22	25,284	3,208	4,473 (growing)

Two honest observations:

1. The bounded-prompt signature is real. skillstate's largest prompt stayed flat as the horizon doubled (2,073 chars on a 2×2 run, 2,161 on 4×2) while the baseline transcript grows linearly with every step (4,473 chars by step ~22). The gap widens with horizon length.
2. Small horizons don't show the cost crossover yet. The complete-state-update contract makes the model echo the full state every step, so completion tokens are high (26K vs 3.2K) and at this tiny scale skillstate consumed more total tokens than the baseline. What matters: state echo grows with state size (bounded by the task), while history replay grows with step count (unbounded). The interesting experiment is scaling the horizon (8×4, then 100+ steps) until the curves cross.

Both modes completed the task with zero invalid updates. Reproduce with:

python3 examples/run_demo.py --model <your-model> --sources 4 --batches 2 --mode both

Contributed results tables are welcome — bring the exact command, model, and horizon so anyone can reproduce your numbers.

Repository layout

skillstate/           core package (stdlib only)
  runner.py           the SKILL.state loop
  baseline.py         naive append-only baseline
  llm.py              OpenAI-compatible client + token accounting
  state.py            tiny JSON-schema subset validator
  spec.py             skill specification loader
  envs/pipeline.py    simulated long-horizon backfill environment
skills/
  data_backfill/      example skill: connect → fetch → store, with retries
examples/
  run_demo.py         run and compare both modes
docs/
  architecture.md     design notes + hypotheses to test

Roadmap

• more environments: terminal, web interaction, API workflows
• noise and corruption experiments: state recovery vs transcript recovery
• snapshot + resume mid-skill; state portability across models
• long horizons (100+ steps) and cost/accuracy curves per model family
• composite skills (a skill as an action inside another skill's state)

Contributing

Yes, please — this is an experiment bench and the interesting results come from many hands. Environments, skills, benchmarks, adapters, and reproduced results are all welcome. Ground rules: stdlib-only, files under 500 lines, every claim ships with a reproduction command. See CONTRIBUTING.md.

Citation

This repo implements ideas from:

Tiwari, Chung, et al. SKILL.state: Scalable Long-Horizon Agent Skills. arXiv:2608.26263.

Independent implementation for experimentation; not affiliated with or endorsed by the authors.

License

MIT