Skip to content

v0.4.3

Choose a tag to compare

View all tags
@itsarbit itsarbit released this 22 Feb 15:57
· 24 commits to master since this release
v0.4.3
8073dd3

What's New in v0.4.3

Total-cost-aware budget enforcement

_compute_max_tokens now subtracts estimated input token cost before computing the output cap, ensuring total cost (input + output) stays within the budget ceiling. Previously only output price was considered, allowing input cost to push total spend over budget. A 1.2x safety margin on input token estimates accounts for tokenizer variance.

Parallel reservation-based execution

Independent plan steps now run concurrently via async DAG scheduling. Each step reserves its estimated cost before launch, preventing parallel steps from collectively overshooting the budget. Deadlock detection catches cyclic dependencies.

Improved ledger coverage

Every LLM call — including failed attempts and escalation retries — is recorded in the structured CostLedger. The persistent JSONL ledger (tokenwise ledger --summary) now includes planner cost in aggregate spend.

Configurable minimum output tokens

min_output_tokens is now a setting rather than a hardcoded constant. Configure via TOKENWISE_MIN_OUTPUT_TOKENS, config file, or Executor(min_output_tokens=N). Default remains 100. Set lower for workflows that need tiny outputs under tight budgets.

Clarified async usage

executor.execute(plan) auto-detects an existing event loop (Jupyter, FastAPI) and falls back to sequential execution. For concurrent DAG scheduling in async code, use await executor.aexecute(plan) directly. Budget accuracy footnote added to README — input token estimation is heuristic (chars/4 + 1.2x margin), not tokenizer-based.

Also in this release

  • mypy strict: 0 errors across all 19 source files
  • types-PyYAML added to dev dependencies
  • LLMProvider.astream_completion protocol signature corrected for async generators

Benchmark it yourself — single command, produces benchmarks/results.csv and benchmarks/pareto.png:

uv sync --group benchmark && uv run python benchmarks/pareto.py \
  --models openai/gpt-4.1-nano openai/gpt-4.1-mini openai/gpt-4.1 \
    anthropic/claude-sonnet-4 anthropic/claude-opus-4.6 \
  --csv benchmarks/results.csv --output benchmarks/pareto.png

See the Benchmarks section in the README for full model list and sample results.

Full Changelog: v0.4.2...v0.4.3

Assets 2
Loading