rulelab v0.1.2

rulelab is on PyPI as rulelab-cli. The name rulelab is too close to an existing project there, so the package has a different name; the command is still rulelab.

uvx rulelab-cli demo --open
# or
uv tool install rulelab-cli && rulelab --help

Changed

• Published on PyPI as rulelab-cli; rulelab-cli also works as a command, so uvx rulelab-cli runs it.
• Depends on commitarena==0.1.1 from PyPI instead of its GitHub tag.

Added

• Claude Code plugin (/plugin install rulelab --marketplace diegoperezg7/rulelab) with the ablate-agent-rules skill, which splits your AGENTS.md or CLAUDE.md, shows the free cost estimate and runs the leave-one-out experiment only after you approve it.

The recorded demo results do not change.

rulelab v0.1.1

Report polish. No changes to the experiment, the statistics or the recorded demo data.

Fixed

• The HTML report formats numbers for the active language: Spanish uses 1.953, 0,23 US$; English keeps 1,953, 0.23 USD.
• The terminal summary prints the same two figures with their names: "estimated if resent on every call: ≈4,305 · measured difference vs no file: +776".
• The cost-projection footnote is fully translated (the price label was in English in the Spanish view).
• The file-cost card names both figures and keeps them apart: the estimated tokens per run if the file is resent on every model call (≈328 tok × ~13 calls ≈ 4,305) and the measured difference against no file in this run (+776 tok/run).

Recorded demo (unchanged)

mistral/ministral-14b-latest, commitarena 0.1.1, 4 sqlparse tasks x 8 configurations x 2 repeats = 64 agent runs, 0.74 USD. 13 of 64 runs passed; full file 2/8 vs no file 1/8. Every rule is inconclusive at this sample size.

Install

uvx --from git+https://github.com/diegoperezg7/rulelab@v0.1.1 rulelab demo --open

PyPI release comes once commitarena is published on PyPI.

rulelab v0.1.0

First release.

rulelab splits an AGENTS.md or CLAUDE.md into rules, then runs your coding agent on real bug-fix tasks from your repo with no file, the full file, and the full file minus each rule (leave-one-out). Each rule gets a bootstrap interval, a verdict (helps / no effect / hurts / inconclusive), its tokens and a projected monthly cost.

Highlights

• rulelab split, rulelab run, rulelab report, rulelab demo
• Real tasks and a no-network Docker sandbox through the commitarena engine (v0.1.1)
• Cost estimate before running; --max-cost (default 1 USD) checked before every run; --resume; --concurrency
• Static HTML report (EN/ES, light/dark, mobile) with the annotated file: https://diegoperezg7.github.io/rulelab/

Recorded demo (real run)

mistral/ministral-14b-latest, commitarena 0.1.1, a 6-rule AGENTS.md, 4 sqlparse tasks from commitarena's demo set, 8 configurations x 2 repeats = 64 agent runs (27 min, 0.74 USD). 13 of 64 runs passed; full file 2/8 vs no file 1/8 (+12 pp, interval -21 to +39). Every rule is inconclusive at this sample size, and the report says so. Cost: resent on each of ~13 model calls, the ~328-token file adds an estimated ≈4,305 input tokens per run; the difference measured in this run vs no file was +776 tokens per run (wide interval); the repository overview is the most expensive rule (~88 tokens, 27% of the file).

Install

uvx --from git+https://github.com/diegoperezg7/rulelab@v0.1.0 rulelab demo --open

PyPI release (uvx rulelab) comes once commitarena is published on PyPI: PyPI does not accept the current GitHub dependency.