Repository navigation
v0.1.0 — initial release
First release of promptops — a pip-installable, framework-agnostic, Git-native CLI that version-controls LLM prompts.
Highlights
- Git-native storage — plain
.txtfiles under.promptops/prompts/+ JSON registry - Semantic diff — sentence-transformers cosine similarity + sentence-level rich output
- Thompson Sampling A/B — Beta(α, β) posteriors per version, winner declared at p > 0.95
- LLM-as-judge eval —
claude-sonnet-4scores quality/relevance/safety, with heuristic fallback - MCP stdio server — exposes
get_active_version,list_prompts,get_metricsto any MCP client - Drop-in GitHub Actions —
prompt-regression.ymlblocks PRs whose prompts regress >10%
Install
pip install promptopsQuickstart
promptops init
promptops commit payment-agent "You are a payment assistant." --tag v1
promptops commit payment-agent "You are a helpful payment assistant." --tag v2
promptops diff payment-agent v1..v2
promptops ab-test payment-agent v1 v2 --metric quality
promptops rollback payment-agent --to v1