Skip to content

v0.1.0 — initial release

Choose a tag to compare

@vineetha00 vineetha00 released this 01 Jun 00:55
· 13 commits to main since this release

First release of promptops — a pip-installable, framework-agnostic, Git-native CLI that version-controls LLM prompts.

Highlights

  • Git-native storage — plain .txt files under .promptops/prompts/ + JSON registry
  • Semantic diff — sentence-transformers cosine similarity + sentence-level rich output
  • Thompson Sampling A/B — Beta(α, β) posteriors per version, winner declared at p > 0.95
  • LLM-as-judge eval — claude-sonnet-4 scores quality/relevance/safety, with heuristic fallback
  • MCP stdio server — exposes get_active_version, list_prompts, get_metrics to any MCP client
  • Drop-in GitHub Actions — prompt-regression.yml blocks PRs whose prompts regress >10%

Install

pip install promptops

Quickstart

promptops init
promptops commit payment-agent "You are a payment assistant." --tag v1
promptops commit payment-agent "You are a helpful payment assistant." --tag v2
promptops diff payment-agent v1..v2
promptops ab-test payment-agent v1 v2 --metric quality
promptops rollback payment-agent --to v1