Prompt versioning, output validation, and run logging for LLM pipelines.
Most LLM engineering pain comes from three places: prompts that drift without anyone noticing, outputs that break downstream logic, and no record of what ran when. promptlock fixes all three — no cloud account, no dashboard, no framework lock-in.
pip install promptlock| Problem | What promptlock gives you |
|---|---|
| Prompts change silently and break things | Version-controlled prompt registry backed by a local YAML file |
| LLM output shape is unpredictable | Output contracts that validate structure, length, and patterns |
| No record of what ran in production | SQLite run logger with filtering, summary stats, and export |
pip install promptlock
# or with uv
uv add promptlockRequires Python 3.11+. Two external dependencies: pyyaml and openpyxl (for Excel export).
from promptlock import PromptRegistry, OutputContract, RunLogger
from promptlock.exceptions import ContractViolation
# 1. Save and load versioned prompts
registry = PromptRegistry("prompts.yaml")
registry.save("summarizer", "v1.0", "Summarize this document: {doc}\nLanguage: {lang}")
template = registry.load("summarizer", version="latest")
rendered = template.render(doc="AI is transforming healthcare.", lang="English")
# 2. Validate the LLM output
contract = OutputContract(
required_fields=["summary", "keywords"],
max_length=500,
min_length=20,
)
llm_output = {"summary": "AI aids diagnostics.", "keywords": ["ai", "health"]}
try:
contract.validate(llm_output)
validated = True
error = None
except ContractViolation as e:
validated = False
error = str(e)
# 3. Log the run
logger = RunLogger("runs.db")
logger.log(
prompt_name="summarizer",
version="v1.0",
model="gpt-4o",
input=rendered,
output=llm_output,
validated=validated,
error=error,
)
# 4. Export runs for analysis
logger.export("runs.csv", format="csv")
logger.export("runs.xlsx", format="excel")Store and retrieve prompt versions from a local YAML file.
from promptlock import PromptRegistry
registry = PromptRegistry("prompts.yaml")
# save a prompt version
registry.save("classifier", "v1.0", "Classify the following text: {text}")
registry.save("classifier", "v1.1", "Classify this as positive/negative/neutral: {text}")
# load a specific version
template = registry.load("classifier", version="v1.0")
# load the most recent version
template = registry.load("classifier", version="latest")
# list all prompts and versions
registry.list_prompts()
# {'classifier': ['v1.0', 'v1.1']}
# delete a version or all versions
registry.delete("classifier", version="v1.0")
registry.delete("classifier")Render prompt strings with named placeholders.
from promptlock import PromptTemplate
template = PromptTemplate("Translate this to {lang}: {text}")
print(template.variables)
# ['lang', 'text']
rendered = template.render(lang="French", text="Hello world")
# 'Translate this to French: Hello world'
# missing variables raise TemplateRenderError
template.render(lang="French")
# TemplateRenderError: Missing required template variables: ['text']Define and validate the expected shape of an LLM output.
from promptlock import OutputContract
from promptlock.exceptions import ContractViolation
# validate a JSON output
contract = OutputContract(
required_fields=["summary", "keywords"],
max_length=500,
min_length=20,
)
contract.validate({"summary": "Short summary.", "keywords": ["ai"]}) # passes
# validate a plain string
sentiment_contract = OutputContract(
allowed_values=["positive", "negative", "neutral"]
)
sentiment_contract.validate("positive") # passes
sentiment_contract.validate("unknown") # raises ContractViolation
# validate with regex
contract = OutputContract(regex_patterns=[r"\d{4}"])
contract.validate("Report from 2025") # passes
contract.validate("No year here") # raises ContractViolationAvailable rules:
| Rule | Type | Description |
|---|---|---|
required_fields |
list[str] |
Keys that must exist in a JSON output |
max_length |
int |
Maximum character length of the output |
min_length |
int |
Minimum character length of the output |
regex_patterns |
list[str] |
Patterns the output must match (all must pass) |
allowed_values |
list[str] |
Output must be one of these exact strings |
Log every LLM run to a local SQLite file and export when needed.
from promptlock import RunLogger
logger = RunLogger("runs.db")
logger.log(
prompt_name="summarizer",
version="v1.1",
model="gpt-4o",
input="Summarize this: ...",
output={"summary": "AI is evolving.", "keywords": ["ai"]},
validated=True,
)
# retrieve runs with filters
logger.get_runs(prompt_name="summarizer", validated="failed", limit=10)
# quick summary of pass/fail counts
logger.summary("summarizer")
# {'passed': 42, 'failed': 3, 'not_checked': 5}Export logged runs to CSV, JSON, or Excel for analysis or sharing.
# export all runs
logger.export("runs.csv", format="csv")
logger.export("runs.json", format="json")
logger.export("runs.xlsx", format="excel")
# export with filters
logger.export("failed.csv", format="csv", validated="failed")
logger.export("summarizer.xlsx", format="excel", prompt_name="summarizer")All three formats share the same columns: id, prompt_name, version, model, input, output, validated, error, timestamp.
All exceptions inherit from PromptlockError so you can catch broadly or specifically.
from promptlock.exceptions import (
PromptlockError, # base exception
ContractViolation, # output failed validation
PromptNotFound, # prompt name/version not in registry
TemplateRenderError, # missing variable during render
)src/promptlock/
├── __init__.py # public API
├── registry.py # PromptRegistry
├── template.py # PromptTemplate
├── contract.py # OutputContract
├── logger.py # RunLogger
└── exceptions.py # custom exceptions
Pull requests are welcome. For major changes, please open an issue first.
git clone https://github.com/NorthCommits/Promptlock
cd Promptlock
uv sync
uv run pytest -vMIT