PromptGit

Version prompts in Git repositories like your code.

PromptGit brings the power of Git version control to AI prompts, enabling teams to manage, track, and collaborate on prompt engineering with the same rigor as code. Built for production environments with comprehensive tooling for experimentation, policy enforcement, and CI/CD integration.

✨ Key Features

🔄 Git-Native Versioning

• Full Git integration: Store prompts as versioned files with complete commit history
• Branch support: Work with branches, tags, and commit hashes
• Rollback capabilities: Easily restore to any previous version
• Blame tracking: Line-by-line attribution for prompt changes

🎯 Advanced Diff Analysis

• Semantic comparison: TF-IDF and cosine similarity analysis beyond text diffs
• Risk assessment: Automatic detection of high-impact changes
• Behavioral analysis: Identify changes that may affect prompt behavior
• GitHub-style visualization: Side-by-side diffs with syntax highlighting

🧪 A/B Experimentation

• Controlled testing: Compare prompt versions with statistical rigor
• Configurable metrics: Quality, safety, latency, cost, and custom metrics
• Results tracking: Detailed performance analysis and candidate comparison
• Best candidate detection: Automatic identification of top performers

🛡️ Enterprise Policy Management

• Ownership controls: CODEOWNERS-style approval workflows
• Pattern-based rules: Flexible glob patterns for different prompt types
• CI integration: Automated approval checking in pull requests
• Audit trails: Complete change tracking and compliance reporting

🚀 Quick Start

Prerequisites

• Node.js 18.0 or higher
• pnpm 8.0 or higher
• Git 2.20 or higher

Installation

# Clone the repository
git clone https://github.com/your-org/promptgit.git
cd promptgit

# Install dependencies
pnpm install

# Build all packages
pnpm build

Initialize Your First Repository

# Initialize in current directory
./promptgit init

# Or specify a custom location
./promptgit init --repo /path/to/your/prompts

Add Your First Prompt

# Save a prompt with metadata
./promptgit save prompts/agent.system.prompt.md \
  --meta owner=team-ai \
  --meta version=1.0.0 \
  --meta tags=system,core \
  --message "Initial system prompt for AI agent"

Launch the Web Interface

cd apps/prompt-ui
REPO_DIR=/path/to/your/prompts pnpm dev

Visit http://localhost:3000 to start managing your prompts with the web interface.

📋 Core Commands

Prompt Management

# List all prompts
./promptgit list

# Get a specific prompt
./promptgit get prompts/agent.system.prompt.md

# Compare versions
./promptgit diff prompts/agent.system.prompt.md HEAD~1 HEAD

# Rollback to previous version
./promptgit rollback prompts/agent.system.prompt.md --to HEAD~1

Experimentation

# Run an A/B experiment
./promptgit experiment run experiments/assistant-comparison.json

# List experiment results
./promptgit experiment list

Policy Management

# Add ownership rules
./promptgit policy add-owner "prompts/*.md" alice bob

# Check approval status
./promptgit policy check-approvals --approvers alice bob

Core Library (prompt-store)

• Git operations: Pure JavaScript Git implementation using isomorphic-git
• Semantic analysis: TF-IDF and cosine similarity algorithms
• Metadata parsing: YAML frontmatter extraction and validation
• Policy engine: Ownership and approval workflow management

CLI Tool (prompt-cli)

• Command interface: Intuitive CLI with comprehensive help
• Batch operations: Efficient handling of multiple prompts
• Integration ready: Designed for CI/CD and automation
• Cross-platform: Works on Windows, macOS, and Linux

Web Interface (prompt-ui)

• Modern stack: Built with Nuxt 3, Vue 3, and TypeScript
• Real-time updates: Hot module replacement and live collaboration
• Responsive design: Mobile-first approach with dark mode
• Accessibility: WCAG 2.1 compliant interface

📝 Prompt File Format

Prompts are stored as Markdown files with YAML frontmatter:

---
owner: team-ai
description: "Core system prompt for AI agent behavior"
version: "1.2.0"
tags: [system, core, agent]
created: "2024-01-15T10:30:00Z"
updated: "2024-01-20T14:45:00Z"
---

You are a helpful AI assistant designed to...

[Your prompt content here]

Supported Metadata Fields

• owner: Team or individual responsible for the prompt
• description: Human-readable description of the prompt's purpose
• version: Semantic version number (e.g., "1.2.0")
• tags: Array of tags for categorization and search
• created: ISO 8601 timestamp of initial creation
• updated: ISO 8601 timestamp of last modification

🔧 Configuration

Environment Variables

# Web UI Configuration
REPO_DIR=/path/to/your/prompts
AUTH_TOKEN=your-secret-token

# Git Configuration (Optional)
GIT_AUTHOR_NAME=Your Name
GIT_AUTHOR_EMAIL=your.email@example.com

Policy Configuration

Create a .promptowners file in your repository root:

# PromptGit Ownership Rules
# Format: pattern approver1 approver2 ...

# General prompts require team approval
prompts/*.md alice bob

# System prompts require senior approval
prompts/system/*.md charlie david

# Sensitive prompts require multiple approvals
prompts/sensitive/*.md alice charlie david eve

🧪 A/B Experimentation

Creating Experiments

Define experiments using JSON manifests:

{
  "name": "assistant-personality-test",
  "description": "Compare different personality styles for customer service",
  "candidates": [
    "prompts/assistant.prompt.md#main",
    "prompts/assistant.prompt.md#HEAD~1",
    "prompts/assistant.prompt.md#HEAD~2"
  ],
  "metrics": [
    "exact_match",
    "quality",
    "safety",
    "latency",
    "cost"
  ]
}

Running Experiments

# Run with default test data
./promptgit experiment run experiments/personality-test.json

# Run with custom input data
./promptgit experiment run experiments/personality-test.json \
  --input data/test-cases.jsonl \
  --output results/personality-test/

Input Data Format

Create test cases in JSONL format:

{"input": "What is the capital of France?", "expected": "Paris"}
{"input": "Write a short story about a robot", "expected": ""}
{"input": "Explain quantum computing", "expected": ""}

🔄 CI/CD Integration

GitHub Actions

The included workflow automatically analyzes prompt changes:

name: Prompt Analysis
on: [pull_request]

jobs:
  analyze:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Analyze Prompt Changes
        run: |
          ./promptgit policy check-approvals
          node scripts/ci-summary.cjs

Manual CI Analysis

# Generate change summary
A_SHA=HEAD~1 B_SHA=HEAD node scripts/ci-summary.cjs

# Check approval compliance
./promptgit policy check-approvals --approvers $REVIEWERS

🌐 API Reference

REST Endpoints

Endpoint	Method	Description
/api/prompts	GET	List all prompts
/api/prompts/{path}	GET	Get specific prompt
/api/prompts/{path}/diff	GET	Get diff between versions
/api/prompts/{path}/blame	GET	Get blame information
/api/experiments	GET	List experiments
/api/experiments/run	POST	Run new experiment
/api/policy/owners	GET	List ownership rules
/api/policy/check-approvals	POST	Check approval status

Example API Usage

# List all prompts
curl http://localhost:3000/api/prompts

# Get specific prompt
curl "http://localhost:3000/api/prompts/agent.system.prompt.md?ref=main"

# Get diff between versions
curl "http://localhost:3000/api/prompts/agent.system.prompt.md/diff?refA=HEAD~1&refB=HEAD"

🛠️ Development

Building from Source

# Install dependencies
pnpm install

# Build all packages
pnpm build

# Build specific package
pnpm --filter @promptgit/prompt-store build
pnpm --filter @promptgit/cli build
pnpm --filter @promptgit/ui build

Development Mode

# Start web UI in development mode
cd apps/prompt-ui
REPO_DIR=/path/to/your/prompts pnpm dev

# Watch mode for packages
pnpm --filter @promptgit/prompt-store dev
pnpm --filter @promptgit/cli dev

Testing

# Run all tests
pnpm test

# Run tests for specific package
pnpm --filter @promptgit/prompt-store test
pnpm --filter @promptgit/cli test
pnpm --filter @promptgit/ui test

📄 License

This project is licensed under the FSL-1.1-MIT License. See the LICENSE file for details.