CheckMate |

By @Richardsondx

AI-powered specs verification that block bad code, see through hallucination, and prevent AI from breaking your code.

CheckMate is an AI Test Driven Development tool that challenges AI's overconfidence with a logical reasoning verification process using LLM.

It's built with Cursor AI in mind.

Requirements

CheckMate works best with both API keys:

• OpenAI API Key: Required for GPT models (default quick verifier)
• Anthropic API Key: Required for Claude models (default reason generator)

Quick Setup

# Install globally
npm install -g checkmateai

# Initialize in your project
npx checkmateai init

That's it! checkmate init takes care of everything - creating directories, config files, and setting up Cursor integration.

It will setup your app and generate a checkmate rules folder with all the necessary rules: These rules will include:

• pre-task.mdc - Runs before each task
• post-task.mdc - Runs after each task
• post-push.mdc - Runs after each push
• spec-assistant.mdc - Helps with spec creation and format guidance
• spec-linter.mdc - Automated linting and fixing of spec files
• verification-trigger.mdc - Triggers feature verification workflow
• autofix-enforcer.mdc - Enforces auto-fix attempts on failures
• drift-detector.mdc - Detects spec-vs-code drift
• non-interactive-mode.mdc - For headless CheckMate runs
• ai-feature-validation-guidelines.mdc - Instructional guide for AI validation
• ai-verify-llm-reasoning-workflow-docs.mdc - Documentation for LLM reasoning workflow

After initialization, add your API keys to the auto-generated .checkmate file:

openai_key: sk-****      # Your OpenAI API key 
anthropic_key: sk-ant-**** # Your Anthropic API key

For detailed explanations of what happens during initialization or for manual setup instructions, see the Initialization Process guide.

MCP Integration

CheckMate can be set up as a Middleware Control Protocol (MCP) server for deep Cursor AI integration, enabling a seamless TDD experience directly in your editor:

# Set up the MCP integration automatically
npx checkmate setup-mcp

This creates or updates your .cursor/config.json file with the necessary MCP configuration:

{
  "mcpServers": {
    "checkmate": {
      "command": "node",
      "args": [
        "dist/mcp/index.js"
      ],
      "env": {}
    }
  }
}

The MCP server automatically reads configuration values (including API keys and model preferences) directly from your .checkmate file, eliminating redundant configuration and potential inconsistencies.

With MCP integration:

• Simply describe what you want to build in Cursor ("Build a login form")
• CheckMate automatically generates specs and runs checks
• Cursor implements the code against the generated requirements
• Checks are verified to ensure the implementation matches the specification

For detailed MCP configuration options, see the Cursor Integration Guide.

Configuration

CheckMate uses your local .checkmate file for configuration. This file is never committed to version control (it's automatically added to .gitignore).

While you can run with just one key, the recommended configuration uses both:

# In .checkmate file
openai_key: sk-****      # Your OpenAI API key 
anthropic_key: sk-ant-**** # Your Anthropic API key

CheckMate uses each model for different purposes:

• Claude models (Anthropic): Primarily used for spec generation and detailed reasoning (reason)
• GPT models (OpenAI): Primarily used for quick requirement verification (quick)

This dual-model approach ensures:

1. Higher quality spec generation (Claude excels at reasoning)
2. Faster verification cycles (GPT models are typically quicker)
3. Cost optimization (using the right model for each task)

Additional Configuration Options

# Model selection
models:
  reason: claude-3-7-sonnet-20250219  # For spec generation (complex reasoning)
  quick: gpt-4o-mini                  # For requirement verification (faster)

# File discovery
tree_cmd: "git ls-files | grep -E '\\.(ts|js|tsx|jsx)$'"
context_top_n: 40  # Top relevant files to include

# Behavior settings
protect_specs: true  # Detect spec tampering
auto_fix:
  max_attempts: 5    # Auto-fix attempts before human intervention

# See wiki/Configuration-Guide.md for complete options

For full configuration details, see the Configuration Guide.

Environment Variables

Instead of storing API keys in the .checkmate file, you can use environment variables:

# Add to your .env file or set in your shell
export OPENAI_API_KEY=sk-...
export ANTHROPIC_API_KEY=sk-ant-...

CheckMate will automatically detect and use these environment variables if present.

AI Integration and Token Usage

CheckMate supports both User Specs (Markdown) and Agent Specs (YAML) that work together to validate your code. Learn more about when to use each in the Spec Types Guide.

AI Operations and Token Usage

CheckMate strategically uses AI for specific operations to balance quality and cost:

Operation	Model Used	Token Usage	When It Happens
Spec Generation	reason (claude-3-sonnet)	High	Only when you run gen, draft, or warmup commands
Requirement Verification	quick (gpt-4o-mini)	Medium	During status and verify operations
Clarification	reason (claude-3-sonnet)	Medium-High	Only when explicitly using clarify command

Cost Optimization Features

CheckMate includes several features to minimize API costs:

• Smart Caching: Results are cached in .checkmate/cache/ to avoid redundant API calls
• File Hashing: Only re-verifies requirements when files actually change
• Selective Execution: Commands like affected only check specs related to changed files
• Token Usage Tracking: Run checkmate stats to monitor your API usage and costs

You can also swap models to control costs:

# Use a cheaper model for quick verifications
checkmate model set quick gpt-3.5-turbo

# Use a more powerful model for spec generation
checkmate model set reason claude-3-haiku

---

Core Commands

Command	Description
checkmate warmup	Scan repo, analyze code patterns, and suggest specs
checkmate gen "<sentence>"	Create a regular markdown spec from plain text
checkmate gen "<sentence>" --agent	Create an agent spec (YAML) from plain text
checkmate gen -i "<sentence>"	Interactive spec generation with approval workflow
checkmate gen -i "<sentence>" --agent	Interactive agent spec generation with approval workflow
checkmate draft "<sentence>"	Generate spec drafts as JSON without writing to disk
checkmate save --json '<json>'	Save approved spec drafts to disk
checkmate status --target <spec>	Check spec status, report passes/failures
checkmate next	Run the first unchecked step in the current branch
checkmate affected	Print spec names touched by the current diff
checkmate clarify <slug>	Explain why a requirement is failing and suggest fixes
checkmate watch	Live ASCII dashboard that updates in real-time as specs run
checkmate watch --filter todo	Dashboard filtered to specs containing "todo"
checkmate watch --spec user-auth --until-pass	Watch a specific spec until it passes
checkmate model set quick gpt-4o-mini	Swap the model in the config
checkmate stats	Display token usage and estimated costs
checkmate run-script <script-name> [...args]	Run internal CheckMate utility scripts with proper paths

---

Quick Tips

Need to...?	Try this:
Create a specification	checkmate gen "User can reset their password"
Create an agent spec	checkmate gen "User can reset their password" --agent
Check spec status	checkmate status --target user-password-reset
Focus on a specific spec	checkmate status --target user-password-reset
Monitor progress	checkmate watch (in a separate terminal)
Watch a specific feature	checkmate watch --spec user-auth --until-pass
Fix failing specs first	checkmate watch --status FAIL
See which specs changed	checkmate affected

These commands cover 90% of your daily CheckMate workflow. For more options, use --help with any command.

---

Configuration (.checkmate)

openai_key: sk-****
models:
  reason: o3
  quick:  gpt-4o-mini
tree_cmd: "git ls-files | grep -E '\\.(ts|js|tsx)$'"
log: optional       # on | off | optional

Change model names any time and commit nothing sensitive.

Visual Task Indicators in Cursor

CheckMate provides clear visual indicators when tasks are running in Cursor:

Each task type has a distinct visual style:

Task Type	Visual Indicator	Description
pre_task	🔍 SCOPE ANALYSIS (blue border)	Analyzes which specs will be affected by changes
post_task	✓ VERIFICATION (green border)	Verifies that affected specs pass after changes
post_push	🚀 REGRESSION TEST (red border)	Ensures all specs pass before pushing to main

These visual indicators make it immediately obvious when CheckMate is running tasks in Cursor, providing a seamless and integrated experience.

---

📜 Cursor Rule Files

CheckMate automatically injects several .mdc rule files into your .cursor/rules/checkmate/ directory during initialization. These rules improve Cursor's understanding of the CheckMate workflow and provide clear guidance about when and why commands are being executed.

The rules include:

• pre-task.mdc - Analyzes scope and determines which specs are affected by changes
• post-task.mdc - Verifies affected specs and applies auto-fixes as needed
• post-push.mdc - Runs full test suite on pushes to prevent regressions
• spec-assistant.mdc - Helps with spec creation and format guidance
• spec-linter.mdc - Automated linting and fixing of spec files
• verification-trigger.mdc - Triggers feature verification workflow
• autofix-enforcer.mdc - Enforces auto-fix attempts on failures
• drift-detector.mdc - Detects spec-vs-code drift
• non-interactive-mode.mdc - For CI/CD or headless CheckMate runs
• ai-feature-validation-guidelines.mdc - Instructional guide for AI validation
• ai-verify-llm-reasoning-workflow-docs.mdc - Documentation for LLM reasoning workflow

These rules follow Cursor's best practices with short, specific bullets and clear execution steps. View them in .cursor/rules/checkmate/ to understand how CheckMate integrates with your development workflow.

Reset and Logs

• After a full‑pass run CheckMate rewrites every [🟩] or [🟥] back to [ ].
• History lives in checkmate/logs/run.log unless log: off.

Logs stream nicely into your dashboard or CI summary.

Contributors

Star History

Contributing

PRs welcome. Keep each change small and include a spec that proves it works.

See CONTRIBUTING.md for details.

License

MIT See LICENSE.md for details.