Skip to content
BranchesTags

Repository files navigation

Khufu

A test generation skill framework for AI IDEs, following the test pyramid methodology with Evaluation-Driven Development (EDD) and Specification-Driven Development (SDD).

Khufu provides four skills — /khufu-ut, /khufu-it, /khufu-api, /khufu-e2e — that leverage AI agents to generate unit tests, integration tests, API tests, and end-to-end tests for your projects.

Features

  • Test Pyramid: Complete coverage across all four test layers with built-in deduplication — higher layers only test what lower layers structurally cannot
  • Evaluation-Driven Development (EDD): Define quantifiable quality criteria (coverage, performance, correctness) BEFORE generating tests. Every test serves a measurable evaluation standard. Built-in Evaluate → Verify → Improve feedback loop
  • Specification-Driven Development (SDD): Test cases are explicit, versioned Markdown spec files (specs/<feature>/spec.v<N>.md). Supports forward generation (Spec → Tests) and reverse generation (Tests → Spec). Each test links back to its spec entry
  • Two Generation Paths:
    • Spec-based: Generate tests from specification documents (/khufu-ut @docs/spec.md)
    • Code-based: Generate tests by reading existing implementations (auto-generates retroactive specs)
  • Language-Aware: Auto-detects your project language and suggests appropriate test frameworks
  • Testability Assessment (UT): Grades source files A/B/C — directly testable, needs PowerMock (marked fragile), or requires refactoring (recorded to untestable-report.md)
  • H2 Database Support (IT): Lightweight in-memory database option for Java integration tests — zero Docker dependency, <5s startup for fast CI feedback
  • Shared Test Data Factories: Define test data once, reuse across IT, API, and E2E layers
  • TDD-Aligned: Tests follow Test-Driven Development and Agile best practices
  • Claude Code Native: Installs as a Claude Code plugin with slash commands

Quick Start

# Install globally
npm install -g khufu-kit

# Initialize in your project
cd your-project
khufu init

# Use the skills in Claude Code
/khufu-ut          # Generate unit tests
/khufu-it          # Generate integration tests
/khufu-api         # Generate API tests
/khufu-e2e         # Generate E2E tests

Supported Languages (v0.2)

Language UT Frameworks IT Frameworks API Frameworks E2E Frameworks
Java JUnit 5/4, TestNG Spring Boot Test, Testcontainers, H2 REST Assured, MockMvc Playwright, Selenium
Python pytest, unittest pytest + testcontainers pytest + httpx Playwright
TypeScript/JS Jest, Vitest, Mocha Jest + Supertest Supertest Playwright, Cypress
Go testing + testify testing + dockertest testing + httptest Playwright
C# xUnit, NUnit, MSTest xUnit + Testcontainers RestSharp Playwright

Test Pyramid Strategy

Khufu enforces a non-overlapping test pyramid. Each layer owns a distinct risk dimension:

Layer Owns Does NOT Own
UT Business logic, algorithms, validation, state transitions DB queries, HTTP routing, UI
IT Component wiring, transaction boundaries, ORM mappings, DB constraints Business logic details, HTTP status codes
API Status codes, response schemas, headers, auth/authz, content negotiation Component internals, DB state
E2E Critical user journeys, multi-page workflows, browser interactions Edge-case validation, exhaustive error paths

When generating tests at a layer, khufu scans lower-layer tests and skips scenarios that add no new risk — keeping your suite lean and fast.

Spec System (SDD)

Khufu treats test specifications as first-class artifacts. Each feature gets a versioned Markdown spec:

specs/
├── UserRegistration/
│   ├── spec.v1.md        # Versioned specification
│   ├── spec.v2.md
│   └── CHANGELOG.md      # Spec change history

Each spec file contains:

# Feature: UserRegistration
- Version: 1 | Created: 2026-06-25 | Source: spec-based | Criticality: High

## Evaluation Criteria          ← EDD: defined BEFORE tests
### Correctness（正确性）
- [ ] Normal input returns User with ID
- [ ] Duplicate email throws DuplicateEmailException

## Unit Tests
### UT-USER-001: shouldCreateUser_whenValidInputGiven
- Category: happy-path | Status: implemented
- Test File: `__tests__/UserService.test.ts`
- Evaluates: Correctness — normal input returns User
**Scenario:** Given ... When ... Then ...

## Evaluation Summary           ← EDD: filled AFTER test execution
| Criteria | Target | Actual | Status |
|----------|--------|--------|--------|
| Line coverage | ≥85% | 88% | ✅ |
| Branch coverage | ≥75% | 72% | ❌ |

Spec lifecycle: planned → implemented → evaluated → passed (failed criteria trigger re-evaluation).

Fragile Test Handling

For tightly-coupled code that resists standard mocking, khufu-ut offers:

  • PowerMock / Mockito-inline support for Java (static methods, constructors, final classes)
  • All such tests are tagged @Tag("fragile") and excluded from CI by default
  • Fragile tests are tracked as technical debt — target ≤20% of test suite
  • Untestable code (Grade C) is recorded to untestable-report.md with refactoring recommendations

Configuration

After khufu init, a khufu.yaml file is created in your project root:

ide: claude
language: typescript
frameworks:
  ut: jest
  it: jest
  api: supertest
  e2e: playwright
directories:
  ut: __tests__/
  it: __tests__/integration/
  api: __tests__/api/
  e2e: e2e/
coverage:
  ut:
    tool: jest
    thresholds: { line: 80, branch: 70, function: 80 }
    formats: [lcov, html, text]
    excludes: ['**/*.d.ts', '**/index.ts', '**/*.config.*']
  it:
    tool: jest
    thresholds: { line: 60, branch: 50 }
    formats: [lcov, html]

# Fragile test management (v0.2)
categories:
  fragile:
    excludeFromCI: true
    runOnSchedule: weekly

# Spec system (v0.2)
spec:
  enabled: true
  directory: specs/
  format: markdown
  versioning: file-based
  autoGenerateFromTests: true

# Evaluation-Driven Development (v0.2)
evaluation:
  enabled: true
  failOnUnmetCriteria: warning
  dimensions: [correctness, coverage, performance, maintainability]
  thresholds:
    ut: { lineCoverage: 80, branchCoverage: 70, maxTestMs: 100, maxFragileRatio: 20 }
    it: { lineCoverage: 60, branchCoverage: 50, maxTestMsH2: 2000, maxTestMsPostgres: 10000 }

# Shared test data (v0.2)
dataFactories:
  directory: tests/factories/
  reuseAcrossLayers: true

scope: project

Architecture

skills/
├── _shared/                       # Shared modules (v0.2)
│   ├── mode-selection.md          # Spec-based vs code-based mode
│   ├── language-detection.md      # Language & framework detection
│   ├── framework-selector.md      # Framework routing by language × layer
│   ├── directory-resolver.md      # Test directory resolution
│   ├── coverage-config.md         # Coverage tool configuration
│   ├── report-template.md         # Unified report generation
│   ├── test-data-factories.md     # Shared data factory patterns
│   ├── test-pyramid-strategy.md   # Layer deduplication rules
│   └── evaluation-framework.md    # EDD methodology
├── khufu-ut/SKILL.md              # Unit test generator
├── khufu-it/SKILL.md              # Integration test generator
├── khufu-api/SKILL.md             # API test generator
└── khufu-e2e/SKILL.md             # E2E test generator

Skills reference _shared/ modules via Read-tool delegation — shared logic is defined once and consumed by all four skills.

Development

Working on khufu itself

A single command symlinks the skills into place so Claude Code loads them directly from your working tree:

git clone https://github.com/agiledon/khufu.git
cd khufu
npm install
npm run build
npm run setup    # creates symlinks for local development

What setup does:

  • Symlinks skills/_shared/ → _shared/ at the project root (so Read _shared/... references in skill files resolve correctly)
  • Symlinks each skills/khufu-*/ → .claude/skills/khufu-*/ (Claude Code loads skills from .claude/skills/)
  • Replaces any stale copies from a prior khufu init

After setup, edit any file under skills/ and changes are live immediately — no rebuild, no copy step.

npm test         # run khufu's own tests
npm run dev      # watch mode (tsc --watch)

Installing from local source into another project

Use npm link to register the local khufu CLI globally, then initialize it in your target project:

# Step 1: Build khufu-kit from source
cd /path/to/khufu-kit
npm install
npm run build
npm link                    # registers 'khufu' command globally

# Step 2: Install into your project
cd /path/to/your-project
khufu init                  # creates khufu.yaml, installs skills into .claude/skills/

To pick up changes after editing khufu-kit's source:

cd /path/to/khufu-kit
npm run build               # recompile TypeScript
# The 'khufu' command now uses the updated build automatically (npm link)

To undo the global link when done:

cd /path/to/khufu-kit
npm unlink -g khufu-kit

Alternatively, use npm install with a local path (no global link):

cd /path/to/your-project
npm install /path/to/khufu-kit
npx khufu init

License

MIT

About

A test generation skill framework for AI IDEs, following the test pyramid methodology with Evaluation-Driven Development (EDD) and Specification-Driven Development (SDD).

Resources

Readme
Activity

Stars

5 stars

Watchers

1 watching

Forks

0 forks
Report repository

Releases

Packages

Contributors

Languages