spec

spec turns human-written or LLM-written Markdown into executable browser-style test plans in a Bun-first TypeScript codebase. It compiles specs into a strict runtime format, runs them through the Bun CLI flow, and saves compiled plans plus result artifacts under .spec/.

Quick Start

Install dependencies:

bun install

Set project defaults in .spec/spec.toml:

[spec]
base_url = "http://localhost:3000"
results_dir = ".spec/results"

[spec.browser]
browser = "chromium"
viewport = "desktop"
locale = "en-US"

Launch it from Bun:

bun run spec

Example

Create a markdown spec at tests/specs/login.md:

# Suite: Login Flow

## Test: User can open the home page

### Steps
1. Navigate to /
2. Wait for text "Welcome"

### Expect
- URL should contain /
- Text "Welcome" should be visible

Run it from the TUI:

bun run spec

Or run it directly:

bun run spec run tests/specs/login.md

That produces:

• .spec/compiled/login.json
• .spec/results/Login-Flow/result.json
• .spec/results/Login-Flow/report.md
• .spec/results/Login-Flow/report.html

Authoring Modes

spec always uses auto mode by default.

• It tries fixed grammar first for lower token cost and more deterministic parsing.
• If a test does not look structured, it falls back to freeflow mode and lets the LLM extract steps and expectations.

For Devs

Use fixed grammar when you want predictable parsing, cleaner diffs, and lower token cost.

• best for CI-critical tests
• best when you want exact Steps and Expect sections
• best when engineers are maintaining test suites long-term

Fixed grammar example:

# Suite: Checkout

## Test: Buyer completes checkout

### Steps
1. Navigate to /
2. Click the "Buy now" button
3. Wait for text "Checkout"

### Expect
- URL should contain /checkout
- Text "Checkout" should be visible

Why this is recommended for devs:

• spec can usually parse it without spending extra LLM tokens on outline extraction
• the resulting action/expectation graph is more stable
• it is easier to review in pull requests

For Non-Devs

Use freeflow when you want to describe intent in plain language and let spec turn it into executable steps.

• best for PMs, QA, founders, designers, and support teams
• best when speed of authoring matters more than rigid structure
• best for capturing a behavior before refining it into fixed grammar later

Freeflow example:

# Suite: Checkout

## Test: Buyer completes checkout

Open the storefront, click buy now, continue to checkout, and confirm the checkout page is visible.

How it works for non-dev authors:

• write the test like a short scenario
• spec detects that it is not fixed grammar
• the LLM extracts steps and expectations before execution

Recommendation:

• Use fixed grammar for CI-critical and frequently edited tests.
• Use freeflow when non-devs or PMs need to author tests quickly.
• Let auto mode handle both in the same repo.

Best Practice

Start broad, then tighten over time.

• non-devs can begin with freeflow to capture behavior quickly
• devs can later convert important flows into fixed grammar for stronger determinism
• both styles can live in the same repository because auto mode handles them together

Workflow

• bun run spec opens an interactive picker for discovered markdown specs.
• bun run spec init creates .spec/spec.toml and .spec/results/.
• bun run spec compile tests/specs/example.md writes .spec/compiled/<name>.json.
• bun run spec run tests/specs/example.md writes suite artifacts to .spec/results/<suite-id>/.
• bun run spec report .spec/results/<suite-id>/result.json --format markdown regenerates reports.

Inside the interactive runner you can:

• choose a markdown file
• run Compile + Run
• run Compile Only

Output Layout

• .spec/compiled/<spec-name>.json
• .spec/results/<suite-id>/result.json
• .spec/results/<suite-id>/report.md
• .spec/results/<suite-id>/report.html
• .spec/results/<suite-id>/summary.json

Testing

bun test