spectest

Validate running APIs against OpenAPI/Swagger specs. Auto-generates requests for every endpoint, sends them to your server, and verifies response schemas match the spec.

Why This Exists

You wrote an OpenAPI spec. Your API server is running. But are they actually in sync?

API specs and implementations drift apart constantly. Endpoints return extra fields, miss required ones, use wrong types, or return undocumented status codes. Schema-first tools validate specs in isolation. Runtime testing tools need you to hand-write every test case.

spectest closes the gap. Point it at your spec and your running server. It auto-generates valid requests from the spec, fires them at the server, and validates every response against the declared schema. Zero test authoring. Catches real production drift.

$ spectest run api.yaml --base-url http://localhost:3000

spectest - API Spec Compliance Testing
-----------------------------------------
  Spec:      My API v2.0.0
  Base URL:  http://localhost:3000
  Endpoints: 12
-----------------------------------------

  PASS  GET     /users                 (getUsers)           23ms
  PASS  POST    /users                 (createUser)         31ms
  FAIL  GET     /users/{id}            (getUser)            15ms
        [SCHEMA] Missing required property 'email'
          at /email
  PASS  DELETE  /users/{id}            (deleteUser)         8ms
  PASS  GET     /health                (healthCheck)        5ms

--- Results ---
  12 endpoints tested: 11 passed, 1 failed, 0 errors
  Duration: 0.4s

Quick Start

# Install
npm install -g spectest

# Validate spec syntax (no server needed)
spectest validate openapi.yaml

# Test all endpoints against a running server
spectest run openapi.yaml --base-url http://localhost:3000

# Test specific endpoints
spectest run api.yaml --base-url http://localhost:3000 --filter "GET /users"

# Generate JSON report for CI
spectest run api.yaml --base-url http://localhost:3000 -o report.json

# With authentication
spectest run api.yaml --base-url http://localhost:3000 --bearer "my-jwt-token"
spectest run api.yaml --base-url http://localhost:3000 --api-key "sk-abc123"
spectest run api.yaml --base-url http://localhost:3000 --basic-user admin --basic-pass secret

Features

Spec Support

• OpenAPI 3.0 / 3.1 (YAML and JSON)
• Swagger 2.0 (YAML and JSON)
• $ref resolution (components, definitions, nested refs)
• allOf / oneOf / anyOf composition
• Path-level and operation-level parameter merging

Request Generation

• Auto-generates values from example, default, enum, or type-based smart defaults
• Handles path, query, header, cookie, and body parameters
• Generates format-aware values (email, UUID, date-time, URI, IPv4, etc.)
• Skips readOnly properties in request bodies
• Swagger 2.0 body and formData parameter support

Response Validation

• JSON Schema validation via ajv with format checking
• Status code verification against spec-declared responses
• Content-Type matching (OpenAPI 3.x content map, Swagger 2.0 produces)
• Response header presence checking
• Nullable field support (OpenAPI 3.0 nullable keyword)

CLI

• spectest run - test endpoints against a live server
• spectest validate - validate spec syntax without running tests
• spectest report - run tests and generate JSON report
• Filter by method/path (--filter "GET /users"), tags (--tag users), wildcards (--filter "/api*")
• Exclude deprecated endpoints (--exclude-deprecated)
• Auth: Bearer, API key (header or query), Basic
• Custom headers (-H "X-Custom: value")
• Configurable timeout (--timeout 5000)
• Verbose mode with full request/response details (--verbose)
• Dry-run mode to preview generated requests (--dry-run)

Exit Codes

Code	Meaning
0	All endpoints passed
1	One or more endpoints failed validation
2	Error (network, timeout, invalid spec, etc.)

Programmatic API

import {
  loadSpec,
  extractEndpoints,
  testAllEndpoints,
  filterEndpoints,
  validateSpec,
  buildReport,
  serializeReport,
} from 'spectest';

// Load and validate spec
const spec = await loadSpec('api.yaml');
const validation = validateSpec(spec);

if (!validation.valid) {
  console.error('Spec errors:', validation.errors);
  process.exit(1);
}

// Extract and filter endpoints
const endpoints = extractEndpoints(spec);
const filtered = filterEndpoints(endpoints, {
  filter: 'GET',
  excludeDeprecated: true,
});

// Run tests
const results = await testAllEndpoints(spec, filtered, {
  specPath: 'api.yaml',
  baseUrl: 'http://localhost:3000',
  auth: { type: 'bearer', token: 'my-token' },
  timeout: 10000,
  verbose: false,
  dryRun: false,
  headers: {},
  excludeDeprecated: true,
});

// Generate report
const report = buildReport(
  spec.info.title,
  spec.info.version,
  'http://localhost:3000',
  results,
  1000,
);
console.log(serializeReport(report));

Comparison

Feature	spectest	Schemathesis	Dredd
Language	TypeScript/Node	Python	Node/JS
OpenAPI 3.0/3.1	Yes	Yes	Partial
Swagger 2.0	Yes	Yes	Yes
Auto-generate requests	Yes (example/default/smart)	Yes (property-based)	Yes (from examples)
Schema validation	ajv (JSON Schema)	jsonschema	Gavel
Auth support	Bearer, API Key, Basic	Yes	Yes
CI-friendly JSON report	Yes	Yes	Yes (hooks)
Install	npm install	pip install	npm install
Zero config	Yes	Yes	Needs hooks for auth
Filter by method/path	Yes	Yes	No
Filter by tag	Yes	No	No
Dry-run mode	Yes	No	Yes
No external deps	Node 18+ only	Needs Python	Needs Node
Property-based testing	No	Yes (hypothesis)	No

When to use spectest: You want a fast, zero-config Node.js tool that validates your API against its spec in CI. Install, point at spec, get results.

When to use Schemathesis: You want property-based testing to find edge cases with randomized inputs. More thorough but requires Python.

When to use Dredd: You have a mature spec-first workflow with custom hooks. Established but heavier setup.

JSON Report Format

{
  "specTitle": "My API",
  "specVersion": "2.0.0",
  "baseUrl": "http://localhost:3000",
  "timestamp": "2026-04-03T12:00:00.000Z",
  "duration": 1234,
  "summary": {
    "total": 12,
    "passed": 11,
    "failed": 1,
    "errors": 0
  },
  "results": [
    {
      "endpoint": { "method": "GET", "path": "/users", "operationId": "getUsers" },
      "request": { "method": "GET", "url": "http://localhost:3000/users", "headers": {} },
      "response": { "status": 200, "statusText": "OK", "headers": {}, "body": [...] },
      "passed": true,
      "errors": [],
      "duration": 23
    }
  ]
}

Architecture

src/
  cli.ts             CLI entry point (commander)
  parser.ts          OpenAPI/Swagger spec loading and parsing
  generator.ts       Auto-generate HTTP requests from endpoints
  executor.ts        Execute requests and capture responses
  validator.ts       Validate responses against schemas (ajv)
  filter.ts          Endpoint filtering (method, path, tags, deprecated)
  spec-validator.ts  Spec syntax validation
  reporter.ts        CLI output formatting and JSON report generation
  types.ts           TypeScript type definitions
  index.ts           Public API exports

License

MIT