k6gen — AI-Assisted k6 Script Generator

Turn an OpenAPI spec into a runnable, opinionated k6 load test in one command. Deterministic templates produce the correct skeleton; an optional LLM pass adds the parts templates can't infer.

pip install -e .
k6gen generate examples/petstore-mini.yaml -o loadtest.js --rps 20
k6 run loadtest.js

Design principle: AI refines, it doesn't rescue

Most "AI test generators" hand the whole job to an LLM and hope. k6gen inverts that:

1. Deterministic core (no API key needed): parses the spec, resolves $ref schemas, builds example payloads, and emits a script that already encodes performance-engineering opinion —
   • open workload model (ramping-arrival-rate), not VU loops, to avoid coordinated omission
   • per-endpoint tags + per-endpoint p95 thresholds, so one slow route can't hide inside a global average
   • explicit TODO markers where human judgment is required (auth, traffic weighting, data correlation)
2. --ai refinement (Anthropic API): replaces placeholder data with varied realistic data, correlates CRUD flows (capture created ids, reuse them), resolves TODOs the spec answers — while structurally constrained to keep the scenario, tags, and thresholds intact. Output is sanity-checked before it's accepted; on any failure the deterministic script is kept.

This means the tool degrades gracefully: no key, no network, no problem — you still get a correct script.

Usage

k6gen generate api.yaml -o loadtest.js                # deterministic REST
k6gen generate api.yaml -o loadtest.js --ai           # + LLM refinement
k6gen generate api.yaml --base-url https://staging.example.com --rps 50 --p95 300

# --stack llm: generate an LLM-endpoint test instead of REST
k6gen generate --stack llm --model llama3.2:3b -o llm.js              # tokens/sec + e2e
k6gen generate --stack llm --model mock --streaming -o ttft.js       # TTFT/ITL (needs xk6-sse)

# Calibrate thresholds from a baseline run instead of guessing
k6 run --summary-export=base.json loadtest.js                        # capture a baseline
k6gen generate api.yaml --baseline base.json -o tuned.js            # SLOs from observed p95/p99

# Weight the endpoint mix by real traffic (HAR capture from a browser/proxy)
k6gen generate api.yaml --har prod-traffic.har -o weighted.js       # picks endpoints by frequency

Flag	Default	Meaning
--stack	rest	rest (OpenAPI → REST test) or llm (TTFT / tokens-per-sec test)
--base-url	first servers[].url (or http://localhost:11434 for llm)	Target host
--rps	10	Peak arrival rate of the ramp
--p95	500	p95 threshold (ms), REST mode (ignored when --baseline is given)
--baseline	—	k6 summary JSON to derive per-endpoint/global thresholds from (REST)
--margin	1.2	Safety margin applied to baseline percentiles
--har	—	HAR capture; weight the endpoint mix by observed traffic (REST)
--model	llama3.2:3b	Model id (--stack llm)
--max-tokens	128	max_tokens per request (--stack llm)
--streaming	off	--stack llm: emit the SSE TTFT variant (requires an xk6-sse build)
--ai	off	REST LLM refinement (ANTHROPIC_API_KEY required)

--stack llm

The LLM generator reuses the engineering opinions of the llm-inference-load-testing repo — open workload model, decode tokens/sec, and (with --streaming) TTFT and inter-token latency — and is fully deterministic (no spec or API key required; pass a spec only to pick up its servers[].url and title).

CI: PR regression gate

k6gen compare diffs a run's summary against a committed baseline and exits non-zero on regression, so it doubles as a CI gate:

k6 run --summary-export=current.json loadtest.js
k6gen compare --baseline baseline.json --current current.json --margin 1.2
# exit 0 if every metric is within baseline x margin, else exit 1

This repo also ships a composite GitHub Action (action.yml) that runs the comparison and comments the pass/fail table on the PR, failing the check on regression. A ready-to-copy consumer workflow is in examples/k6gen-pr-check.yml:

- uses: vsapiens/k6gen@main
  with:
    baseline: baseline.json
    current: current.json
    margin: "1.2"

What generated output looks like

See examples/. Every script: env-overridable BASE_URL, ramp scenario sized from --rps, per-endpoint error Rate metric, grouped requests with status checks against the spec's declared success codes.

Roadmap

• --stack llm mode: emit LLM-endpoint tests (TTFT/tokens-per-sec) instead of REST
• Threshold suggestions from a baseline run (read k6 summary JSON, propose SLOs)
• Traffic weighting from a HAR file or access-log sample
• GitHub Action wrapper: PR comment with pass/fail vs baseline

License

MIT