Skip to content

Releases

v1.0.0-beta.4

v1.0.0-beta.4 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 20 Nov 21:05

Highlights

  • MCP load testing (protocol: mcp): drive Model Context Protocol servers (Streamable HTTP, JSON or SSE replies) from resonate run, with one session per virtual user, reported initialize/initialized steps, and result.isError counted as a failure unless expect_tool_error is set. See docs/mcp.md.
  • Fluent builder support for MCP (MCPList, MCPMethod).
  • New example MCP servers: a Go server and an official-Python-SDK server, plus an opt-in interop test (make test-mcp-interop).

Fixes

  • MCP: no more free-spin when tool arguments can't be rendered; clearer truncation errors; JSON media types (e.g. application/problem+json) are handled.
  • Go bumped to 1.27.4 (GO-2026-6859, net/http).

Full Changelog: v1.0.0-beta.3...v1.0.0-beta.4

v1.0.0-beta.3

v1.0.0-beta.3 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 23:17

v1.0.0-beta.2

v1.0.0-beta.2 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 02:46

v1.0.0-beta.1

v1.0.0-beta.1 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 10 Nov 01:31

v1.0.0-alpha.4

v1.0.0-alpha.4 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 06 Nov 19:34

Fourth alpha. Three things for anyone running load tests in CI, and the Go builder now covers every protocol.

Highlights

Per-step results

A flow reported one blended latency distribution, so a slow checkout hid behind a fast login. Every HTTP request is now attributed to its step (its name:, or METHOD url with the query and credentials stripped), and the report breaks the run down per step: success rate, p50/p95/p99, status codes and errors.

http:
  flow:
    - { name: login,    method: POST, url: /login }
    - { name: checkout, method: POST, url: /checkout }
assertions:
  - { metric: latency_p95, step: login, max: 300ms }   # fails as: latency_p95{step=login}: got 412ms, want <= 300ms

Shown in the stdout summary (with 2+ steps), the HTML report (a Steps table) and the JSON report (steps). A step assertion naming a step that doesn't exist is rejected before the run starts, and a step that made no requests fails rather than passing.

Abort on threshold

An assertion with abort: true stops the run as soon as it fails instead of at the end, so a dead server or a blown latency budget doesn't burn the rest of a ten-minute test. The report is still written (marked ABORTED, with aborted in the JSON) and the exit is non-zero.

assertions:
  - { metric: success_rate, min: "0.95", abort: true, abort_after: 10s }

abort_after (default 5s in the CLI) lets the run warm up first. Not allowed on requests/success, which can only be judged at the end.

resonate compare: regression gating

Compare two runs' JSON reports and fail a pipeline on a regression:

resonate compare baseline.json report.json \
  --fail-on 'latency_p95>10%' --fail-on 'rate<-5%' --fail-on 'checkout:latency_p99>50ms'

Rules are relative (%) or absolute, per run or per step. A rule that can't be evaluated (a step missing from either run, an unknown metric) is an error, never a silent pass.

The Go Scenario builder covers every protocol

New: SSE (SSE, Events, StreamDuration) and gRPC (GRPC, Call, Message/Messages, Receive, SendInterval, SendRepeat, Metadata, ExpectCode, Plaintext, Connections, ProtoFiles/ImportPaths/DescriptorSet), plus the HTTP transport options it lacked (FollowRedirects, MaxIdleConns, MaxResponseBody, H2C, DisableKeepAlive, CAFile, ClientCert) and Name. The library guide now has a table of which method applies to which protocol, and worked examples.

Behaviour changes to know about (alpha)

  • JSON reports gain a steps array for HTTP runs (and aborted when applicable), including hit --json.
  • Go API: Assertion gains Step, Abort, AbortAfter; Failure gains Step; Summary gains Steps and Aborted; Options gains AbortOn.
  • Builder misuse is now an error instead of being silently ignored. Message/Wait/Binary outside a WebSocket scenario, mixing protocols in one Scenario, and HTTP-only options on a WebSocket or gRPC scenario all fail at Build(). A few long-standing no-ops (e.g. Extract on a plain Target) are unchanged.
  • Fixed: ExpectStatus/ExpectHeader/ExpectBody on an SSE builder scenario panicked.

Install

Download the archive for your platform below and check it against SHA256SUMS.txt:

# macOS / Linux, for the archive you downloaded (swap in your platform's name):
grep resonate_v1.0.0-alpha.4_darwin_arm64.tar.gz SHA256SUMS.txt | shasum -a 256 -c -

resonate --version reports v1.0.0-alpha.4. See the install guide; the full docs cover everything above.

Full Changelog: v1.0.0-alpha.3...v1.0.0-alpha.4

v1.0.0-alpha.3

v1.0.0-alpha.3 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 05 Nov 03:03

v1.0.0-alpha.2

v1.0.0-alpha.2 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 04 Nov 17:39

v1.0.0-alpha.2

resonate now load-tests gRPC, Server-Sent Events and HTTP/3, and can feed requests from a Kafka topic. The HTML report is more useful too. It is still alpha: flags, the YAML schema and the Go API may change without notice.

Highlights

gRPC (protocol: grpc)

  • Unary, server-streaming, client-streaming and bidirectional-streaming calls. The kind is read from the method, so you don't declare it.
  • The schema comes from server reflection, .proto files, or a compiled descriptor set. No code generation.
  • JSON messages with templating, metadata, response chaining (extract), and expect_code / expect_body checks.
  • connections for spreading load over several HTTP/2 connections.
  • A runnable demo server (make example-grpc-server) and a full guide: docs/grpc.md.

Server-Sent Events (protocol: sse)

  • Open a stream per iteration, then end it after N events or a duration. Reports time to headers and the gap between events.
  • POST bodies for streaming APIs (such as LLM token streams), identities, and streams the server closes itself.
  • A demo server (make example-sse-server) and a guide: docs/sse.md.

HTTP/3 (QUIC)

  • --http3 on resonate hit, http3: true in scenarios (HTTP and SSE), and Scenario.HTTP3() in the Go library.
  • Opt-in with no fallback to TCP, so an HTTP/3 test can't quietly measure HTTP/1.1.
  • A demo server that serves the same URL over HTTP/3 and TCP (make example-http3-server) and a guide: docs/http3.md.

Feeders

  • Kafka feeder: each message of a topic becomes a row (feeder.kafka), under every protocol and in the Go library (resonate.NewKafkaFeeder). It reads partitions directly with no consumer group and never commits offsets, keeps memory bounded however large the topic is, and wraps around when it runs out (or waits for new messages with on_end: wait). Supports TLS and SASL (PLAIN, SCRAM).
  • New Feeders guide (docs/feeders.md) covering CSV/JSON formats, the three modes, sizing, when a row is picked per protocol, and the Go API. The detail for Kafka is in docs/kafka-feeder.md.

Reports

  • p75 added next to p50/p90/p95/p99 (text, HTML and JSON output, and latency_p75 assertions).
  • The Requests over time chart now has a grid, a time axis, and hover tooltips showing each bucket's time, requests, rate and mean latency. It is still script-free.
  • The HTML report links to the project on GitHub.
  • Latency statistics now come from an HDR histogram, so memory is fixed whatever the request count. Percentiles carry at most 0.1% quantization error, and latencies above 1 hour are clamped.

Changes to be aware of

  • Stream feeder validation. mode: stream now scans the whole file once at startup and fails with the row number on a malformed row. Previously it checked only the header and first row, and a bad row later in the file silently restarted the file, so the rows after it were never used. Startup takes longer in proportion to the file (an 83 MB, 2-million-row CSV took under a second).
  • JSON report has a new p75 latency field.
  • Binary size. The Kafka client added about 7 MB (a macOS arm64 build went from about 29.6 MB to about 37 MB).
  • New dependencies: grpc-go, protobuf and protocompile, quic-go, an HDR histogram library, and franz-go. All are pure Go, so release builds stay CGO_ENABLED=0.

Good to know

  • A run paced with --rate starts with a full token bucket, so --rate 50 --duration 30s sends about 1,550 requests, not 1,500. Documented in Execution Models; behaviour unchanged.
  • The default Go transport speaks HTTP/1.1 once custom TLS is configured, so without --http3 a TLS run shows up as HTTP/1.1 on the server.
  • resonate hit is still HTTP-only. gRPC, SSE, WebSocket and feeders are scenario-file and library features. gRPC and SSE are available in the library through their struct APIs, not the fluent Scenario builder.

Known limitations

  • Kafka TLS and SASL are untested against a real secured cluster. The options are validated and wired in, but the feeder has been exercised only against an in-process broker and a plain local broker.
  • gRPC and HTTP/3 have had limited real-world testing. Both have been tested mainly against the bundled example servers.
  • HTTP/3 uses one QUIC connection per host, so concurrency is bounded by the server's per-connection stream limit (commonly about 100). There is no connections setting as there is for gRPC.
  • Plain HTTP and flow requests can still report a few context deadline exceeded errors as a run ends.
  • Streams (WebSocket, SSE, gRPC streaming) are reported by message, and their results are held until the stream ends, so memory grows with messages per stream.

Contributors

Thanks to @garyjyao for improved contributor setup and CLI guidance (#1), and the JSON Schema and run command help (#2).

Full changelog: v1.0.0-alpha.1...v1.0.0-alpha.2

v1.0.0-alpha.1

v1.0.0-alpha.1 Pre-release
Pre-release

Choose a tag to compare

@github-actions github-actions released this 24 Sep 07:25