Skip to content

MockServer 7.0.0

Choose a tag to compare

View all tags
@jamesdbloom jamesdbloom released this 06 Jun 18:51
mockserver-7.0.0
9755e9c

[7.0.0] - 2026-06-06

This cycle centres on first-class LLM / AI-agent mocking and a major platform modernisation, alongside broader resilience-testing and dashboard improvements. Highlights (see the per-item entries below for detail):

  • HTTP/3 streaming responses — SSE, chunked proxy forwarding, and LLM streaming are now fully supported over HTTP/3 (QUIC). Each body chunk is sent as an HTTP/3 DATA frame with backpressure via StreamingBody.requestMore(); the QUIC stream is cleanly shut down on completion or error. Bundled native QUIC removes the need for a separately downloaded BoringSSL library.
  • TPROXY (IP_TRANSPARENT) transparent proxy — a new default-off transparentProxyTproxy configuration property enables IP_TRANSPARENT socket binding so that with iptables TPROXY rules the kernel preserves the original destination as the listening socket's local address, which MockServer reads via channel.localAddress() — avoiding the conntrack SO_ORIGINAL_DST lookup used with REDIRECT rules. Requires Linux, epoll transport, and CAP_NET_ADMIN. Verified end-to-end with a real Docker NET_ADMIN integration test.
  • Testcontainers 1.21.4 — upgrades from 1.20.6, fixing DockerClientFactory.isDockerAvailable() returning false on Docker Desktop 4.67 / Engine API 1.54 (docker-java 3.4.2 probe fix).
  • Clustered MockServer state (opt-in) — a new mockserver-state-infinispan module provides an embedded Infinispan StateBackend that can replicate expectations and scenario state across a JGroups cluster. Single-node behaviour is completely unchanged (the in-memory StateBackend remains the default). New configuration properties: stateBackend, clusterEnabled, clusterName, clusterTransportConfig, blobStoreType.
  • LLM / AI-agent mocking suite — provider-correct mock completions and streaming for seven providers (Anthropic, OpenAI, OpenAI Responses, Azure OpenAI, Gemini, Bedrock, Ollama), with embeddings for OpenAI and Azure OpenAI; multi-turn scripted conversations with per-session isolation and deterministic prompt normalisation; and a runtime-LLM client SPI (off unless configured, fails closed) that powers the opt-in features. A broad MCP toolset drives it from an agent: mock_llm_completion, create_llm_conversation, verify_tool_call, explain_agent_run (with a correlated call graph), verify_structured_output, verify_cost_budget, detect_llm_drift, mock_adversarial_llm_response, and run_mcp_contract_test.
  • Agent resilience & correctness testing — structured-output (JSON-Schema) validation on both the response path (outputSchema, fail-soft) and the verification path (verify_structured_output); a deterministic CI cost-budget gate (verify_cost_budget) over a built-in pricing table; declarative LLM fault/chaos profiles (probabilistic provider errors, mid-stream truncation, malformed SSE) plus a stateful request-quota rate limit; VCR record/replay with strict mode and body/header redaction; a prompt-injection / adversarial-response harness; and OpenTelemetry GenAI span + metrics export. The dashboard surfaces all of it (conversation wizard, sessions & call-graph, metrics view, export).
  • HTTP chaos/fault injection — a general HttpChaosProfile (probabilistic error status + latency) attachable to any mocked or forwarded response, making MockServer usable as a chaos proxy for unreliable upstreams.
  • Platform modernisation (breaking) — minimum runtime raised to Java 17; full Jakarta EE 10 / Servlet 6 migration (Spring 7 / Boot 4, Tomcat 11, Jetty 12, Jersey 4, Netty 4.2); json-schema-validator 3.x; a bundled DataFaker template helper; and ZGC tuning guidance.

Security

  • Released Docker images are now cosign-signed by digest (Docker Hub and ECR Public), using the same signing key infrastructure as the Helm OCI chart. Consumers can verify image provenance with cosign verify. Signing is non-fatal in the pipeline if the key is unavailable, so it never blocks a release.
  • Website security hardening — the documentation site (mock-server.com) now sends Strict-Transport-Security, Content-Security-Policy, X-Content-Type-Options, X-Frame-Options, and Referrer-Policy response headers via CloudFront, and the domain publishes CAA records pinning certificate issuance to Amazon.
  • Build/release infrastructure hardening (internal) — least-privilege scoping of CI secrets per Buildkite agent queue, removal of release-only permissions (ECR push) from the PR-build queue, secrets passed to release containers via 0600 files instead of docker -e environment variables, robust git-push-token cleanup, scoped cross-account AssumeRole (ExternalId) and tfstate IAM, full VPC flow logging, GuardDuty→SNS alerting, CloudTrail data-events on secrets/state, and SSE-KMS on the state and AWS Config buckets. See docs/infrastructure/aws-infrastructure.md, docs/infrastructure/ci-cd.md, and docs/operations/website.md.

Added

  • Added a daily performance-regression pipeline (notify-only) that guards response latency, throughput, and CPU/memory against drift across releases. It runs on a dedicated, pinned, on-demand, scale-to-zero Buildkite perf queue and fires once per day only when master moved since the last run. Each run measures four behaviours (mock match, forward/proxy, Velocity template, large-body) over HTTP and HTTPS/HTTP-2 (k6/regression.js), a sustained resource-growth run that surfaces "increases over time" regressions such as the issue #2329 O(n) log-eviction CPU climb (k6/growth.js, CPU/heap/latency slope ratios), and the JMH MatchingBenchmark allocation backstop. Results are persisted to S3 and each run is compared against a rolling median+MAD baseline of recent runs, posting a Buildkite annotation table when a metric regresses. See docs/operations/performance-tuning.md.

LLM & AI-agent mocking

  • Added a dedicated retrieve_logs MCP tool so an AI assistant debugging a failing test can pull MockServer's recorded log messages (request matching, mismatches, actions and errors) directly. It is a thin, discoverable wrapper over the existing LOGS retrieval path (shared with raw_retrieve), with an optional correlationId filter (trace one request's full lifecycle) and a limit (most-recent N, default 100, max 500). This fills the gap left by its sibling tools retrieve_recorded_requests / retrieve_request_responses, which already existed. See the AI/MCP tools page.
  • Added a runtime-LLM client SPI (org.mockserver.llm.client) that lets MockServer call a real LLM you already run, as the foundation for opt-in features such as drift detection and exploratory semantic matching. Mirrors the existing codec registry: an LlmClient per provider (Ollama, OpenAI, OpenAI Responses, Azure OpenAI, Anthropic, Gemini, Bedrock) registered in LlmClientRegistry, an immutable LlmBackend config (with the API key redacted in logs), and a three-layer LlmBackendResolver (provider env vars → mockserver.llmProvider/llmApiKey/llmModel/llmBaseUrl → named-backends JSON via mockserver.llmBackendsConfig). All runtime-LLM use goes through LlmCompletionService, which is off unless a backend is configured, fails closed on any timeout/error/non-2xx (never flipping a deterministic result), and caches per normalised prompt for reproducibility. Ollama is the reference backend (no key, local); Bedrock builds the Anthropic-on-Bedrock request and relies on the headers escape hatch pending automatic SigV4 signing. See the configuration properties page and docs/code/llm-mocking.md.
  • LLM conversation mocks can now opt into deterministic prompt normalisation before the latestMessageContains / latestMessageMatches predicates are evaluated, so a match is not blocked by cosmetic differences in dynamically-assembled agent prompts. A new normalization block on conversationPredicates (also exposed per-turn in the create_llm_conversation MCP tool and the dashboard conversation wizard) supports collapsing whitespace, lowercasing, sorting JSON object keys, dropping built-in volatile values (ISO-8601 timestamps, UUIDs, req_/msg_/call_ ids), and dropping named JSON fields. Normalisation is pure and idempotent — it never makes a test flaky — and has no effect unless a text predicate is set. See the AI/MCP tools page and docs/code/llm-mocking.md.
  • Added two MCP tools for agent-run analysis and tool-call assertions, both backed by a new deterministic org.mockserver.llm.analysis.AgentRunAnalyzer that reconstructs an agent run by decoding the LLM requests MockServer recorded. verify_tool_call asserts that an agent called a named tool a given number of times (atLeast/atMost, with an optional regex over the tool-call arguments); explain_agent_run summarises the run's structure (message and assistant-turn counts, the ordered tool-call sequence, tool results, and the latest message role). Read-only and offline — no LLM call. See the AI/MCP tools page and docs/code/llm-mocking.md.
  • Added a correlated agent-run call graph. AgentRunAnalyzer.buildCallGraph reconstructs a recorded run as a graph — a node per message and per assistant tool call, with NEXT (sequence), INVOKES (turn→tool call), and RESULT (tool call→its result, correlated by tool-call id) edges — exposed in the explain_agent_run MCP result as a callGraph field. The dashboard Sessions view renders it per session (a "Call graph" button loads it via explain_agent_run): each step shows the message role and the tool calls it made, with a result indicator, plus a copyable Mermaid flowchart source. Deterministic and read-only. See docs/code/llm-mocking.md.
  • Added opt-in, exploratory semantic prompt matching for LLM conversations: a semanticMatch turn predicate (the intent the latest message should express) judged by a runtime LLM via the client SPI. It is off by default and never on the assertion path — the predicate is ignored unless mockserver.llmSemanticMatchingEnabled is set and a runtime backend resolves, so deterministic matching is never affected by default. Non-deterministic by nature (a live LLM judge), so it is documented for exploration only, never for CI assertions; fails closed (a non-affirmative/empty/errored judge does not match). Exposed in the Java TurnBuilder.whenSemanticMatch, the create_llm_conversation MCP tool, and the dashboard wizard (clearly flagged exploratory). See docs/code/llm-mocking.md.

LLM resilience, validation & cost testing

  • Added a verify_structured_output MCP tool: validate that the structured (JSON) output of recorded LLM responses conforms to a JSON Schema. It decodes each recorded response for a given provider (via the runtime-LLM client SPI), extracts the assistant's output text, and checks it against the schema — so you can assert that an agent (or a mocked model) produced schema-valid structured output. Read-only and deterministic; responses with no text output are reported separately as skipped, and the result gives per-response conformance with validation errors. See the AI/MCP tools page and docs/code/llm-mocking.md.
  • A mock LLM completion can now declare an outputSchema (a JSON Schema) that its response text is expected to conform to. As the response is encoded, MockServer validates the configured text against the schema and, on a mismatch, fail-soft: the response body is returned exactly as configured but an x-mockserver-structured-output-invalid diagnostic header is added and a warning logged — so a malformed structured-output fixture is surfaced immediately while a deliberately non-conforming fixture still returns unchanged. A blank schema, absent text, or a malformed schema are all treated as "nothing to check" and never affect the response. Exposed on the Java Completion.withOutputSchema(...), the outputSchema field in expectation JSON, and the mock_llm_completion MCP tool (string or inline object). Complements the read-side verify_structured_output tool. See the AI/MCP tools page and docs/code/llm-mocking.md.
  • Added a verify_cost_budget MCP tool: a deterministic, read-only cost gate for agent runs. It decodes each recorded LLM response for a provider (via the runtime-LLM client SPI), sums the input/output tokens from each response's usage, prices them with a new built-in pricing table (org.mockserver.llm.cost.LlmPricing, mirroring the dashboard's llmPricing.ts — same prefixes/rates), and asserts the total estimated USD cost is at or below maxCostUsd. The model can be pinned via a model param or read per-response from the recorded request body; responses with no usage are skipped and responses whose model has no known price are reported as unpriceable and excluded from the total. The result gives token/cost totals, withinBudget, and a per-response breakdown. Pricing is public list pricing captured 2025-Q4 (an estimate, not an invoice). See the AI/MCP tools page and docs/code/llm-mocking.md.
  • Added declarative LLM fault/chaos profiles for resilience testing, attachable to any mock LLM response (mock_llm_completion, each create_llm_conversation turn, the Java LlmConversationBuilder, and raw expectation JSON via a chaos block). Supports probabilistic provider errors (e.g. 429/529 with a Retry-After header), mid-stream truncation of an SSE stream (keep a leading fraction of events), and appending a malformed (broken-JSON) SSE chunk. Errors are deterministic at probability 0.0/1.0 and reproducible at fractional probabilities via a seed; truncation and malformed-SSE are always deterministic. A new LLM_CHAOS_INJECTED_COUNT metric tracks injections. The dashboard conversation wizard exposes the profile per turn. See the AI/MCP tools page and docs/code/llm-mocking.md.
  • Added a stateful request quota to the LLM chaos profile — a deterministic fixed-window rate limit, the stateful counterpart to the existing probabilistic 429. Set quotaName, quotaLimit, and quotaWindowMillis (optional quotaErrorStatus, default 429) on a chaos block and requests beyond the limit within the window are rejected with that status and the retryAfter header. Expectations sharing a quotaName share one counter (model an upstream account limit across several mocks); the count resets when the window elapses and on server reset. Backed by a new process-wide, thread-safe org.mockserver.llm.LlmQuotaRegistry (injectable clock for deterministic tests). Exposed in expectation JSON, the mock_llm_completion/create_llm_conversation chaos MCP parameter, and the Java LlmChaosProfile. A misconfigured/partial quota fails open (never rate-limits). See the AI/MCP tools page and docs/code/llm-mocking.md.
  • Added a prompt-injection / adversarial-response harness for testing agent resilience. A new mock_adversarial_llm_response MCP tool returns a curated adversarial payload as the mock LLM response — prompt-injection ("ignore previous instructions…"), jailbreak persona-swaps, data-exfiltration requests, malformed/truncated JSON, an empty response, and an over-long repetition — so you can verify your agent resists hostile or malformed model/tool output. Backed by AdversarialResponseLibrary (deterministic; the payloads are benign test fixtures, not working exploits). A defensive testing aid. See the AI/MCP tools page and docs/code/llm-mocking.md.
  • Added drift detection for LLM fixtures (detect_llm_drift MCP tool): replays a recorded cassette's exchanges against the live provider (via the runtime-LLM client SPI) and reports structural drift — new/removed fields and type changes in the responses — not semantic differences, so benign wording changes never flag. Built on a reusable, pure StructuralShapeDiff and a DriftDetector that fails closed per exchange (a network error or non-2xx live response is reported as could-not-check, never as drift, never thrown). Off unless a runtime backend is configured. Intended for an opt-in/scheduled CI lane (real API keys + tokens), never the per-commit build. See the AI/MCP tools page and docs/code/llm-mocking.md.
  • Completed the VCR (record/replay) toolkit for LLM fixtures with three additions. (1) Strict mode — load_expectations_from_file accepts strict (or set mockserver.llmVcrStrict), which registers a low-priority catch-all per cassette path so a request matching no recorded fixture returns HTTP 599 instead of silently falling through. (2) Body-field redaction — record_llm_fixtures accepts redactBodyFields (or set mockserver.fixtureBodyRedactFields) to redact named JSON fields from recorded request/response bodies, complementing the existing header redaction. (3) Replay field normalisation — load_expectations_from_file accepts normalizeRequestBodyFields to drop volatile JSON fields from each recorded request body and match the remainder loosely (ignoring extra fields), so per-run values (request ids, timestamps) do not block replay. These are operational settings exposed via config and MCP. See the AI/MCP tools and configuration properties pages.

HTTP chaos & protocol contract testing

  • Added a time-to-live (auto-revert) to service-scoped chaos — an optional ttlMillis on a PUT /mockserver/serviceChaos registration makes the chaos automatically revert after that many milliseconds (a "dead-man's switch" so a fault self-heals even if the matching clear is never sent — e.g. an external chaos orchestrator crashes mid-experiment). It is also the one-shot time-box form: a single call breaks a host for a bounded window. Expiry is measured with the controllable clock (real-time by default, deterministic under PUT /mockserver/clock) and is applied lazily on the next lookup. Exposed via the endpoint, the Java/Node/Python/Ruby clients (setServiceChaos(host, chaos, ttlMillis) / ttl_millis), and the manage_service_chaos MCP tool. See the Chaos Testing page.
  • Added service-scoped chaos — register one HttpChaosProfile for an upstream host and have it applied to all matched forwards to that host, instead of attaching a chaos block to every forwarding expectation (the "break service X" control for running MockServer as a chaos proxy). Manage it through a new control-plane endpoint PUT/GET /mockserver/serviceChaos ({"host":...,"chaos":{...}} to register, {"host":...,"remove":true} to remove, {"clear":true} to clear all), protected by control-plane authentication. Resolution happens only on the matched-forward path keyed by the request Host header (case-insensitive, port-ignored); an expectation's own chaos always takes precedence, the anonymous proxy fall-through is unaffected, and registrations clear on server reset. Backed by a new process-wide org.mockserver.mock.action.http.ServiceChaosRegistry. Convenience wrappers are exposed in all four clients (setServiceChaos/removeServiceChaos/clearServiceChaos/serviceChaosStatus in Java/Node, the snake-case equivalents in Python/Ruby) and via the manage_service_chaos MCP tool. See the Chaos Testing page.
  • Added gradual degradation to the HTTP chaos block — a degradationRampMillis that linearly ramps errorProbability and dropConnectionProbability from 0 up to their configured values over the window from the expectation's first match, modelling a dependency that deteriorates over time (for alerting / SLO-burn tests). The ramp is measured with MockServer's controllable clock, so it is deterministic under clock freeze/advance with no real-time waiting; only the probabilistic rates ramp (latency, body corruption, slow response and quota are unaffected). Exposed in expectation JSON, the Java/Node/Python/Ruby clients, and the create_expectation chaos MCP parameter. See the Chaos Testing page.
  • Added a stateful request quota to the HTTP chaos block — a deterministic fixed-window rate limit, the HTTP counterpart of the existing probabilistic 429 and of the LLM quota. Set quotaName, quotaLimit and quotaWindowMillis (optional quotaErrorStatus, default 429) and requests beyond the limit within the window are rejected with that status and the retryAfter header. Expectations sharing a quotaName share one counter (model an upstream account limit across several mocks); the count resets when the window elapses and on server reset. The quota gate takes priority over the probabilistic error and the body/slow faults (after connection-drop). Backed by a new process-wide, thread-safe org.mockserver.mock.action.http.HttpQuotaRegistry (separate from the LLM quota registry). Exposed in expectation JSON, the Java/Node/Python/Ruby clients, and the create_expectation chaos MCP parameter; metered as fault_type=quota. See the Chaos Testing page.
  • Added a slow (dribbled) response fault to HttpChaosProfile — slowResponseChunkSize + slowResponseChunkDelay trickle the response body to the client in small chunks with a delay between each (via chunked transfer-encoding), for testing read timeouts and slow-network handling (distinct from latency, which delays the whole response by a fixed amount). Both fields are required; deterministic; applies to the real mocked or forwarded response within the active count and outage windows; skipped for streaming bodies; metered as fault_type=slow. Exposed in expectation JSON, the Java/Node/Python/Ruby clients, and the create_expectation chaos MCP parameter. See the Chaos Testing page.
  • Added response-body corruption faults to HttpChaosProfile — truncateBodyAtFraction keeps only a leading fraction of the body bytes (e.g. 0.5 returns the first half, 0.0 empties it) and malformedBody appends a broken-JSON fragment so the payload fails to parse, for testing client-side body-parsing and partial-response resilience. Both are deterministic (no probability draw), apply to the real mocked or forwarded response within the active count and outage windows, preserve the Content-Type and drop any stale Content-Length (the encoder then sets the correct length) so the response stays well-framed, and are skipped for streaming bodies. Connection-drop and error injection still take priority (an injected error body is never corrupted). Exposed in expectation JSON, the Java/Node/Python/Ruby clients, and the create_expectation chaos MCP parameter; metered as fault_type=truncate / fault_type=malformed. See the Chaos Testing page.
  • Added time-based outage windows (outageAfterMillis / outageDurationMillis) to HttpChaosProfile — chaos becomes active a configurable time after the expectation's first match and (optionally) self-heals after a bounded duration, modelling a dependency that degrades for a transient window then recovers. The window is measured with MockServer's controllable clock, so it is deterministic under clock freeze/advance (PUT /mockserver/clock) with no real-time waiting; it composes with the count window and the probability fields.
  • Added connection-drop chaos fault (dropConnectionProbability) to HttpChaosProfile — probabilistic TCP connection drops (no response sent) on both mocked and forwarded responses, simulating hard network failures. Drop faults take priority over error and latency injection (drop > error > latency). Uses a derived seed for independent but reproducible draws alongside errorProbability.
  • Added declarative HTTP chaos/fault injection (HttpChaosProfile) for resilience testing, attachable to any expectation via a top-level chaos block. Supports probabilistic error-status injection (e.g. 500, 503, 429 with an optional Retry-After header) and latency injection. Works on both mocked responses (RESPONSE, RESPONSE_TEMPLATE, RESPONSE_CLASS_CALLBACK) and forwarded/proxied responses (FORWARD, FORWARD_TEMPLATE, FORWARD_CLASS_CALLBACK, FORWARD_REPLACE, FORWARD_VALIDATE), making MockServer usable as a chaos proxy for testing how applications handle unreliable upstream dependencies. Deterministic at errorProbability 0.0/1.0; reproducible at fractional probabilities via a seed. Exposed in the Java client (ForwardChainExpectation.withChaos()), REST API, and expectation JSON. See the new Chaos Testing & Fault Injection documentation page.
  • Added count-based stateful faults to the HTTP chaos block — a succeedFirst / failRequestCount request-count window so an expectation can succeed the first N matches, then fault the next M, then recover. Expresses fail-first-N-then-recover (retry/backoff testing), succeed-N-then-fail, and fail-only-the-Nth, on both mocked and forwarded responses; deterministic by match index, composes with errorProbability, and is backward compatible (no window fields = unchanged). See the Chaos Testing page.
  • Added a Driving MockServer from Chaos Orchestrators guide showing how external chaos-engineering tools drive MockServer's service-scoped chaos through the control-plane endpoint — concrete inject/verify/revert recipes for Chaos Toolkit, AWS FIS (SSM RunShellScript), Azure Chaos Studio (Automation runbook / pipeline), LitmusChaos (BYOC cmdProbe/httpProbe), and any cron/CI/Step Functions scheduler — all using the ttlMillis dead-man's switch so a fault auto-reverts even if the orchestrator never sends the clear. See the Chaos Orchestrators page.
  • Added a Chaos Proxy in Kubernetes guide showing how to deploy MockServer as a chaos proxy in Kubernetes to inject faults into real service-to-service and external API calls — reverse-proxy, egress/forward-proxy, and sidecar deployment patterns with concrete Kubernetes manifests and expectation JSON examples. See the Chaos Proxy in Kubernetes page.
  • Added a chaos-proxy example to the Helm chart — a commented reverse-proxy + chaos initializerJson block in values.yaml and a "Chaos Proxy (fault injection)" section in the chart README, showing how to deploy MockServer in front of an upstream Service and inject faults through the chart's inline configuration. Links to the Chaos Testing and Chaos Proxy in Kubernetes guides.
  • Added an MCP server conformance tester (run_mcp_contract_test MCP tool): point it at a target MCP (Model Context Protocol) server's Streamable HTTP endpoint and it runs the required JSON-RPC handshake and core methods — initialize, notifications/initialized, ping, tools/list, and unknown-method rejection (expects error code -32601) — validating the shape of each response (JSON-RPC 2.0 envelope and required result fields), never the semantics of any tool. Optionally exercises one tools/call (skipped by default, since a call may have side effects on the target). Fully deterministic and offline-from-LLMs (no model is involved); each request has a 10-second timeout. Backed by a network-free, unit-testable McpContractTest orchestrator with an injected transport. See the AI/MCP tools page and docs/code/llm-mocking.md.

Observability & dashboard

  • Added an active service-scoped chaos gauge — a Prometheus mock_server_active_service_chaos gauge (when metricsEnabled) labeled by fault_type (drop/error/latency/truncate/malformed/slow/quota), reporting per fault type how many currently-active service-scoped chaos profiles are configured with that fault (a profile with several faults counts under each). It is a callback gauge that reads ServiceChaosRegistry at scrape time, so each series drops to 0 as profiles are cleared or their TTLs lapse (making sum(mock_server_active_service_chaos) > 0 a natural "chaos still live" alert and letting you alert on a specific fault type), and it is mirrored over OTLP alongside the chaos-fault-injection counter. See the Chaos Testing page.
  • The dashboard Metrics view "HTTP Chaos Faults" section now shows every fault type the server emits (drop, error, latency, truncate, malformed, slow, quota) — previously only error and latency — with a per-fault-type chart of cumulative injections and a separate per-fault-type chart of the active service-scoped chaos gauge (plotted by type rather than as a single counter). Fault types are discovered from the scrape, so a future type renders automatically without a UI change. See docs/code/dashboard-ui.md.
  • Added a Chaos tab to the dashboard UI for managing service-scoped chaos interactively (ServiceChaosPanel): register a host with an error status / error probability / drop probability / latency (and an optional TTL), see every active registration with a summary of its faults, watch the live TTL auto-revert countdown, and remove a single host or clear them all. It polls GET /mockserver/serviceChaos and drives the same control-plane endpoint as the clients and the manage_service_chaos MCP tool. The /mockserver/serviceChaos responses now carry CORS headers unconditionally (matching the metrics and MCP endpoints), so the dashboard works when served from a different origin (e.g. the UI dev server) without needing enableCORSForAPI. See the Chaos Testing page and docs/code/dashboard-ui.md.
  • Added optional OpenTelemetry (OTLP) export, in two independent, off-by-default parts. (1) Metrics export — MockServer's existing metrics (the same explicitly-defined gauges already exposed for Prometheus: REQUESTS_RECEIVED_COUNT, RESPONSE_EXPECTATIONS_MATCHED_COUNT, the LLM/SSE/chaos counters, etc.) can also be pushed to an OTLP collector as an alternative to Prometheus (mockserver.otelMetricsEnabled). Implemented as OTel observable gauges reading the current values, so the Prometheus and OTLP views stay in lock-step. (2) GenAI span export — MockServer emits one explicit OpenTelemetry GenAI semantic-convention span per LLM completion it serves (gen_ai.system, gen_ai.request.model, gen_ai.usage.input_tokens/output_tokens, gen_ai.response.finish_reasons, tool-call count) (mockserver.otelTracesEnabled). These are spans MockServer codes deliberately — no auto-instrumentation is added. Both use the OTLP HTTP/protobuf exporter with the JDK HttpClient sender (no gRPC/OkHttp), share mockserver.otelEndpoint, and are fail-soft (a setup error logs one line and never stops the server or affects a response). io.opentelemetry.* is relocated in the shaded JAR. See the configuration properties page.
  • Added JVM runtime metrics to MockServer's Prometheus endpoint (GET /mockserver/metrics, when metricsEnabled): heap and non-heap memory (used / committed / max, labelled by area), live and daemon thread counts, and total GC collection count and time. Exposed via a dependency-free collector that reads JDK MX beans, so Grafana and the dashboard Metrics view can chart process health alongside the existing request/action counters.
  • Added a request-latency histogram to MockServer's Prometheus endpoint (mock_server_request_duration_seconds, when metricsEnabled): classic histogram buckets from 0.5 ms to 10 s, recorded per request from receipt to response. Enables latency percentiles (p50 / p95 / p99 via histogram_quantile) in Grafana and the dashboard. Recording is fully gated behind metricsEnabled, so it adds nothing to the request path when metrics are off.
  • Added a Metrics view to the dashboard UI: a new top-bar tab that polls MockServer's Prometheus endpoint (GET /mockserver/metrics) and renders live activity — request / matched / not-matched / forwarded counts with inline sparklines, a derived requests-per-second throughput chart, a per-action breakdown, JVM heap / thread / GC panels, and request-latency percentiles (p50 / p95 / p99) — the JVM and latency panels appear only when the server exposes those metrics — plus the served MockServer version. Time-series charts use @mui/x-charts, lazy-loaded so they add nothing to the initial dashboard load. It degrades gracefully: when MockServer is started without metricsEnabled the endpoint returns 404 and the view shows guidance to enable it (-Dmockserver.metricsEnabled=true / MOCKSERVER_METRICS_ENABLED=true). See docs/code/dashboard-ui.md.
  • Recorded requests can now be exported as cURL commands. A new CURL value for the /mockserver/retrieve format parameter (valid for type=REQUESTS and type=REQUEST_RESPONSES) renders one curl command per recorded request via the existing HttpRequestToCurlSerializer; the expectation scopes return a clear "not supported" message. Surfaced in the dashboard Export page. See the configuration/retrieve docs.

Templating & runtime

  • Added a clock-control endpoint (PUT /mockserver/clock, GET /mockserver/clock) for deterministic time-based testing. Freeze the server clock at a specific ISO-8601 instant, advance it by a duration in milliseconds, or reset it to real wall-clock time. The controllable clock affects response template date/time helpers (now_iso_8601, now_epoch, now_rfc_1123, and the dates helper object) and expectation TimeToLive expiry, so frozen time prevents expectations from expiring mid-test. Protected by control-plane authentication (JWT/mTLS) when configured. Limitation: event-log timestamps and JWT token issuance use a separate time source and are not affected. See the Clearing, Resetting & Clock Control page.
  • DataFaker (net.datafaker:datafaker:2.5.4) is now bundled as a template helper. A single shared Faker instance is exposed as faker in all three response-template engines (Velocity, Mustache, JavaScript) via TemplateFunctions.BUILT_IN_HELPERS, giving templates access to 250+ realistic-fake-data providers (faker.name().firstName(), faker.internet().emailAddress(), faker.address().city(), etc.). The instance is thread-safe and produces fresh random values on each call. See the consumer docs (response templates page) for the full provider list and per-engine syntax. Java 17 unlocked this — DataFaker 2.x requires Java 17; the previous Java 11 floor pinned us to the abandoned 1.9.0 line.
  • Documented ZGC (-XX:+UseZGC) as a recommended GC for deployments with large heaps (≥ 4 GB) or deep maxLogEntries ring buffers. Java 17 ships production-ready ZGC; for matcher-path latency this can reduce p99 pauses from tens or hundreds of milliseconds (G1 under sustained allocation) into single-digit milliseconds. ZGC is not the default because typical MockServer fixtures run small heaps where Parallel/G1 are fine and ZGC's fixed memory overhead hurts sub-2 GB scenarios. Includes container-memory headroom guidance (size container limit at ~1.5× heap when using ZGC). See the performance tuning page on the website.

HTTP/3, transparent proxy & infrastructure

  • HTTP/3 streaming / SSE responses (Http3ResponseWriter): StreamingBody responses (Server-Sent Events, chunked proxy forwarding, LLM streaming) are now fully supported over HTTP/3. Http3ResponseWriter subscribes to the StreamingBody, sends HTTP/3 headers immediately, and forwards each chunk as an HTTP/3 DATA frame with backpressure via StreamingBody.requestMore(). The QUIC stream output is shut down on completion or error. Resolves the previous limitation where only static response bodies could be returned over HTTP/3. See docs/code/http3.md.
  • gRPC streaming over HTTP/3 — server-streaming and bidi-streaming (completes the gRPC-over-HTTP/3 work). A grpcStreamResponse expectation now streams each message as its own HTTP/3 DATA frame (with per-message delays) followed by a trailing grpc-status HEADERS frame; HttpActionHandler routes the GRPC_STREAM_RESPONSE action to the new transport-neutral GrpcStreamResponseWriter seam (implemented by Http3GrpcResponseWriter) for HTTP/3, while HTTP/2 is unchanged. A grpcBidiResponse expectation now drives true bidirectional streaming over a single full-duplex QUIC stream via the new Http3GrpcBidiStreamHandler (gated by the existing grpcBidiStreamingEnabled flag, same two-phase peek-then-consume matching and responseInProgress lifecycle as the HTTP/2 path). Message encoding and rule matching are shared across transports via new GrpcStreamMessageEncoder / GrpcBidiRuleMatcher core helpers. Covered by native-QUIC integration tests (Http3GrpcStreamingIntegrationTest). With this, gRPC over HTTP/3 reaches full parity with HTTP/2 (unary, server-streaming, bidi-streaming). See docs/code/http3.md.
  • Bundled native QUIC — the netty-incubator-codec-http3 dependency pulls in netty-incubator-codec-native-quic classifiers for all five supported platforms (linux-x86_64, linux-aarch_64, osx-x86_64, osx-aarch_64, windows-x86_64) automatically; no separately downloaded BoringSSL library is required. An in-JVM Netty QUIC-client integration test verifies the full pipeline parity including streaming, gated on Quic.isAvailable() so the suite degrades gracefully where native QUIC is absent.
  • TPROXY (IP_TRANSPARENT) transparent-proxy strategy — a new default-off transparentProxyTproxy configuration property (-Dmockserver.transparentProxyTproxy=true / MOCKSERVER_TRANSPARENT_PROXY_TPROXY=true) enables IP_TRANSPARENT socket binding so that, with iptables TPROXY rules, the kernel preserves the original destination as the listening socket's local address — which MockServer reads directly via channel.localAddress(), as an alternative to the existing conntrack SO_ORIGINAL_DST strategy (REDIRECT rules). Requires Linux, the epoll transport (NIO unsupported), and CAP_NET_ADMIN. The transparent proxy enabled flag (transparentProxyEnabled) is unchanged; the new property selects the kernel mechanism only. Verified end-to-end with a real Docker NET_ADMIN integration test for both SO_ORIGINAL_DST and TPROXY paths. eBPF sockmap-based redirection is deferred (placeholder added). See docs/infrastructure/service-mesh.md.
  • Testcontainers 1.21.4 — upgraded from 1.20.6, picking up docker-java 3.4.2 which fixes DockerClientFactory.isDockerAvailable() returning false on Docker Desktop 4.67 / Engine API 1.54 (the 3.4.1 /info probe sent the wrong Content-Type header and received HTTP 400, causing a false-negative result). No API or behaviour change for callers; tests that previously skipped on Docker Desktop 4.67+ now run correctly.

Clustered state (opt-in, mockserver-state-infinispan)

  • Added a StateBackend SPI in mockserver-core (org.mockserver.state.StateBackend) — a pluggable interface that abstracts all shared MockServer state into three store types: a versioned KeyValueStore<ExpectationEntry> (expectations), a KeyValueStore<String> (scenario states), KeyValueStore<ObjectNode> (CRUD entities per namespace), and a BlobStore (persisted cassettes and fixtures). InvalidationListener callbacks allow clustered implementations to trigger node-local rebuilds when a remote write arrives. The default implementation is InMemoryStateBackend, which wraps the existing concurrent data structures — single-node behaviour and performance are completely unchanged.

  • Added mockserver-state-infinispan, a new optional Maven module providing an embedded Infinispan StateBackend that can replicate MockServer expectations and scenario state across a JGroups cluster. Classpath-auto-discovered when mockserver.stateBackend=infinispan is configured (via StateBackendFactory reflection — mockserver-core has no compile-time dependency on Infinispan). Two modes: LOCAL (single-node, no JGroups, heap-only Infinispan cache, permissive serialization allow-list) and CLUSTERED (clusterEnabled=true, REPL_SYNC caches, JGroups transport, explicit serialization allow-list covering exactly the MockServer domain types). Expectations and scenario states use REPL_SYNC so all writes are synchronously replicated to every cluster member. An Infinispan @Listener(clustered=true) fires InvalidationListener.onChanged() on remote writes, triggering RequestMatchers.reconcileFromBackend() on the receiving node to rebuild its local HttpRequestMatcher cache. Approximate eviction (maxCount) on the expectations cache matches the maxExpectations configuration property. See docs/code/clustered-state.md.

  • New configuration properties for state clustering:

    Property Env var Default Description
    mockserver.stateBackend MOCKSERVER_STATE_BACKEND memory Backend type: memory or infinispan
    mockserver.blobStoreType MOCKSERVER_BLOB_STORE_TYPE filesystem Blob store type: filesystem or memory
    mockserver.clusterEnabled MOCKSERVER_CLUSTER_ENABLED false Enable JGroups cluster transport
    mockserver.clusterName MOCKSERVER_CLUSTER_NAME mockserver-cluster JGroups cluster identifier
    mockserver.clusterTransportConfig MOCKSERVER_CLUSTER_TRANSPORT_CONFIG (built-in loopback) Path to a custom JGroups XML transport config

    Setting stateBackend=infinispan without clusterEnabled=true starts Infinispan in LOCAL mode (single-node, functionally equivalent to the default in-memory backend but adds Infinispan on the classpath). A misconfigured stateBackend=infinispan where the module is absent fails fast with IllegalStateException rather than silently falling through to in-memory (which would cause split-brain). Scenario-state transitions are atomic cluster-wide (versioned compare-and-set), and shared Times counters (per-expectation match limits) are enforced cluster-wide via backend CAS (exactly-once across nodes). Remaining node-local aspects: the request/event log and verify() are per-node (verification queries a single node's log). See docs/code/clustered-state.md.

Changed

  • Upgraded the Prometheus metrics client (io.prometheus:prometheus-metrics-core, -exposition-formats, -model) from 1.6.1 to 1.7.0. Source- and behaviour-compatible (metrics are emitted only when metricsEnabled); the metrics exposition format is unchanged. io.netty:netty-tcnative-boringssl-static is deliberately not bumped alongside it — tcnative is version-locked to Netty (its per-platform classifier artifacts arrive transitively at Netty's tcnative version, so an independent bump breaks Maven dependencyConvergence); it is now in the Dependabot ignore list and is upgraded manually in lockstep with the netty.version bump.
  • LlmChaosProfile now validates its numeric fields in its withX builder methods, matching the validation HttpChaosProfile already enforces: errorProbability / truncateAtFraction must be in [0.0, 1.0], errorStatus / quotaErrorStatus in [100, 599], and quotaLimit / quotaWindowMillis ≥ 1. An out-of-range value now throws IllegalArgumentException with a clear message when a profile is built via the Java client or parsed from the chaos MCP parameter, instead of being silently accepted.
  • Reworked the dashboard Export page: choose the scope (Active expectations / Recorded requests) with a radio and the file format with a dropdown, instead of one long combined list. Added JAVA (expectations), log-entries (requests) and cURL (requests) formats, filtered by the chosen scope, and the best-effort caveat is now shown only when it applies. Export is now the first Library tab. The run comparison tool moved out of Library into a new Compare tab under Sessions (where it belongs, since it diffs sessions).
  • Upgraded the chicory WASM interpreter (com.dylibso.chicory:runtime) from 0.0.12 to 1.7.5, moving off the old pre-1.0 release onto the stable 1.x line. WasmRuntime is migrated to the new API (Parser.parse(bytes) → WasmModule, Instance.builder(module).build(), and ExportFunction.apply(long…) returning long[]). The experimental WASM custom-rule feature's behaviour and module ABI (match(i32 ptr, i32 len) -> i32) are unchanged.
  • Upgraded com.networknt:json-schema-validator from 1.5.9 to 3.0.3. The 3.x line uses the tools.jackson (Jackson 3.x) namespace internally and snakeyaml-engine for YAML schemas. MockServer's external Jackson usage stays on 2.22.0; the two Jackson namespaces coexist because they are in different Java packages. JsonSchemaValidator is rewritten against the new Schema / SchemaRegistry / SpecificationVersion API and uses the string-based getSchema(String, InputFormat.JSON) and validate(String, InputFormat.JSON) entry points to avoid passing Jackson 2.x JsonNode objects into Jackson 3.x APIs. PathType.JSON_PATH is configured so validation messages keep the existing $.property format and no test fixture had to change. The shaded uber-JAR adds two new relocations (tools.jackson and org.snakeyaml).
  • BREAKING: minimum supported Java runtime raised from Java 11 to Java 17. mockserver/pom.xml maven.compiler.source and maven.compiler.target are now 17, so published artifacts are Java 17 bytecode and will not run on a Java 11 JVM. The CodeQL workflow, Buildkite build agent image, and local dev scripts have all been aligned to JDK 17.
  • BREAKING: coordinated upgrade to the Jakarta EE 10 / Servlet 6 stack and the upstream dependencies that required it. The full javax.* → jakarta.* namespace migration (servlet, ws.rs, annotation, inject, persistence) is now complete. Library bumps: Spring Framework 5.3 → 7.0, Spring Boot 2.7 → 4.0, Tomcat embed 9 → 11, Jetty 9.4 → 12, Jersey 3.1 → 4 (jersey-apache-connector → jersey-apache5-connector with Apache HttpClient 5), jakarta.xml.bind-api 3 → 4, jakarta.servlet-api 4 → 6, jakarta.ws.rs-api 2.1 → 4, jakarta.annotation-api 1.3 → 3, JUnit Jupiter 5.14 → 6.1, json-unit 2 → 5, json-path 2 → 3, Netty 4.1 → 4.2.15.Final (introduced via netty-bom so the new netty-codec-base / netty-codec-compression / netty-codec-http3 sub-modules stay aligned).
    • Runtime deployment in a servlet container now requires a Servlet 6 / Jakarta EE 10 host: Tomcat 11+, Jetty 12+, WildFly 32+, or equivalent. Servlet 5 / Jakarta EE 9 containers are no longer supported.
    • MockServerServlet and ProxyServlet runtime contract is unchanged for consumers using jakarta.servlet.*. Consumers still importing javax.servlet.* must update their imports.
    • WAR test scaffolding that configured TLS via the removed Connector.setAttribute("keystoreFile"/"keystorePass"/…) API must migrate to the Tomcat 11 SSLHostConfig + SSLHostConfigCertificate pattern. The four WAR/proxy-war integration test classes in this repo show the working shape.
    • Servlet 6 preserves RFC 6265 surrounding double quotes on cookie values returned by Cookie.getValue(). MockServer's request decoder now strips them so cookie semantics are unchanged for clients.
    • Spring 7 requires the -parameters javac flag for @PathVariable / @RequestParam name resolution; this is now enabled project-wide in maven-compiler-plugin.
    • Spring 7's MappingJackson2HttpMessageConverter is deprecated for removal in favour of JacksonJsonHttpMessageConverter. MockServer keeps Jackson at 2.22.0 for now because swagger-parser is still locked to Jackson 2; Jackson 3 upgrade will land once swagger-parser ships a Jackson 3 line (see #1970).
  • BREAKING: Nashorn (org.openjdk.nashorn:nashorn-core:15.7) removed as a managed dependency. JavaScriptTemplateEngine now uses the GraalVM Polyglot API directly (org.graalvm.polyglot.Context with HostAccess.ALL + allowHostClassLookup for the existing class-deny-list security policy). GraalJS 25.x dropped the JSR-223 javax.script bridge, so the previous Nashorn-or-GraalJS-via-JSR-223 fallback would have silently returned a null engine and broken every JavaScript template at runtime. Downstream consumers that previously relied on Nashorn arriving transitively must add org.openjdk.nashorn:nashorn-core to their own dependencies, or migrate to GraalVM polyglot directly.
  • Drop the --add-exports=java.base/sun.security.{x509,util}=ALL-UNNAMED javac flags inherited from the Java 11 era. Repo-wide audit found zero sun.security.* references after the Java 17 / jakarta migration, so the flags were dead weight.
  • Performance: the request-matching hot path no longer builds the human-readable "did not match because…" diagnostic string (the per-field message assembly and per-field hint generation) when it would only be discarded — i.e. when the log level is below INFO. The match evaluation, the match-difference data behind detailedMatchFailures / debugMismatch / explainUnmatched / verification, and the match result are unchanged; only the discarded narrative is skipped, and the per-matcher StringBuilder is no longer allocated in that case. For a server with many registered expectations running below INFO under sustained load this measurably cuts per-request allocation and GC pressure (JMH -prof gc: ~36% less matching-path allocation at 1000 expectations and log level WARN; no change at the default INFO). See the performance documentation's note on logLevel and matching throughput. A new on-demand mockserver-benchmark JMH module (excluded from the default build) backs these numbers.

Fixed

  • CPU no longer climbs as the request/event log fills (issue #2329). CircularConcurrentLinkedDeque — the bounded ring used for the request/event log — checked capacity on every insert with ConcurrentLinkedDeque.size(), which is O(n) (it walks the whole list). Once the log reached maxLogEntries (default 100,000) each request paid an O(n) traversal per log entry, so CPU rose as the log filled and stayed high (and clearing expectations does not clear the log, so it never recovered). Size is now tracked in an AtomicInteger, making the eviction check and size() O(1). Measured per-insert cost at the default capacity dropped from ~210µs to ~15ns (~14,000× at 100k entries; the old cost scaled linearly with maxLogEntries). No behaviour change — same bounded FIFO semantics and eviction callback. Tip for high-throughput users: also clear the log (PUT /mockserver/clear?type=LOG or ?type=ALL, or PUT /mockserver/reset), not just expectations, or lower maxLogEntries.
  • Regex matching in the GraphQL, JSON-RPC and LLM-conversation matchers is now ReDoS-bounded. User-supplied regular expressions for a GraphQL operationName, a JSON-RPC method, and an LLM conversation's latestMessageMatches are now evaluated under the shared mockserver.regexMatchingTimeoutMillis timeout via MatchingTimeoutExecutor — the same protection RegexStringMatcher already applies to path/header/body regexes — so a pathological pattern can no longer pin a worker thread (ReDoS). A timed-out evaluation is treated as a non-match. (Resolves CodeQL alert for GraphQLMatcher; the same fix is applied to the two sibling matchers.)
  • Dashboard Log Messages panel: a non-breaking space is now rendered after each expandable JSON block, so the text that follows (e.g. } matched expectation:) no longer butts directly against the closing brace.
  • CORS for the dashboard served cross-origin. When mockserver.corsAllowOrigin is blank (the default) MockServer now reflects the request's Origin in Access-Control-Allow-Origin instead of emitting an empty (invalid) header, and falls back to sensible Access-Control-Allow-Methods / Access-Control-Allow-Headers when those are blank (reflecting the requested headers on preflight). The MCP endpoint (/mockserver/mcp) now answers the CORS preflight and exposes Mcp-Session-Id via Access-Control-Expose-Headers. Together these let the dashboard (and any browser client) call the control-plane API and MCP endpoint from a different port or domain. An explicit corsAllowOrigin is still honoured as an allow-list, and * is never combined with Access-Control-Allow-Credentials: true.
  • CORS for the metrics endpoint (/mockserver/metrics). The endpoint now adds the same Access-Control-Allow-Origin headers as the rest of the API, so the dashboard's Metrics view can fetch metrics when served cross-origin (e.g. the UI dev server on a different port). The disabled-state 404 carries the headers too, so the UI reads it cleanly and shows its "metrics disabled" guidance instead of a browser CORS fetch error.
  • Helm chart downloads for older versions: every chart listed in index.yaml now returns a valid .tgz from https://www.mock-server.com/. Previously, releases that created a new versioned site could leave older chart archives missing from the live bucket while index.yaml still referenced them, so helm pull / helm install failed for any version other than the latest. The release pipeline now syncs the full set of charts on every run, making the bucket self-healing (fixes #2282).
  • Content-Encoding no longer leaks across requests on a reused (pooled) connection. When a compressed request (e.g. Content-Encoding: gzip) was followed by an uncompressed request on the same keep-alive connection, the second request was incorrectly recorded with the first request's Content-Encoding header. The preserved-headers state is now reset per request, so each recorded request carries only its own encoding headers (fixes #2322).
  • Compressed request bodies now retain their original on-the-wire bytes. When an HTTP/1.1 request arrives with a Content-Encoding (e.g. gzip), MockServer still decompresses it for matching/recording as before, but now also keeps the original compressed bytes alongside the decompressed body. A new HttpRequest#getBodyAsOriginalRawBytes() returns the exact bytes the client sent (the compressed payload when compressed, otherwise the decompressed bytes), so you can verify a client actually compressed its body; getBodyAsRawBytes() is unchanged (decompressed). A BinaryBody expectation now matches against either the decompressed body or the original compressed bytes, so a mixture of compressed and uncompressed requests matches automatically with no configuration. The original bytes are serialised (as originalBody) so they survive retrieveRecordedRequests and persistence (fixes #2326).
  • WASM custom-rule security controls are now enforced. The wasmEnabled (default false) and wasmMaxMemoryPages (default 256) configuration properties were documented as gating the experimental WASM custom-rule feature but were never actually read. WASM support is now disabled by default and fails closed: the WASM module control-plane endpoints (PUT/GET/DELETE /mockserver/wasm/modules) return 403 and WasmBodyMatcher does not match unless mockserver.wasmEnabled=true, and a loaded module's linear memory is now capped at wasmMaxMemoryPages via chicory MemoryLimits at instance creation. Set wasmEnabled=true to opt in.

Removed

  • Removed the xDS route discovery feature (REST endpoint GET /mockserver/xds/routes, gRPC RDS server, xdsEnabled/xdsPort configuration properties, and Helm sidecar.xdsEnabled/sidecar.xdsPort values). The feature shipped behind default-off flags and saw no adoption; real service mesh integration routes traffic to MockServer via an Istio VirtualService rather than having MockServer act as an RDS server. The transparent proxy / sidecar mode (transparentProxyEnabled, conntrack SO_ORIGINAL_DST, iptables init container) is fully retained.
Assets 2
Loading