Releases: pranaysparihar/InfernoSIM
Release list
v4.0.1
InfernoSIM v4.0.1
Safety and recovery
- Independent monitor assertions detect absent verdicts, false healthy status,
stale response revisions, and bad request-side state estimates against a
declared simulated effect ledger. - Bound, single-use approval assertions check matching arguments/tenant and a
logical expiry window. Request predicates support synthetic identity and
authorization fixtures. Compensation assertions require later,
identity-matched rollback effects. - Shared call, recorded-token, and explicitly priced cost budgets fail on
unknown usage. Costs are configured estimates, not provider invoices. agent run/stress --restart-after-call Nkills and restarts the explicit
application at a simulated response boundary. Ledger/deduplication state and
a private application checkpoint directory survive the restart.- Assertion coverage distinguishes exercised, violated, and unexercised paths;
require_exercisedmakes absent safety behavior a release failure.
Deterministic exploration and regression gates
- Explicit call-ID admission schedules and bounded pairwise fault enumeration.
Every tool entry in a complete supported provider JSON envelope can be
selected. Combined faults no longer erase earlier transport failures. agent compareruns two explicit application argv arrays against the same
corpus/cases/schedules and detects failed cases, missing assertions, and
safety-coverage regressions. Reports include JSON, JUnit, SARIF, and HTML.agent reducefinds a one-minimal fault set preserving a named violated
assertion on repeated executions, with a hard execution budget.- MCP stdio correlates out-of-order responses to the correct runtime request
IDs, preserves notifications, remaps cancellation IDs, and rejects duplicate
in-flight IDs and incomplete transcripts. Already admitted work is distinct
from a new call after cancellation.
Fixes, compatibility, and packaging
- Missing effect identity/deduplication fields fail closed instead of collapsing
unrelated effects into a shared key. Request identity hashing preserves large
integer precision. JSON-RPC/tool errors cannot satisfy retry verification. - Updated
golang.org/x/netto v0.56.0 (andx/systo v0.46.0), removing
GO-2026-5942 from the dependency graph. Injected HTTP waits honor cancellation. - Agent harness generation retains combined fault and schedule selections.
- The release build targets the complete CLI package. Packaging is explicitly
eight platform archives plus checksums; GitHub's two automatic source
archives retain the established total of 11. Reports and internal JSON
metadata are not release assets. - Existing agent configuration defaults, backend replay features, and encrypted
bundle-v2 format remain compatible. New configuration fields are opt-in.
Validation and scope
Local validation includes the full Go race suite, vet, the existing safe/unsafe
agent smoke, the new recovery/safety/compare/reduce CLI smoke, official MCP Go
SDK v1.7.0 record/replay/reconnect integration, a Docker image build, a real
Testcontainers run, Kafka/AsyncAPI against Redpanda, and the installed Ollama
model smoke. The full suite was also run with the release toolchain Go 1.26.6.
After the dependency update, govulncheck reported no vulnerabilities in the
main-module and pinned SDK integration scans. The existing agent benchmark
passed 100/100 safe executions and rejected 80/80 unsafe controls across 20
iterations; the incident-to-test benchmark passed 100 runs with one stable
configuration hash, one stable harness hash, and zero detected fixture leaks.
The new crash/recovery example also passed inside a Linux/arm64 container
with external networking disabled. A local GoReleaser snapshot produced all eight platform
archives and their checksums verified. Cross-builds are not native Windows or
Intel macOS runtime tests.
These checks establish the behavior of the tested fixtures, not universal
agent safety, full MCP conformance, or superiority over competitors. Admission
schedules do not control arbitrary OS races. Approvals/credentials are local
test predicates, not real identity-provider enforcement. Process restart is
not power-loss recovery. Live model execution is separate from deterministic
recorded-envelope gates. Windows child termination does not contain descendants.
Start with the runnable safety/recovery guide and
the upgrade notes. Existing v4.0.0 release history follows.
InfernoSIM v4.0.0
InfernoSIM v4.0.0
Status: generally available
v4 adds a local, incident-derived reliability gate for tool-using agents. It
replays recorded LLM and tool protocols, injects deterministic semantic and
transport faults, accounts for real side-effect consequences, and emits
reviewable CI evidence without requiring a hosted InfernoSIM service or live
model during the test.
Highlights
infernosim agent cases,agent run, andagent stressplan stable
baseline/single-fault matrices and execute only the explicit child command
supplied after--.- MCP JSON-RPC over HTTP is recognized inside ordinary HTTP/HTTPS replay. The
newagent mcp recordandagent mcp replaycommands support bounded
newline-delimited stdio transcripts with semantic request matching and
runtime request-ID rewriting. Lost responses keep the replay server alive for
retries, and EOF evaluates assertion/fault coverage with optional reports. - Protocol adapters recognize OpenAI Responses, OpenAI Chat Completions,
Anthropic Messages, and Ollama chat tool-call envelopes. Generic HTTP
selectors remain available for custom gateways. - Faults can target response kind, provider, tool, HTTP method, host/path RE2,
and occurrence. Actions include semantic JSON/SSE/NDJSON mutation, status and
header changes, bounded delay/timeout, reset, truncation, empty success, and
“side effect committed but response lost.” - An explicit side-effect ledger tracks hashed identities, idempotency
deduplication, commit state, response delivery, and ambiguous completion
without retaining raw tool arguments or results. - Consequence assertions cover exactly-once, at-most-once, forbidden effects,
call budgets, verified retry, recorded-universe-only execution, and
deadlines. Verification can require a delivered 2xx response matching a
JSONPath/value predicate. - Streaming HTTP capture can preserve bounded SSE, NDJSON, and JSON-sequence
response frames and inter-frame delays; replay can mutate structured frames
without flattening their framing. agent otel importnormalizes OTLP JSON or JSONL into an owner-only
correlation log. Raw prompt, argument, and result attributes are excluded;
optional one-way content hashes support correlation.- Agent results produce private JSON plus JUnit, SARIF, and standalone HTML.
JUnit contains one testcase per planned case. A separate JSON reliability
surface reports unweighted observed pass rates by validated fault category;
it is evidence for that matrix, not a general model score. testgen --profile agentgenerates a stable case manifest and agent-aware
Testcontainers, Compose, or GitHub Actions wiring. Generated Actions run the
assertion engine and fail if a selected fault is never exercised.- Bundle-v2 archives automatically include and encrypt optional
mcp.logand
agent-spans.jsonlfiles because they remain regular incident files; the
encrypted format itself is unchanged.
Public validation fixture
examples/agent-reliability contains a sanitized refund incident and recorded
MCP/OpenAI-style exchanges. examples/agentlab contains a defensive loop and
an intentionally unsafe control. The matrix covers:
- a refund commit whose response is lost;
- an HTTP-200 policy response missing its decision field;
- an LLM refund tool call missing its arguments; and
- an MCP tools list missing the required refund tool.
The checked-in raw benchmark completed 20 identical matrix iterations. The
defensive loop passed 100/100 executions, the unsafe baseline passed 20/20,
and the unsafe loop was rejected in 80/80 fault executions. Case-plan hashes
were stable and raw content was not retained. Timing is environment-specific.
See benchmarks/results/agent-reliability.json.
The optional local Ollama smoke passed with llama3.1-local:latest by producing
a native policy_check tool call that the adapter recognized. Ollama sampling
is compatibility evidence, not the deterministic release oracle.
Validation gates
- Root and Testcontainers-module tests pass with the Go 1.26.6 race detector;
module consistency andgo vetare clean. The Go vulnerability scan reports
zero reachable vulnerabilities. - Ten bounded fuzz targets pass, including the agent fault engine, semantic
JSONPath implementation, MCP request matcher, OpenAPI matcher, bundle-v2,
gRPC, streaming-template, healer, and message surfaces. - The safe/unsafe agent smoke confirms five defensive passes, the unsafe
baseline control, and rejection of all four unsafe fault cases. The MCP stdio
CLI fixture evaluates fault coverage and consequence assertions at EOF. - The final production image builds, runs as the unprivileged
infernosim
user, enumerates the five checked-in cases, and passes the real
Testcontainers lifecycle test. Existing Node, Go Compose, and
Kafka/AsyncAPI/Redpanda smokes remain green. - Generated agent GitHub Actions and Compose harnesses pass
actionlintand
docker compose config. All repository workflows passactionlint. - A local GoReleaser snapshot builds eight platform archives plus
checksums.txt; every archive checksum verifies. The tagged release workflow
repeats the mandatory release gates before publishing that same artifact set.
Safety and compatibility
- Existing incidents and replay configurations remain valid; the
agent
section is opt-in. - The minimum Go toolchain is 1.26.6. This is the first 1.26 patch that fixes
the reachable standard-library advisories found by the release vulnerability
scan; CI is pinned to it and the container builder downloads that exact
toolchain through Go's automatic toolchain selection. - Both simulator listeners bind to loopback in the runner. Calls outside the
recorded universe fail closed and can be asserted explicitly. - Agent bodies are capped at 16 MiB, MCP messages at 16 MiB, MCP transcripts at
100,000 records, OTel imports at 64 MiB, call/case counts at 10,000, and fault
delays at 60 seconds. JSONPath length/depth and regex length are bounded. - Configuration is strictly decoded. Duplicate IDs, unknown fields, invalid
selectors, unsupported mutations, conflicting terminal faults, and an
ambiguous-response fault without a matching declared effect are rejected. - Proofs expose hashes, operation names, counts, assertion results, and
divergence reasons—not captured bodies, prompt text, or raw tool values.
Bounded child stdout/stderr is stored only in owner-only JSON and should
still be treated as sensitive application output. - Case execution has a bounded timeout and process-output limit. Incident data
never supplies executable commands. - The release artifact contract is unchanged: eight platform archives plus
checksums.txtare uploaded by GoReleaser. Benchmark JSON, reports, fixtures,
and internal GoReleaser metadata are not release assets.
Deliberate boundaries
- v4 explores the baseline plus one configured fault at a time. It does not
claim combinatorial search or formal verification. - The first recognized tool call in a provider response is independently
selectable. Parallel multi-tool envelopes are replayable but not yet
individually fault-addressable within one response. - MCP stdio support targets newline-delimited JSON-RPC. Other stdio framing and
binary MCP transports are not claimed. - OpenTelemetry support is an offline OTLP JSON/JSONL importer, not a live OTLP
collector. - Provider adapters recognize envelopes; they do not replace an LLM server or
make live model sampling deterministic. - Existing gRPC compression, reflection/remote descriptor, large-body, and
bidirectional-stream branching limitations remain.
Upgrade and documentation
- New guide:
docs/AGENT_RELIABILITY.md - Upgrade notes:
docs/UPGRADING.md - Maintainer gates:
docs/RELEASING.md - Example fixture:
examples/agent-reliability - Safe/unsafe reference loop:
examples/agentlab - Local gates:
scripts/agent-smoke.shandscripts/ollama-smoke.sh
InfernoSIM v3.4.0
InfernoSIM v3.4.0
Status: generally available
v3.4 is a single GA release train. No alpha, beta, v3.5, or v3.6 aliases are
used. Publication is allowed only after the mandatory release workflow passes
on the exact commit being tagged.
v3.4 completes InfernoSIM's local incident-to-test path. A sanitized production
incident can now become an editable container test, an explainably stabilized
matcher configuration, a cross-protocol contract gate, and a deterministic
proof artifact without an InfernoSIM-hosted service.
Install
brew install pranaysparihar/infernosim/infernosimHighlights
infernosim serveruns captured HTTP, HTTPS, HTTP/2, and gRPC dependency
behavior as a standalone simulator with a separate health/reset/status/proof
control API.infernosim testgenproduces readable Testcontainers-Go, Docker Compose, or
GitHub Actions harnesses. A maintained Testcontainers-Go adapter is shipped
as an independent module underintegrations/testcontainers-go.infernosim healinfers narrow semantic matchers from repeated observations,
validates candidates on a held-out observation, protects security/business
fields, hashes evidence, and refuses ambiguous proposals.- Kafka-compatible capture and replay preserve topic, partition, offset, key,
headers, payload, schema name, correlation ID, timestamp, and payload hash. - Kafka connections support TLS, mTLS, SASL/PLAIN, SCRAM-SHA-256, and
SCRAM-SHA-512 without accepting passwords on the command line. - Deterministic Kafka delay, drop, duplicate, poison, and reorder plans make
asynchronous failure tests repeatable. - AsyncAPI 3.x validates JSON payload schemas, message/channel references,
required fields, types, enums, patterns, and additional-property drift. - Explicit workflows verify ordered HTTP, gRPC, and Kafka observations with
optional correlation and per-step timing bounds. - HTTP and Kafka captures share configurable deterministic tokenization,
redaction, and drop rules. - Simulator and Kafka proof JSON records semantic fingerprints; workflow,
AsyncAPI, and Kafka commands emit JUnit, SARIF, and HTML reports.
Safety and compatibility
- Existing incidents and replay configuration remain valid. The new
workflowssection is optional and strictly validated. - Healing writes
replay.proposed.yaml; it never silently replaces
replay.yaml. Explicit--applypromotion createsreplay.yaml.bakfirst. - Authorization, credentials, tenant/account boundaries, permissions, money,
status, and personal-data fields cannot be automatically relaxed. - UUIDs and timestamps are relaxed only at allowlisted volatile locations such
as request, trace, correlation, nonce, or timestamp fields; identity IDs stay
exact. Regeneration removes stale InfernoSIM-managed healing rules. - A proposal that makes distinct recorded responses match the same request
fails without writing configuration. - Control APIs do not expose captured bodies, keys, or headers.
- Generated incident files remain owner-only on the host and are copied
read-only into the isolated, unprivileged container. - Kafka capture subscribes only to topics explicitly selected by the user.
- Kafka capture requires a privacy policy unless raw sensitive-data storage is
explicitly enabled. Replayable policy-based capture also requires
capture_bodies: true. - Invalid report formats, topics, authentication combinations, non-finite
timing/confidence values, duplicate message IDs, duplicate privacy rules,
unsupported AsyncAPI schema features, and unsafe generated paths fail before
network side effects. - Homebrew publication does not require a cross-repository personal access
token. The tap's own scheduled/manual workflow reads the latest public
release and publishes the source-built formula with its repository-scoped
GITHUB_TOKEN; a maintainer can also update it from an authenticated local
session.
Releases upload only eight platform archives pluschecksums.txt;
benchmark/report JSON is excluded.
Validation evidence
- The checked-in category baseline completed 100 independent heal/testgen runs
with the expected accept/reject behavior, one configuration hash, one harness
hash, zero ambiguities, and zero seeded-secret leaks. On the recorded local
run, healing p95 was 0.896 ms and test generation p95 was 0.607 ms. Raw
results are inbenchmarks/results/infernosim.json; timing is environment
specific and is not a competitor claim. - Root and nested-module race tests, module consistency, vet, and reachable
vulnerability analysis pass. The vulnerability scan reports zero reachable
vulnerabilities. - The Kafka CLI wire test passes capture → AsyncAPI validation → prefixed replay
→ consume and verifies JUnit, SARIF, HTML, and proof files. - The production Dockerfile builds successfully, runs unprivileged, exposes the
proxy/control ports, and passes the real Testcontainers lifecycle/proxy/reset
test. Node and Go Compose smoke profiles pass against that image. - Targeted matcher, gRPC, template, OpenAPI, bundle, healer, and message fuzz
smoke tests pass. GoReleaser builds all eight platform archives and a
checksum manifest; the source-built Homebrew formula passesbrew test.
The upload-asset assertion reports nine project assets, which GitHub displays
as 11 after its two automatic source downloads. - The Docker smoke passes capture → validation → replay → consume against
Redpanda v25.2.9 using a pinned multi-architecture image digest and separate
internal/external listeners. This remains a mandatory release-workflow gate.
Deliberate boundaries
- v3.4 supports Kafka-compatible brokers, not RabbitMQ, NATS, MQTT, SQS, or SNS.
- AsyncAPI validation covers JSON payloads. Avro, Schema Registry, and remote
schema resolution are not implemented. - Kafka capture uses a consumer subscription rather than a transparent Kafka
protocol proxy. - Healing is deterministic rule inference rather than an LLM and declines
values it cannot classify safely. - Existing gRPC compression, reflection/remote descriptor, large-body, and
bidirectional-stream branching limitations remain.
v3.3.0
v3.2.0
Changelog
- e9ce7c4 First commit
- eced5d6 Merge pull request #1 from pranaysparihar/feature/test
- a6ffbef Merge pull request #3 from pranaysparihar/feature/https
- b411f69 Release InfernoSIM v3.2 and v3.3
- 8a4a3bb Unify feature branch with main branch logic preserving stubproxy and timectl
- 45ed56c Update Readme.md
- c02426d Update Readme.md
- f363800 Update Readme.md
- 4e33fdc Update SECURITY.md
- 50aa7e8 cleanup
- 1594303 compose-fix
- 54f7337 load injection
- e30df1e load injection
- f3d91f8 load injection
- 23088de load injection
- 48cef79 more tests
- 8b88a0f new features
- 3876325 reaply.go
- c886f32 reaply.go-working
- 053d27a replay basics
- 1a502c1 tests
- e51e395 tests introduced
v3.0.1
Changelog
- 658a813 add homebrew tap configuration
v3.0.0
Changelog
- e9ce7c4 First commit
- eced5d6 Merge pull request #1 from pranaysparihar/feature/test
- a6ffbef Merge pull request #3 from pranaysparihar/feature/https
- 8a4a3bb Unify feature branch with main branch logic preserving stubproxy and timectl
- 45ed56c Update Readme.md
- f363800 Update Readme.md
- c02426d Update Readme.md
- 4e33fdc Update SECURITY.md
- 387047f chore: fix goreleaser config and fixtures for v3 release
- 50aa7e8 cleanup
- 1594303 compose-fix
- 5041641 fix: linux/386 build by using unix package for getsockopt
- 4dbaa07 infernosim v3 release candidate
- e30df1e load injection
- 23088de load injection
- f3d91f8 load injection
- 54f7337 load injection
- 48cef79 more tests
- 8b88a0f new features
- 3876325 reaply.go
- c886f32 reaply.go-working
- 27cdfc3 release: infernosim v3
- 6fd1d35 release: infernosim v3
- 053d27a replay basics
- 8b9f5d8 some-def
- 1a502c1 tests
- e51e395 tests introduced
- 77baa0d v3