Skip to content

v2.2.0 — Test pillar: assertions, CI runner, regression coverage

Choose a tag to compare

@thomas-stegemann thomas-stegemann released this 05 Jul 19:32

v2.2 turns Bowire from "drive a call and look at the response" into a real testing tool. A flow can now carry assertions; bowire test runs those flows headless and emits JUnit / SARIF / HTML so a pipeline fails on a regression; responses can be pinned with snapshot baselines, parameterised with data-driven rows, verified against Pact-style contracts, and stressed with mock fault injection. A new Regression Coverage surface tracks per-method run history, and the variable resolver gains its last three sources — OS keyring, AI re-roll, and per-frame streaming resolution. Underneath, the Mocks + Traffic rails collapse into a single Intercept rail, the reverse proxy moves to YARP, and the pluggable-workbench cut-over (#294 / #306) finishes.

60 commits since v2.1.1. Two breaking changes for embedded hosts and operators: the Mocks + Traffic rails merged into one Intercept rail (state migrates automatically at first paint), and Kuestenlogik.Bowire.Protocol.TacticalApi is now opt-in rather than bundled. See Breaking changes below for the migration paths. Existing workspaces, collections, recordings, benchmarks and flows load identically.

Highlights

Flow Assertions — the expectations engine (#342)

A flow step now carries an expectations list evaluated after the response returns: status, body-path (JSONPath + equals / exists / matches / less-than / …), header, and latency checks. The schema, the runtime FlowExpectationEvaluator, and a UI editor ship together (T1 — the foundation the rest of the Test pillar builds on). Each expectation reports pass/fail with the actual value inline so a failure reads like a test assertion, not a diff.

bowire test — the headless CI runner (#344, #181, #100)

bowire test <flow.json> executes a flow end-to-end against a real backend, evaluates its expectations, and reports through the format your pipeline wants: --report path.html, --junit path.xml, --sarif path.sarif (GitHub Code Scanning), and --annotations (inline ::error on the PR diff). --fail-on any|never, --workspace <dir> (aggregate every flow in a git-native workspace), --env / --env-file / --vars, and --base-url round out the surface. The recording-format test-collection runner (#100) and the flow runner auto-detect the input shape, so one command covers both.

Snapshot testing — capture-once, diff-on-change (#171)

A flow step can carry "snapshot": { "mode": "exact" | "structural", "ignore": ["$.ts", …] }. The first run captures the response as a baseline under __snapshots__/<flow>/<step>.snap.json (checked in next to the flow); every later run diffs against it and fails with the drifted JSON paths. ignore marks dynamic fields (timestamps, generated ids) whose value may vary — their kind is still checked. --update-snapshots re-baselines after an intended change.

Data-driven flow steps — inline / CSV / generator (#174)

A step can carry a data source and run once per row: an inline JSON array, a CSV file (resolved relative to the flow, RFC-4180 quoting), or a deterministic generator (range, or seeded random — same seed, same rows, on every .NET version). Row columns join the {{var}} resolver scope and shadow --env values of the same name. Each row reports as stepId[label] so JUnit / SARIF / HTML group the parameterisation as one step family. Zero-row sources and expansions beyond 100 000 rows fail loudly instead of passing vacuously or hanging CI.

Assertion DSL for collection / recording replay (#180)

The v2.1 test-collection format gains a real assertion DSL so a recording replay can assert structural checks ("status is OK", "body.id matches") the same way a flow does — the shared expectation vocabulary that both the recording runner and the flow runner evaluate.

Contract testing — Pact-style publish / verify (#191)

bowire contract publish turns a recording into a Pact-shaped consumer contract; bowire contract verify replays it against a provider and structural-matches the response, projecting the result into the same run report the test runner emits. The broker integration (--broker-url) is opt-in so nothing egresses by default. Structural body matching reuses the flow runner's FlowSnapshotComparer.

Regression Coverage surface (#343)

The Discover sidebar now carries per-method coverage chips backed by a run-history store (T3): at a glance, which methods have been exercised, when they last passed, and where a regression crept in. Coverage is the third leg of the Test pillar next to assertions (T1) and the CLI runner (T2).

Mock-server fault injection (#170)

A mock can now inject failure instead of only replaying the happy path: added latency, error status codes, dropped connections, and partial / truncated responses — so a client can be tested against 503s, timeouts, slow responses and dropped frames without a bespoke broken server.

Variable resolver Phase 5 — OS keyring, AI re-roll, streaming (#208)

The resolver gains its last three sources. {{keyring.service/account}} reads secrets straight from the OS credential store (Windows Credential Manager / macOS Keychain / libsecret) via the new optional Kuestenlogik.Bowire.Keyring package — local-first, on-demand, scrubbed to *** on export, and available headless via bowire test --keyring. {{ai.*}} vars get a ↻ re-roll button in the resolver preview, plus bowire test --ai-seed <seed> for byte-reproducible CI resolution without a model call. And streaming sends now resolve per outbound frame (WebSocket / gRPC duplex), so {{runtime.now}} / {{ai.*}} / {{keyring.*}} resolve live on every frame instead of only the initial connect.

Mocks + Traffic → a single Intercept rail, now on YARP (#334, #315, #335, #336, #341, #323)

The Mocks and Traffic rails collapse into one Intercept rail with four sub-tabs in a locked order — Captured, Live overrides, Mock servers, Settings — with cross-rail transition CTAs (#335) and an activation empty-state that offers a Reverse-Proxy CTA (#336). The interceptor's reverse-proxy forwarder swaps the hand-rolled HttpClient for YARP IHttpForwarder (#341 / #323), so streaming, upgrades, and header handling match a production proxy. See Breaking changes for the rail-merge migration.

Plugin lifecycle backend (#340)

Every loaded plugin gains a lifecycle surface — Restart / Unload / Load / Reset-storage / Health — behind POST /api/plugins/{id}/lifecycle/{action}, wired into the Settings → Plugins rows so an operator can recycle a misbehaving plugin without restarting the host.

Pluggable field-detector auto-discovery (#345)

Semantic field detectors (the seam the Map extension uses to light up coordinate.wgs84 payloads) are now auto-discovered via the [BowireExtension] attribute — a package drops a detector in and the workbench picks it up, no Core edit.

Workspace-deletion hardening (#337, #348)

Workspace delete now does a cascade purge with a Soft / Hard toggle, and its Undo is decoupled from the Trash drawer: the action-log entry carries the full snapshot inline so Undo reads from the entry itself. The two surfaces get independent retention (Trash: operator-curated days; Action log: last-200 sliding window). Settings → Workspace… expands into four scope-clear sub-pages (#348).

Pluggable workbench, finished — Phase G + Bootcamp (#306, #347)

The #294 pluggable-workbench cut-over completes: every remaining rail / module is a package contribution behind the #314 renderer-key seam, and the last core-resident descriptors (Home / Discover / Workspaces) are consolidated into Core. New Bootcamp lessons (#347) walk operators through Intercept, Flow Assertions, the bowire test CLI, Coverage, plugin lifecycle, and workspace deletion.

Fixes + test infrastructure (#346, #349, #338, #213, #312, #38)

Compose tab Duplicate now deep-copies request state (morphdom stale-closure fix, #346); the Parameter → Header sub-tab preserves Parameter row inputs (#349); rail-load calls are guarded for embedded hosts that don't reference every rail package (#338). The SidecarFake JSON-RPC handshake is finished to unlock SidecarBowireProtocol coverage (#213), a coverage-gap audit kicked off the cycle (#312), and CLI Phase-3 polish landed completion + validators + error rendering (#38). A new JS unit-test net (node --test over the wwwroot fragments) + in-process CLI integration tests guard the workbench and CLI against regressions.

Breaking changes

Mocks + Traffic rails merged into a single Intercept rail (#334)

The Mocks rail and the Traffic rail (which itself unified the earlier Proxy + Intercepted rails) collapse into one Intercept rail with four sub-tabs in a locked order:

  • Captured — passive observation of flows captured by UseBowireInterceptor() (was Traffic → "Flows").
  • Live overrides — selective response substitution inside the interceptor pipeline (was Traffic → "Mock Rules").
  • Mock servers — standalone mock-server-from-recording hosts (was the entire Mocks rail).
  • Settings — interceptor / proxy config; adapts to Standalone vs Embedded deployment.

Migration is automatic at first paint (idempotent). On boot, prologue.js:

  • Rewrites localStorage.bowire_rail_mode from mocks / traffic / proxy / interceptedintercept, and seeds the sub-tab discriminator (bowire_intercept_sub_tab) from the legacy mode (mocksmock-servers; traffic+flowscaptured; traffic+mockslive-overrides; traffic+settingssettings; proxy / interceptedcaptured).
  • Rewrites localStorage.bowire_sidebar_view (mocks / traffic / proxy / interceptedintercept).
  • Collapses any mocks / traffic entry in localStorage.bowire_enabled_rails into a single intercept.

Embedded hosts that referenced the deleted descriptors must update their DI registrations:

  • BowireTrafficRailContributionBowireInterceptRailContribution.
  • BowireProxyRailContribution, BowireInterceptedRailContribution, BowireMocksRailContribution, BowireEnvironmentsRailContribution → no replacement (the Environments surface renders inside Workspaces; Mocks now lives inside the Intercept rail's Mock servers sub-tab).

.bww workspace files are unaffected — they don't persist rail mode.

Kuestenlogik.Bowire.Protocol.TacticalApi retired from Bundle.Workbench

The TacticalApi protocol is now opt-in. Operators who need the Rheinmetall Situation service install it explicitly:

dotnet add package Kuestenlogik.Bowire.Protocol.TacticalApi

or, once the plugin marketplace ships, via Settings → Plugins → Install. Bundle.Workbench stays the universal-web-protocol set (REST, gRPC, GraphQL, MQTT, WebSocket, SSE, MCP, SignalR, JSON-RPC, OData, Socket.IO); domain-specific protocols follow the opt-in pattern. Nothing to do if you didn't use TacticalApi — if you did, a one-line dotnet add package restores the surface. Related upstream fix: Bowire.Protocol.TacticalApi v1.0.4 gates its DiscoverAsync on the tacticalapi@ URL-scheme prefix, so it no longer surfaces Situation methods for unrelated sources like a plain Petstore OpenAPI URL.

Acknowledgements

Thanks to everyone who exercised the v2.2 release candidates and filed issues against the Test-pillar surfaces.


The full commit list, contributors, and compare-URL diff are auto-generated below.

What's Changed

  • chore: bump floor to 2.1.1-dev (after v2.1.1) by @thomas-stegemann in #354
  • chore(npm): Bump sharp from 0.35.2 to 0.35.3 by @dependabot[bot] in #372
  • chore(actions): Bump docker/login-action from 4.3.0 to 4.4.0 by @dependabot[bot] in #374
  • chore(deps): Bump Microsoft.Identity.Web from 4.12.0 to 4.12.2 by @dependabot[bot] in #375
  • chore(deps): Bump Testcontainers from 4.12.0 to 4.13.0 by @dependabot[bot] in #376
  • chore(deps): Bump Testcontainers and Testcontainers.Pulsar by @dependabot[bot] in #378

Full Changelog: v2.1.1...v2.2.0