Skip to content

Releases: BotResources/br-e2e-harness

conformance-passport 0.1.1

Choose a tag to compare

@github-actions github-actions released this 20 Oct 10:22
b74dc97

0.1.1

Fixed

  • The Go reference subject listened on 0.0.0.0 and answered /internal/passport without authentication on every interface of the test machine. It binds 127.0.0.1 only (listenAddr in conformance-subjects/identity-passport), pinned by a Go test and by the the_subject_does_not_answer_on_a_non_loopback_address real-infra test, which fails on the old binary.

  • P8 passed when the call after the bucket loss failed (refused connection, crash, or no answer): it proved no answer, not a 5xx. P8 now passes only on a 5xx received within 10 seconds; any transport error fails the check, and the observed text carries the full error chain (operation timed out, Connection refused), so a hang and a crash read differently; this applies to the failure text of every check that calls the endpoint. The README contract and the P8 row say so.

  • run_spawn returns ConformanceError::InvalidInput when a scenario is listed twice, before touching any infra; a repeated p8 seeded into the bucket the first one deleted and panicked in the harness, losing the whole report.

  • PassportHarness::start_with_binary and run_spawn return ConformanceError::BinaryNotExecutable { path } for a caller-supplied path that is not an executable file (the message names that path: build it and mark it executable), instead of a panic when the child is spawned.

  • The tests never compile anything. The identity-passport binary is looked up on PATH, then in the cache directory $XDG_CACHE_HOME/br-e2e-harness/conformance-passport-{version}, then downloaded from this crate's own release (conformance-passport/v{version}, asset identity-passport-<os>-<arch>.tar.gz with its .sha256; linux and darwin, amd64 and arm64), failing closed on a SHA-256 mismatch, a redirect to a host other than github.com / objects.githubusercontent.com / release-assets.githubusercontent.com, or an archive over 64 MiB. A binary found nowhere is ConformanceError::SubjectBinaryUnavailable, naming the binary and what to do. PassportHarness::start uses this lookup instead of building the subject.

  • The crate writes nothing to the system temporary directory: the former build output is gone, and the scratch directories of the tests live under the test target directory. The one exception is outside the crate: every PassportHarness::start* starts a br-test-harness NATS server whose store br-test-harness creates in the system temporary directory.

  • PassportHarness::start_with_binary and run_spawn resolve a bare program name (identity-passport) on PATH, as spawning it directly did in 0.1.0, and hand the absolute result to the child. A bare name found nowhere is ConformanceError::BinaryNotOnPath { name }, which names the caller's binary and not identity-passport; only a path with a directory part can be BinaryNotExecutable. The bare-name test is #[ignore] (real-infra: it starts nats-server) and its child run asserts it really ran.

  • run_spawn returns ConformanceError::InvalidInput when timeout cannot be added to the current instant (for example Duration::MAX), before touching any infra; it used to start the broker and the subject, then panic in the harness wait.

  • run_spawn reports a subject that never answers /readyz as timed out waiting for the passport subject to report /readyz=200; logs: ...; the message used to read timed out waiting for the passport subject never reported /readyz=200.

Removed

  • The public items of the build module, removed with the module: build_subject, ensure_go_available, subject_dir and run_retired_model_guard. The crate no longer runs the Go toolchain or make, and no Go-toolchain check remains in it; the retired-model guard, the vet and the Go tests of the subject run in CI.
  • ConformanceError::GoUnavailable and ConformanceError::Build.

Added

  • The companion CLI conformance-passport-cli (binary conformance-passport) runs the battery against a service binary in any language and ships as release binaries of this library's tag; the README has a "Run in CI" section for it.
  • ConformanceError::BinaryNotOnPath { name }: a caller-supplied bare program name that is not on PATH.
  • locate_subject() and SUBJECT_BINARY (the module binary) and ConformanceError::SubjectBinaryUnavailable { binary, detail }: the lookup above, SubjectBinaryUnavailable is returned only by locate_subject() (PATH, cache, download).
  • ConformanceError::BinaryNotExecutable { path }: a caller-supplied binary path that is not an executable file.

Changed

  • The identity-passport Go anchor is published as release assets identity-passport-<os>-<arch>.tar.gz (with .sha256) of each conformance-passport release by the release-anchor-passport workflow, built with CGO_ENABLED=0; the linux builds are asserted static. CI builds the anchor once onto PATH and runs the battery against it; .github/scripts/go-free-gate.sh fails CI if a conformance crate named in GO_FREE_CRATES (this one and its CLI) holds, in a *.rs, *.toml, *.sh or *.yml file (not markdown), go or make followed by build, run, install, test, vet, generate or -C, a program literal "go" or "make" (also ending a path), ensure_go_available, or, in a *.rs file anywhere in the crate, temp_dir, tempdir or tempfile; a scan that errors fails the gate. Not detected: a program chosen through a variable, and a Rust build through cargo or rustc.
  • Subject::spawn hands the subject a PATH led by the directory of its binary, through the child's environment, when that directory is absolute, is valid UTF-8 and contains no :; otherwise the child inherits PATH unchanged.
  • Pins br-test-harness/v0.1.2; the br-rust-common pins stay on their latest tags (0.1.0).
  • New dependency sha2 to verify the release asset.

br-test-harness 0.1.2

Choose a tag to compare

@github-actions github-actions released this 17 Oct 12:01
b910515

0.1.2

Fixed

  • GraphqlClient::query, query_unauthenticated, query_with_passport_header, post_raw and get_raw panic with graphql response body read failed when the response body cannot be read (a body cut off by the server), instead of turning it into an empty string. query no longer reports a cut-off body as graphql response not JSON; that panic now carries the parse error and the body.
  • EventCapture and CommandCapture panic with fabric capture drain failed on count(), first(), for_correlation() and correlation_ids() when their pull stream has ended (client drained, connection closed) or the NATS connection is not connected (the broker died), instead of reporting count() == 0 and first() == None; a "no event / no command was published" assertion no longer passes on a dead or disconnected broker. The panic reads fabric capture drain failed: the NATS connection is Disconnected.
  • await_integration_event returns None only when the deadline passes on a connected client. A missing stream, a consumer that cannot be created, a pull stream that ends or errors, a payload that is not JSON, and a deadline that passes on a dead or disconnected broker panic; they were all None, so a "no event arrived" assertion passed on a broken broker.
  • FabricTestNats::await_event (FabricAwaiter), FabricTestNats::await_command (CommandAwaiter) and DurableTap (next_within, expect_quiet, deliveries_within) panic with fabric capture drain failed: the NATS connection is Disconnected when their deadline passes on a dead or disconnected broker, instead of returning None or Timeout, so expect_quiet no longer passes on a dead broker. FabricAwaiter::new keeps its signature; an awaiter built with it directly has no connection to watch, only the one FabricTestNats::await_event returns does.
  • BareFabricNats::{command_stream_absent, event_stream_absent, published_language_absent} and FabricTestNats::{raw_message_absent, durable_filter_subjects_if_present, fixed_streams_present, published_language_present, ephemeral_auth_present} read "absent" only from the broker's own "not found" answer. A dead or timed-out broker panics; any error used to read as absent, so an absence assertion passed on a dead broker.
  • WsSubscription returns WsError::ErrorFrame for a next frame whose payload carries errors and WsError::Transport for one with no data key, instead of Ok(Null); an explicit data: null is still a push. Behaviour change: a next frame carrying both data and errors used to return Ok(data) and now returns ErrorFrame, so a consumer that relied on partial results must handle the error.
  • recreate_stream and recreate_kv ignore only the broker's stream-not-found answer on the delete; any other delete failure is retried in the bounded loop and then panics, instead of the create silently returning the old stream or bucket with its data.
  • EventCapture and CommandCapture also treat a drain task that finished without recording a cause as a dead drain (fabric capture drain failed: the drain task stopped without recording a cause), and stop() panics when the drain task itself panicked, instead of reading count() == 0.
  • verdict::is_ack no longer acks a response whose errors member is present but not an array ("errors": "boom"); only null, an absent member and an empty array mean no errors.
  • SpawnedNats, SpawnedProcess::spawn and run_once start a small sh guard that kills the nats-server or the spawned program (and, for SpawnedNats, removes its store directory) when the test process dies without running destructors (SIGKILL of the process or of its whole process group, a timeout kill); the guard runs in its own process group so a group kill does not take it down, and, for SpawnedNats, it removes the store directory whether or not the server is still alive; kill_on_drop alone left the server, or the service binary under test with its ports, pools and durable consumers, running after a killed suite (Unix only).
  • EventCapture and CommandCapture reads of a dead drain all panic with fabric capture drain failed; the first panic no longer poisons the capture mutex so later reads do not panic with capture mutex poisoned.
  • SpawnedNats start failures show a failed read of the server's stderr as <stderr read failed: ...> instead of an empty text.
  • SseSubscription panics on a next frame whose data is missing or not JSON, whose payload is not a JSON object or has no data key ({}, {"foo":1}, null, 42, "x"), like the WebSocket twin, instead of skipping it as if nothing was pushed (expect_silence passed while the service pushed a broken frame); an explicit data: null is still skipped. The data: lines of one frame are joined with a newline as the SSE format defines.
  • EventCapture and CommandCapture stop their drain on a message that is not an envelope with metadata.correlation_id; every later read panics with fabric capture drain failed: {subject}: payload is not an envelope ... instead of skipping the message and reporting count() == 0. The capture replays the whole stream (deliver-all): on an attached shared broker, junk left on that coordinate by an earlier scenario makes the capture panic; clear it with purge_command_subject / purge_event_subject. CommandAwaiter::await_correlation panics on a non-envelope payload on its subject for the same reason.
  • The fabric-nats CLI exits 3 with the cause of the failure (the panic text) instead of the fixed text failed to connect to NATS; a provisioning failure on a reachable broker no longer reads as a connection failure. provision runs its buckets and durables under the same exit 3 (a refused durable creation names the durable) and verify runs its bucket inventory under it (a dead broker names the address), where both used to exit 101 with a raw panic.
  • EventCapture, CommandCapture and CommandAwaiter keep UnwindSafe and RefUnwindSafe (their connection handle is held behind AssertUnwindSafe), so the auto traits of the public handles are unchanged from 0.1.1; tests/auto_traits.rs pins them.
  • TestServer::spawn panics when the server does not answer within 5 seconds (it used to return anyway), including a / handler that never resolves, and its serve task panics with test server stopped serving instead of ending silently.
  • SpawnedProcess records a failed read of the child's pipe in logs() ([harness] log capture stopped) and keeps appending to, and logs() keeps reading, a poisoned log mutex, instead of dropping output without a trace.
  • E2eDatabase teardown prints a warning when a REVOKE fails, like its other teardown steps.
  • fabric-nats verify exits 4 with the broker's text (probing durable ... : the broker did not answer "absent") when the durable probe gets a broker error, instead of reading it as an absent durable; exit 3 means connect or provisioning failed. The README exit-code line says so.
  • The copy-me TestContext::kv_get in the README panics on a KV transport error or a non-JSON value instead of reading both as an absent key.
  • TestServer::spawn is covered by tests/server.rs: a router whose handler panics and a router whose handler never resolves both make the spawn panic after 5 seconds; a healthy router returns its base URL.
  • WsSubscription returns WsError::Transport for a binary frame, for a text frame whose type is missing or unknown (nxt), instead of skipping it as if nothing was pushed (Err(Timeout), so a "nothing was pushed" assertion passed while the service pushed a broken frame); pong and connection_ack stay ignored.
  • SseSubscription panics on a block that carries data: lines under an event: that is not next or complete (event: message, a data-only block, event: nxt), instead of reading it as no push; comment-only blocks, blocks without data: and complete stay ignored.

br-test-harness 0.1.1

Choose a tag to compare

@github-actions github-actions released this 15 Oct 19:22
7d104af

0.1.1

Fixed

  • verdict::is_ack (and so expect_ack) is true only for a response carrying a non-empty data object; {}, {"data": null}, a JSON string (the non-JSON body of GraphqlClient::post_raw and query_with_passport_header) and an HTTP error page no longer ack. expect_rejected no longer says "got ack" when there is no rejection code.
  • SseSubscription frames on CRLF, CR and LF line ends, and carries a CRLF split across two chunks and a multi-byte character split across two chunks. A server ending lines with CRLF was never decoded: next_outcome read Timeout while events arrived and expect_silence passed.
  • SpawnedNats binds nats-server to 127.0.0.1 (-a 127.0.0.1); the unauthenticated JetStream server no longer listens on every interface of the machine.
  • CommandAwaiter::await_correlation panics when its subscription has ended, like the event awaiter, instead of returning the None of a passed deadline; a "no command was sent" assertion no longer passes on a dead subscription.

oidc-test-idp 0.7.0

Choose a tag to compare

@github-actions github-actions released this 14 Oct 14:14
f37c3ec

0.7.0

Changed

  • First registry release of oidc-test-idp as its own library, tagged oidc-test-idp/v0.7.0. The library is imported at 0.7.0: the code is the one shipped by the workspace release v1.2.1; what changes is the versioning: the single workspace version and the single vX.Y.Z tag are gone, and this crate now carries its own version line.
  • Container image: ghcr.io/botresources/br-oidc-test-idp:0.7.0, published from the oidc-test-idp/v0.7.0 tag to the existing package. The earlier tags of the package (0.1.0 to 0.6.0, 1.0.0 to 1.2.1) are images of earlier releases of the harness line and are untouched. Moving from :1.2.1, change the tag: the number goes down because the library line is 0.7.
  • Dependency pin: { git = "https://github.com/BotResources/br-e2e-harness", package = "oidc-test-idp", tag = "oidc-test-idp/v0.7.0", version = "0.7.0" }.

conformance-scope 0.1.0

Choose a tag to compare

@github-actions github-actions released this 14 Oct 15:06
e5d06ab

0.1.0

Changed

  • First registry release of conformance-scope as its own library, tagged conformance-scope/v0.1.0. The crate is the black-box conformance runner for the scope-declaration wire handshake (batteries S1-S6), reusable against any service binary through start_with_binary. It is a test fixture only. The code is the one shipped by the workspace release v1.2.1; what changes is the versioning: the single workspace version and the single vX.Y.Z tag are gone, and this crate now carries its own version line, 0.1. The earlier v1.x workspace tags are the legacy line and are untouched.
  • Dependency pin: { git = "https://github.com/BotResources/br-e2e-harness", package = "conformance-scope", tag = "conformance-scope/v0.1.0", version = "0.1.0" }.
  • The br-rust-common dependencies are pinned to the per-crate 0.1.0 tags (br-core-integration, br-core-scope, br-scope-declaration-contract, br-util-nats-fabric), each {crate}/v0.1.0.
  • br-test-harness is pinned by its own tag, { git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0", default-features = false, features = ["nats-fabric"] }, the same git url and tag string a consumer uses, so a consumer that also depends on br-test-harness resolves a single copy of its types.

conformance-passport 0.1.0

Choose a tag to compare

@github-actions github-actions released this 14 Oct 15:03
e5d06ab

0.1.0

Changed

  • First registry release of conformance-passport as its own library, tagged conformance-passport/v0.1.0. The crate is the black-box conformance runner for the bearer/PAT to Passport resolution endpoint (battery G1, GET /internal/passport), seeded from the committed Go-frozen wire vectors. It is a test fixture only. The code is the one shipped by the workspace release v1.2.1; what changes is the versioning: the single workspace version and the single vX.Y.Z tag are gone, and this crate now carries its own version line, 0.1. The earlier v1.x workspace tags are the legacy line and are untouched.
  • Dependency pin: { git = "https://github.com/BotResources/br-e2e-harness", package = "conformance-passport", tag = "conformance-passport/v0.1.0", version = "0.1.0" }.
  • The br-rust-common dependencies are pinned to the per-crate 0.1.0 tags (br-core-auth, br-util-nats-fabric), each {crate}/v0.1.0.
  • br-test-harness is pinned by its own tag, { git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0", default-features = false, features = ["nats-fabric"] }, the same git url and tag string a consumer uses, so a consumer that also depends on br-test-harness resolves a single copy of its types.

conformance-nats-fabric 0.1.0

Choose a tag to compare

@github-actions github-actions released this 14 Oct 15:01
e5d06ab

0.1.0

Changed

  • First registry release of conformance-nats-fabric as its own library, tagged conformance-nats-fabric/v0.1.0. The crate is the black-box conformance runner for the v1 integration.* subject grammar and the PUBLISHED_LANGUAGE KV, anchored against an independent Go renderer. It is a test fixture only. The code is the one shipped by the workspace release v1.2.1; what changes is the versioning: the single workspace version and the single vX.Y.Z tag are gone, and this crate now carries its own version line, 0.1. The earlier v1.x workspace tags are the legacy line and are untouched.
  • Dependency pin: { git = "https://github.com/BotResources/br-e2e-harness", package = "conformance-nats-fabric", tag = "conformance-nats-fabric/v0.1.0", version = "0.1.0" }.
  • The br-rust-common dependencies are pinned to the per-crate 0.1.0 tags (br-core-directory, br-core-integration, br-util-nats-fabric), each {crate}/v0.1.0.
  • br-test-harness is pinned by its own tag, { git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0", default-features = false, features = ["nats-fabric"] }, the same git url and tag string a consumer uses, so a consumer that also depends on br-test-harness resolves a single copy of its types.

conformance-identity 0.1.0

Choose a tag to compare

@github-actions github-actions released this 14 Oct 14:59
e5d06ab

0.1.0

Changed

  • First registry release of conformance-identity as its own library, tagged conformance-identity/v0.1.0. The crate is the black-box conformance runner for the scope-acceptance side of the scope-declaration handshake (battery G2): it drives an Identity registry against the frozen Go declaring subject. It is a test fixture only. The code is the one shipped by the workspace release v1.2.1; what changes is the versioning: the single workspace version and the single vX.Y.Z tag are gone, and this crate now carries its own version line, 0.1. The earlier v1.x workspace tags are the legacy line and are untouched.
  • Dependency pin: { git = "https://github.com/BotResources/br-e2e-harness", package = "conformance-identity", tag = "conformance-identity/v0.1.0", version = "0.1.0" }.
  • The br-rust-common dependencies are pinned to the per-crate 0.1.0 tags (br-core-integration, br-core-scope, br-identity-domain, br-scope-declaration-contract, br-util-nats-fabric), each {crate}/v0.1.0.
  • br-test-harness is pinned by its own tag, { git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0", default-features = false, features = ["nats", "spawned-nats", "nats-fabric"] }, the same git url and tag string a consumer uses, so a consumer that also depends on br-test-harness resolves a single copy of its types.

conformance-directory 0.1.0

Choose a tag to compare

@github-actions github-actions released this 14 Oct 14:57
e5d06ab

0.1.0

Changed

  • First registry release of conformance-directory as its own library, tagged conformance-directory/v0.1.0. The crate is the black-box conformance runner for the Identity published-language (directory) wire: the Px publisher and Cx consumer batteries on real NATS and Postgres, plus the offline wire-deserialization gate on br-core-directory. It is a test fixture only. The code is the one shipped by the workspace release v1.2.1; what changes is the versioning: the single workspace version and the single vX.Y.Z tag are gone, and this crate now carries its own version line, 0.1. The earlier v1.x workspace tags are the legacy line and are untouched.
  • Dependency pin: { git = "https://github.com/BotResources/br-e2e-harness", package = "conformance-directory", tag = "conformance-directory/v0.1.0", version = "0.1.0" }.
  • The br-rust-common dependencies are pinned to the per-crate 0.1.0 tags (br-core-directory, br-util-directory, br-util-nats-fabric), each {crate}/v0.1.0.
  • br-test-harness is pinned by its own tag, { git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0", default-features = false, features = ["nats-fabric", "e2e-db"] }, the same git url and tag string a consumer uses, so a consumer that also depends on br-test-harness resolves a single copy of its types.

br-test-harness 0.1.0

Choose a tag to compare

@github-actions github-actions released this 14 Oct 14:33
c8a1819

0.1.0

Changed

  • First registry release of br-test-harness as its own library, tagged br-test-harness/v0.1.0. The crate is the real-infrastructure end-to-end test harness of the platform: real Postgres and NATS (spawned or external), a spawned service binary, forged Passports, GraphQL, SSE and WebSocket clients, the mutation verdict and the fabric-nats CLI, with oidc-test-idp re-exported in-process behind the oidc feature. It is a test fixture only. The code is the one shipped by the workspace release v1.2.1; what changes is the versioning: the single workspace version and the single vX.Y.Z tag are gone, and this crate now carries its own version line, 0.1. The earlier v1.x workspace tags are the legacy line and are untouched.
  • Dependency pin: { git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0" }.
  • The br-rust-common dependencies are pinned to the per-crate 0.1.0 tags (br-core-auth, br-util-nats-fabric, br-core-integration, br-core-directory, br-scope-declaration-contract), each {crate}/v0.1.0.
  • The oidc feature depends on oidc-test-idp through its git tag oidc-test-idp/v0.7.0, the same git url and tag string a consumer uses, so a consumer that also depends on oidc-test-idp resolves a single copy of its types.