Repository navigation
Releases: BotResources/br-e2e-harness
Release list
conformance-passport 0.1.1
0.1.1
Fixed
-
The Go reference subject listened on
0.0.0.0and answered/internal/passportwithout authentication on every interface of the test machine. It binds127.0.0.1only (listenAddrinconformance-subjects/identity-passport), pinned by a Go test and by thethe_subject_does_not_answer_on_a_non_loopback_addressreal-infra test, which fails on the old binary. -
P8 passed when the call after the bucket loss failed (refused connection, crash, or no answer): it proved no answer, not a 5xx. P8 now passes only on a 5xx received within 10 seconds; any transport error fails the check, and the observed text carries the full error chain (
operation timed out,Connection refused), so a hang and a crash read differently; this applies to the failure text of every check that calls the endpoint. The README contract and the P8 row say so. -
run_spawnreturnsConformanceError::InvalidInputwhen a scenario is listed twice, before touching any infra; a repeatedp8seeded into the bucket the first one deleted and panicked in the harness, losing the whole report. -
PassportHarness::start_with_binaryandrun_spawnreturnConformanceError::BinaryNotExecutable { path }for a caller-supplied path that is not an executable file (the message names that path: build it and mark it executable), instead of a panic when the child is spawned. -
The tests never compile anything. The
identity-passportbinary is looked up onPATH, then in the cache directory$XDG_CACHE_HOME/br-e2e-harness/conformance-passport-{version}, then downloaded from this crate's own release (conformance-passport/v{version}, assetidentity-passport-<os>-<arch>.tar.gzwith its.sha256;linuxanddarwin,amd64andarm64), failing closed on a SHA-256 mismatch, a redirect to a host other thangithub.com/objects.githubusercontent.com/release-assets.githubusercontent.com, or an archive over 64 MiB. A binary found nowhere isConformanceError::SubjectBinaryUnavailable, naming the binary and what to do.PassportHarness::startuses this lookup instead of building the subject. -
The crate writes nothing to the system temporary directory: the former build output is gone, and the scratch directories of the tests live under the test target directory. The one exception is outside the crate: every
PassportHarness::start*starts abr-test-harnessNATS server whose storebr-test-harnesscreates in the system temporary directory. -
PassportHarness::start_with_binaryandrun_spawnresolve a bare program name (identity-passport) onPATH, as spawning it directly did in 0.1.0, and hand the absolute result to the child. A bare name found nowhere isConformanceError::BinaryNotOnPath { name }, which names the caller's binary and notidentity-passport; only a path with a directory part can beBinaryNotExecutable. The bare-name test is#[ignore](real-infra: it startsnats-server) and its child run asserts it really ran. -
run_spawnreturnsConformanceError::InvalidInputwhentimeoutcannot be added to the current instant (for exampleDuration::MAX), before touching any infra; it used to start the broker and the subject, then panic in the harness wait. -
run_spawnreports a subject that never answers/readyzastimed out waiting for the passport subject to report /readyz=200; logs: ...; the message used to readtimed out waiting for the passport subject never reported /readyz=200.
Removed
- The public items of the
buildmodule, removed with the module:build_subject,ensure_go_available,subject_dirandrun_retired_model_guard. The crate no longer runs the Go toolchain ormake, and no Go-toolchain check remains in it; the retired-model guard, the vet and the Go tests of the subject run in CI. ConformanceError::GoUnavailableandConformanceError::Build.
Added
- The companion CLI
conformance-passport-cli(binaryconformance-passport) runs the battery against a service binary in any language and ships as release binaries of this library's tag; the README has a "Run in CI" section for it. ConformanceError::BinaryNotOnPath { name }: a caller-supplied bare program name that is not onPATH.locate_subject()andSUBJECT_BINARY(the modulebinary) andConformanceError::SubjectBinaryUnavailable { binary, detail }: the lookup above,SubjectBinaryUnavailableis returned only bylocate_subject()(PATH, cache, download).ConformanceError::BinaryNotExecutable { path }: a caller-supplied binary path that is not an executable file.
Changed
- The
identity-passportGo anchor is published as release assetsidentity-passport-<os>-<arch>.tar.gz(with.sha256) of eachconformance-passportrelease by therelease-anchor-passportworkflow, built withCGO_ENABLED=0; the linux builds are asserted static. CI builds the anchor once ontoPATHand runs the battery against it;.github/scripts/go-free-gate.shfails CI if a conformance crate named inGO_FREE_CRATES(this one and its CLI) holds, in a*.rs,*.toml,*.shor*.ymlfile (not markdown),goormakefollowed bybuild,run,install,test,vet,generateor-C, a program literal"go"or"make"(also ending a path),ensure_go_available, or, in a*.rsfile anywhere in the crate,temp_dir,tempdirortempfile; a scan that errors fails the gate. Not detected: a program chosen through a variable, and a Rust build throughcargoorrustc. Subject::spawnhands the subject aPATHled by the directory of its binary, through the child's environment, when that directory is absolute, is valid UTF-8 and contains no:; otherwise the child inheritsPATHunchanged.- Pins
br-test-harness/v0.1.2; thebr-rust-commonpins stay on their latest tags (0.1.0). - New dependency
sha2to verify the release asset.
br-test-harness 0.1.2
0.1.2
Fixed
GraphqlClient::query,query_unauthenticated,query_with_passport_header,post_rawandget_rawpanic withgraphql response body read failedwhen the response body cannot be read (a body cut off by the server), instead of turning it into an empty string.queryno longer reports a cut-off body asgraphql response not JSON; that panic now carries the parse error and the body.EventCaptureandCommandCapturepanic withfabric capture drain failedoncount(),first(),for_correlation()andcorrelation_ids()when their pull stream has ended (client drained, connection closed) or the NATS connection is not connected (the broker died), instead of reportingcount() == 0andfirst() == None; a "no event / no command was published" assertion no longer passes on a dead or disconnected broker. The panic readsfabric capture drain failed: the NATS connection is Disconnected.await_integration_eventreturnsNoneonly when the deadline passes on a connected client. A missing stream, a consumer that cannot be created, a pull stream that ends or errors, a payload that is not JSON, and a deadline that passes on a dead or disconnected broker panic; they were allNone, so a "no event arrived" assertion passed on a broken broker.FabricTestNats::await_event(FabricAwaiter),FabricTestNats::await_command(CommandAwaiter) andDurableTap(next_within,expect_quiet,deliveries_within) panic withfabric capture drain failed: the NATS connection is Disconnectedwhen their deadline passes on a dead or disconnected broker, instead of returningNoneorTimeout, soexpect_quietno longer passes on a dead broker.FabricAwaiter::newkeeps its signature; an awaiter built with it directly has no connection to watch, only the oneFabricTestNats::await_eventreturns does.BareFabricNats::{command_stream_absent, event_stream_absent, published_language_absent}andFabricTestNats::{raw_message_absent, durable_filter_subjects_if_present, fixed_streams_present, published_language_present, ephemeral_auth_present}read "absent" only from the broker's own "not found" answer. A dead or timed-out broker panics; any error used to read as absent, so an absence assertion passed on a dead broker.WsSubscriptionreturnsWsError::ErrorFramefor anextframe whose payload carrieserrorsandWsError::Transportfor one with nodatakey, instead ofOk(Null); an explicitdata: nullis still a push. Behaviour change: anextframe carrying bothdataanderrorsused to returnOk(data)and now returnsErrorFrame, so a consumer that relied on partial results must handle the error.recreate_streamandrecreate_kvignore only the broker's stream-not-found answer on the delete; any other delete failure is retried in the bounded loop and then panics, instead of the create silently returning the old stream or bucket with its data.EventCaptureandCommandCapturealso treat a drain task that finished without recording a cause as a dead drain (fabric capture drain failed: the drain task stopped without recording a cause), andstop()panics when the drain task itself panicked, instead of readingcount() == 0.verdict::is_ackno longer acks a response whoseerrorsmember is present but not an array ("errors": "boom"); onlynull, an absent member and an empty array mean no errors.SpawnedNats,SpawnedProcess::spawnandrun_oncestart a smallshguard that kills thenats-serveror the spawned program (and, forSpawnedNats, removes its store directory) when the test process dies without running destructors (SIGKILL of the process or of its whole process group, a timeout kill); the guard runs in its own process group so a group kill does not take it down, and, forSpawnedNats, it removes the store directory whether or not the server is still alive;kill_on_dropalone left the server, or the service binary under test with its ports, pools and durable consumers, running after a killed suite (Unix only).EventCaptureandCommandCapturereads of a dead drain all panic withfabric capture drain failed; the first panic no longer poisons the capture mutex so later reads do not panic withcapture mutex poisoned.SpawnedNatsstart failures show a failed read of the server's stderr as<stderr read failed: ...>instead of an empty text.SseSubscriptionpanics on anextframe whose data is missing or not JSON, whose payload is not a JSON object or has nodatakey ({},{"foo":1},null,42,"x"), like the WebSocket twin, instead of skipping it as if nothing was pushed (expect_silencepassed while the service pushed a broken frame); an explicitdata: nullis still skipped. Thedata:lines of one frame are joined with a newline as the SSE format defines.EventCaptureandCommandCapturestop their drain on a message that is not an envelope withmetadata.correlation_id; every later read panics withfabric capture drain failed: {subject}: payload is not an envelope ...instead of skipping the message and reportingcount() == 0. The capture replays the whole stream (deliver-all): on an attached shared broker, junk left on that coordinate by an earlier scenario makes the capture panic; clear it withpurge_command_subject/purge_event_subject.CommandAwaiter::await_correlationpanics on a non-envelope payload on its subject for the same reason.- The
fabric-natsCLI exits 3 with the cause of the failure (the panic text) instead of the fixed textfailed to connect to NATS; a provisioning failure on a reachable broker no longer reads as a connection failure.provisionruns its buckets and durables under the same exit 3 (a refused durable creation names the durable) andverifyruns its bucket inventory under it (a dead broker names the address), where both used to exit 101 with a raw panic. EventCapture,CommandCaptureandCommandAwaiterkeepUnwindSafeandRefUnwindSafe(their connection handle is held behindAssertUnwindSafe), so the auto traits of the public handles are unchanged from 0.1.1;tests/auto_traits.rspins them.TestServer::spawnpanics when the server does not answer within 5 seconds (it used to return anyway), including a/handler that never resolves, and its serve task panics withtest server stopped servinginstead of ending silently.SpawnedProcessrecords a failed read of the child's pipe inlogs()([harness] log capture stopped) and keeps appending to, andlogs()keeps reading, a poisoned log mutex, instead of dropping output without a trace.E2eDatabaseteardown prints a warning when aREVOKEfails, like its other teardown steps.fabric-nats verifyexits 4 with the broker's text (probing durable ... : the broker did not answer "absent") when the durable probe gets a broker error, instead of reading it as an absent durable; exit 3 means connect or provisioning failed. The README exit-code line says so.- The copy-me
TestContext::kv_getin the README panics on a KV transport error or a non-JSON value instead of reading both as an absent key. TestServer::spawnis covered bytests/server.rs: a router whose handler panics and a router whose handler never resolves both make the spawn panic after 5 seconds; a healthy router returns its base URL.WsSubscriptionreturnsWsError::Transportfor a binary frame, for a text frame whosetypeis missing or unknown (nxt), instead of skipping it as if nothing was pushed (Err(Timeout), so a "nothing was pushed" assertion passed while the service pushed a broken frame);pongandconnection_ackstay ignored.SseSubscriptionpanics on a block that carriesdata:lines under anevent:that is notnextorcomplete(event: message, a data-only block,event: nxt), instead of reading it as no push; comment-only blocks, blocks withoutdata:andcompletestay ignored.
br-test-harness 0.1.1
0.1.1
Fixed
verdict::is_ack(and soexpect_ack) is true only for a response carrying a non-emptydataobject;{},{"data": null}, a JSON string (the non-JSON body ofGraphqlClient::post_rawandquery_with_passport_header) and an HTTP error page no longer ack.expect_rejectedno longer says "got ack" when there is no rejection code.SseSubscriptionframes on CRLF, CR and LF line ends, and carries a CRLF split across two chunks and a multi-byte character split across two chunks. A server ending lines with CRLF was never decoded:next_outcomereadTimeoutwhile events arrived andexpect_silencepassed.SpawnedNatsbindsnats-serverto127.0.0.1(-a 127.0.0.1); the unauthenticated JetStream server no longer listens on every interface of the machine.CommandAwaiter::await_correlationpanics when its subscription has ended, like the event awaiter, instead of returning theNoneof a passed deadline; a "no command was sent" assertion no longer passes on a dead subscription.
oidc-test-idp 0.7.0
0.7.0
Changed
- First registry release of
oidc-test-idpas its own library, taggedoidc-test-idp/v0.7.0. The library is imported at 0.7.0: the code is the one shipped by the workspace releasev1.2.1; what changes is the versioning: the single workspace version and the singlevX.Y.Ztag are gone, and this crate now carries its own version line. - Container image:
ghcr.io/botresources/br-oidc-test-idp:0.7.0, published from theoidc-test-idp/v0.7.0tag to the existing package. The earlier tags of the package (0.1.0to0.6.0,1.0.0to1.2.1) are images of earlier releases of the harness line and are untouched. Moving from:1.2.1, change the tag: the number goes down because the library line is0.7. - Dependency pin:
{ git = "https://github.com/BotResources/br-e2e-harness", package = "oidc-test-idp", tag = "oidc-test-idp/v0.7.0", version = "0.7.0" }.
conformance-scope 0.1.0
0.1.0
Changed
- First registry release of
conformance-scopeas its own library, taggedconformance-scope/v0.1.0. The crate is the black-box conformance runner for the scope-declaration wire handshake (batteries S1-S6), reusable against any service binary throughstart_with_binary. It is a test fixture only. The code is the one shipped by the workspace releasev1.2.1; what changes is the versioning: the single workspace version and the singlevX.Y.Ztag are gone, and this crate now carries its own version line,0.1. The earlierv1.xworkspace tags are the legacy line and are untouched. - Dependency pin:
{ git = "https://github.com/BotResources/br-e2e-harness", package = "conformance-scope", tag = "conformance-scope/v0.1.0", version = "0.1.0" }. - The
br-rust-commondependencies are pinned to the per-crate0.1.0tags (br-core-integration,br-core-scope,br-scope-declaration-contract,br-util-nats-fabric), each{crate}/v0.1.0. br-test-harnessis pinned by its own tag,{ git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0", default-features = false, features = ["nats-fabric"] }, the same git url and tag string a consumer uses, so a consumer that also depends onbr-test-harnessresolves a single copy of its types.
conformance-passport 0.1.0
0.1.0
Changed
- First registry release of
conformance-passportas its own library, taggedconformance-passport/v0.1.0. The crate is the black-box conformance runner for the bearer/PAT to Passport resolution endpoint (battery G1,GET /internal/passport), seeded from the committed Go-frozen wire vectors. It is a test fixture only. The code is the one shipped by the workspace releasev1.2.1; what changes is the versioning: the single workspace version and the singlevX.Y.Ztag are gone, and this crate now carries its own version line,0.1. The earlierv1.xworkspace tags are the legacy line and are untouched. - Dependency pin:
{ git = "https://github.com/BotResources/br-e2e-harness", package = "conformance-passport", tag = "conformance-passport/v0.1.0", version = "0.1.0" }. - The
br-rust-commondependencies are pinned to the per-crate0.1.0tags (br-core-auth,br-util-nats-fabric), each{crate}/v0.1.0. br-test-harnessis pinned by its own tag,{ git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0", default-features = false, features = ["nats-fabric"] }, the same git url and tag string a consumer uses, so a consumer that also depends onbr-test-harnessresolves a single copy of its types.
conformance-nats-fabric 0.1.0
0.1.0
Changed
- First registry release of
conformance-nats-fabricas its own library, taggedconformance-nats-fabric/v0.1.0. The crate is the black-box conformance runner for the v1integration.*subject grammar and thePUBLISHED_LANGUAGEKV, anchored against an independent Go renderer. It is a test fixture only. The code is the one shipped by the workspace releasev1.2.1; what changes is the versioning: the single workspace version and the singlevX.Y.Ztag are gone, and this crate now carries its own version line,0.1. The earlierv1.xworkspace tags are the legacy line and are untouched. - Dependency pin:
{ git = "https://github.com/BotResources/br-e2e-harness", package = "conformance-nats-fabric", tag = "conformance-nats-fabric/v0.1.0", version = "0.1.0" }. - The
br-rust-commondependencies are pinned to the per-crate0.1.0tags (br-core-directory,br-core-integration,br-util-nats-fabric), each{crate}/v0.1.0. br-test-harnessis pinned by its own tag,{ git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0", default-features = false, features = ["nats-fabric"] }, the same git url and tag string a consumer uses, so a consumer that also depends onbr-test-harnessresolves a single copy of its types.
conformance-identity 0.1.0
0.1.0
Changed
- First registry release of
conformance-identityas its own library, taggedconformance-identity/v0.1.0. The crate is the black-box conformance runner for the scope-acceptance side of the scope-declaration handshake (battery G2): it drives an Identity registry against the frozen Go declaring subject. It is a test fixture only. The code is the one shipped by the workspace releasev1.2.1; what changes is the versioning: the single workspace version and the singlevX.Y.Ztag are gone, and this crate now carries its own version line,0.1. The earlierv1.xworkspace tags are the legacy line and are untouched. - Dependency pin:
{ git = "https://github.com/BotResources/br-e2e-harness", package = "conformance-identity", tag = "conformance-identity/v0.1.0", version = "0.1.0" }. - The
br-rust-commondependencies are pinned to the per-crate0.1.0tags (br-core-integration,br-core-scope,br-identity-domain,br-scope-declaration-contract,br-util-nats-fabric), each{crate}/v0.1.0. br-test-harnessis pinned by its own tag,{ git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0", default-features = false, features = ["nats", "spawned-nats", "nats-fabric"] }, the same git url and tag string a consumer uses, so a consumer that also depends onbr-test-harnessresolves a single copy of its types.
conformance-directory 0.1.0
0.1.0
Changed
- First registry release of
conformance-directoryas its own library, taggedconformance-directory/v0.1.0. The crate is the black-box conformance runner for the Identity published-language (directory) wire: the Px publisher and Cx consumer batteries on real NATS and Postgres, plus the offline wire-deserialization gate onbr-core-directory. It is a test fixture only. The code is the one shipped by the workspace releasev1.2.1; what changes is the versioning: the single workspace version and the singlevX.Y.Ztag are gone, and this crate now carries its own version line,0.1. The earlierv1.xworkspace tags are the legacy line and are untouched. - Dependency pin:
{ git = "https://github.com/BotResources/br-e2e-harness", package = "conformance-directory", tag = "conformance-directory/v0.1.0", version = "0.1.0" }. - The
br-rust-commondependencies are pinned to the per-crate0.1.0tags (br-core-directory,br-util-directory,br-util-nats-fabric), each{crate}/v0.1.0. br-test-harnessis pinned by its own tag,{ git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0", default-features = false, features = ["nats-fabric", "e2e-db"] }, the same git url and tag string a consumer uses, so a consumer that also depends onbr-test-harnessresolves a single copy of its types.
br-test-harness 0.1.0
0.1.0
Changed
- First registry release of
br-test-harnessas its own library, taggedbr-test-harness/v0.1.0. The crate is the real-infrastructure end-to-end test harness of the platform: real Postgres and NATS (spawned or external), a spawned service binary, forged Passports, GraphQL, SSE and WebSocket clients, the mutation verdict and thefabric-natsCLI, withoidc-test-idpre-exported in-process behind theoidcfeature. It is a test fixture only. The code is the one shipped by the workspace releasev1.2.1; what changes is the versioning: the single workspace version and the singlevX.Y.Ztag are gone, and this crate now carries its own version line,0.1. The earlierv1.xworkspace tags are the legacy line and are untouched. - Dependency pin:
{ git = "https://github.com/BotResources/br-e2e-harness", package = "br-test-harness", tag = "br-test-harness/v0.1.0", version = "0.1.0" }. - The
br-rust-commondependencies are pinned to the per-crate0.1.0tags (br-core-auth,br-util-nats-fabric,br-core-integration,br-core-directory,br-scope-declaration-contract), each{crate}/v0.1.0. - The
oidcfeature depends onoidc-test-idpthrough its git tagoidc-test-idp/v0.7.0, the same git url and tag string a consumer uses, so a consumer that also depends onoidc-test-idpresolves a single copy of its types.