Repository navigation
proef v0.15.0
Published retroactively on 2026-09-09. This version was released on
2026-08-25 — the release commit is onmainand the changelog section below
was cut then — but itsv0.15.0tag was never pushed, so the release
pipeline never ran for it. This release therefore carries no binaries,
checksum sidecars or build attestations; every other release has all
three. Install 0.15.0 from source, or use a later version — 0.16.0
superseded this one six days later.
Breaking
- A quarantined test-failure reaches JUnit as
<skipped>with a message,
not<failure>— Jenkins marked builds UNSTABLE while proef exited 0;
every dashboard now says what the exit code says (ADR-0019). Library:
ScenarioSpecgainsskip,ScenarioOutcome/ScenarioRungain
reason,Event::ScenarioFinishedgains additivereason,
write_junittakes the non-gating list. --shardassignments re-deal: the hash gained a mixing finalizer. Raw
FNV-1a's low bit is the XOR-parity of the input bytes, so a scenario named
after its feature file — the commonest Gherkin convention — collapsed to
one shard atN=2and left odd shards empty atN=4, silently (the empty
shard exits 0).shard_bucketnow finalizes with Murmur3'sfmix64; every
scenario re-buckets, so all jobs of one matrix must run the same proef
version (already true in practice). Round-18 finding, reproduced and
mechanism-verified before fixing; the balance test gained the
name-mirrors-file corpus it was structurally blind to.- Tag atoms glob.
*and?in a--tags/[run] exclusive-tagsatom
are now anchored wildcards (@FRD-*selects the family;?is one
character) — previously they were literal characters that silently matched
nothing, the trap this closes. Metacharacter-free atoms are bit-identical
to before, property-pinned. Case stays sensitive. - JUnit test identity is
classname+name.classnamecarries the
feature file,namethe scenario alone; the old singlenameembedded
file:line, so an edit above a scenario re-identified every test below it
in Jenkins history and GitLab's MR diff. Anything keyed on the old
file:line namestrings must re-key. The suiteskippedcount is now
spelledskipped(wasdisabled, which no consumer reads).
Added
-
[tag-links]turns tag cells into tracker links (RF's
--tagstatlink, reduced to one mechanism): tag glob → URL template with
{tag}substituted, honored by the HTML report's by-tag table and the
GitHub summary; the pattern language is the same anchored glob--tags
uses. Library (Breaking):render_htmltakes the link map;
tags::atom_matches_publicexposes the one matcher. -
--console dotted|quiet(RF wave 3): one glyph per scenario (.
pass,Ffail,sskip,wwarn — lowercase is non-gating, the
pytest/RF convention, flushed per glyph, wrapped at 80) or just the frame.
Purely presentation: the record, every report, the post-pool failure
details and the exit code are identical in every mode;run.logmirrors
the console verbatim, dots included —events.jsonlis the full truth.
Library (Breaking):ConsoleReporter::newtakes aConsoleMode. -
A
--rerunnow produces the one JUnit and the one report that cover
the whole suite (E2's rerun half; Robot Framework'srebot --merge
shape, done as composition): the run head recordsrerun_of, the JUnit
carries the base's not-re-run scenarios as ordinary testcases, and
proef reportoverlays the base into a merged page (banner named, base
timestamps stripped so timelines never mix, rotated-away base degrades
loudly). Exit code and totals stay the rerun's own. -
--meta key=valueand[meta]/[env.<name>.meta]record explicit run
metadata (ADR-0020, RF wave 2): commit, build URL, team — recorded in
the run head, shown by the HTML report, GitHub summary,explain,
diff(which now also warns on cross-env comparisons) and the
--output jsonbody (additive keys). The active--envprofile name and
the--shufflemarker ride the same head. proef never harvests: no git,
no hostname, no CI env sniffing — the shell harvests, proef records.
Everything passes the sink-boundary mask, keys and values both. Library
(Breaking):RunRecord::openandexec::executetake the head inputs. -
Per-tag verdicts in the HTML report and the GitHub summary (RF wave
2): tags now reach the record — additivetagsonscenario_finished
(finished-only: the cancel-skip path emits no start), additive
exclusiveonscenario_started(closes R11-6, the scheduler's own
bool) — and both reports roll them up per tag (suite-only, Warned counts
with passed). Requirement-tagged suites (@FRD-3.1) get their
traceability matrix for free. Tags are deduped at the one accumulation
point (first occurrence wins); the quarantine list is now derived from
the outcomes' own tags — one owner, same behavior, pinned by the exit
suite. Library (Breaking):ScenarioSpec/ScenarioOutcomegaintags. -
@skipand@skip:<reason>park a scenario visibly (ADR-0019, RF
wave 2): counted in every total, reasoned in the console, JUnit, TAP, the
record, the HTML report,explainandflows --output json; the harness
maps it to libtest's ignored flag. All-selected-skipped exits 0; the
empty-selection refusal stays exit 2.--tags "not @skip*"unselects both
spellings; an authored skip is never re-queued by--rerun, anddiff
gives skip transitions their own bucket instead of reading them as fixed. -
flowsshows the feature description. The prose block under
Feature:was parsed and then dropped — the one paragraph written for
exactly the readerflowsserves never reached them. Human output prints
it under the feature header;--output jsonrows gain
featureDescription: string|null(additive). Library:FeatureFilegains
description. -
--shufflere-deals the execution order, seeded by the run id — one
determinism knob for order and fakes alike, so--shuffle --run-id <id>
reproduces an order-dependent failure exactly (Robot Framework's
--randomize, minus the parallel seed it threads separately). Applied
after--shard, so membership never moves; under--watchevery unpinned
rerun re-deals, deliberately. The permutation is version-stable and
pinned. Recording ashuffledmarker in the run head is deferred to the
plannedRunStartedadditions (env/metadata), one wire change instead of
two. -
The failing step's
reproduce: curl …reaches the record. The engine
always computed the redacted curl and the live console always printed it —
and the record dropped it, soexplainand the HTML report knew less than
the console did.StepFinishedgains additivereproduce_hint(absent on
passing steps and every pre-field stream);explainand the report print
it; the sink-boundary mask covers it likedetail. -
README documents every flag the binary exposes, enforced. v0.14.0
shipped--shardand--max-failwith no README mention; the docs gate
gains the flags direction (same vacuity guard as the command half), and the
measured gap — those two plusschema --add-to— is closed. -
JUnit carries what GitLab and Jenkins actually read (R3-6, specced from
GitLab's parser docs and Jenkins'SuiteResult.java):fileon each
testcase (GitLab source linking),timeon suite and root.timestampand
hostnamestay absent deliberately — ignored or substituted by both
consumers, and a hostname would undo R12-1's provenance fix. -
The docs corpus is a website: https://emrecdr.github.io/proef/. mdBook
rendersdocs/on every push tomainthat touches it; the nav is
docs/SUMMARY.md, which the existing docs gates link-check like any other
doc, and the pages workflow refuses a corpus doc that is not on the site.
The cratehomepagepoints there from the next release.
Fixed
-
A failure detail is bounded before it reaches any sink. hurl's rendered
assert error quotes the actual response, so a failed assert on a large body
rode full-size into the record, JUnit, the HTML report and the GitHub
summary at once. The engine now middle-cuts past 40 lines / 8 KiB with a
marker naming the elision; the artifact pointer survives outside the cut,
and the full output is one re-run away (Robot Framework's 40-line rule,
adopted at the boundary where all sinks are covered at once). -
The machine-body contract closes its last two paths: an empty selection
(--scenario/--tagsmatching nothing — loud exit 2 by design) and a
corrupt global-state file both emitted zero stdout bytes under
--output json. -
Identical errors collapse like identical warnings — a broken macro
usually fails to lower everywhere, so the error wall was the more common
fifty-block wall; distinct errors still render separately, and SARIF keeps
every site. -
Injected
[Options]lines respect every section-ending shape. The
section-end move covered one shape of five: an unfenced JSON/XML body after
an author[Options]swallowed the injected lines into invalid hurl (exit
2 on input that worked before), and an entry with an author section but no
response line leaked its pending lines into the next entry, where hurl
parsedretry:as an HTTP header and the artifact validated green. The
section now ends at the first line that could not sit inside it. -
A
#inside abind:value no longer hides the reads after it. The
template probe parsed the value in an unquoted position where#opens a
comment; it now probes the quotedvariable:position bake actually
injects into, so"{{a}} # {{b}}"reports both. -
A setup that fails to load still emits the machine body — the last
terminating path returning zero stdout bytes under--output json. -
SARIF keeps one result per site again. The warning collapse shipped at
the front-end aggregation, which also feeds SARIF — a code-scanning
consumer lost every anchor but the first. The collapse now happens at
console rendering only; SARIF carries all sites, the console one line with
the count. -
Every terminating path emits exactly one machine body (R17-2.3/2.4).
An empty shard printed its prose note as the--output jsonbody —jq
failed on the very path a sharded matrix guarantees one job takes — and a
setup abort printed nothing at all while JUnit carried the failure. The
note now goes to stderr under machine output (and lost a stray-space run);
never-ran paths report ADR-0014's suite-only zeros with the exit code
carrying the verdict. -
A failed teardown reaches JUnit as its own suite (R17-2.5) — #78's
rule made symmetric: a phase appears in the reports when it fails. A gated
pipeline used to read a fully-passing report on an exit-3 run. -
A repeated warning is one warning with a count. One authored mistake in
a macro shared by fifty scenarios rendered fifty times; identical warnings
now collapse to their first occurrence plus "(N sites across the suite)". -
explain's truncated-record fallback counts the suite only — a record
that died mid-setup folded the phase scenario into the totals three lines
above the label saying phases are excluded (ADR-0014). -
bind:values are read by hurl's parser, not a text scan (R17-2.2). A
hurl function ({{newUuid}},{{newDate}}) no longer counts as an unbound
variable — proef refused input stock hurl runs — and a sibling literal bind
whose name sorts earlier now counts as a supplier, since injected
[Options] variable:lines are written and evaluated in name order. The
seam answers the question once:FragmentSupport::template_reads. -
A fragment's own
[Options] variable:lines now evaluate before the
injected ones. Injection used to land at the section head, so the
fragment-supplies-it route the unbound check accepts was assigned too late
to be read at run time — accepted at dry-run, wrong at execution. -
A
{{x}}inside abind:value is validated at--dry-run, not at run
time. hurl templates the injected[Options] variable:line when the entry
runs, so a name nothing supplies used to pass dry-run and die mid-run;
proef::lower::unbound_placeholdernow names the placeholder and the bind key
at lower time, where the capture set is known. What legitimately supplies it:
an earlier step's capture, the fragment's own[Options] variable:, a secret
in scope, or a sibling literal bind whose name sorts earlier (injected lines
are written and evaluated in name order). -
A literal
bind:that shadows an earlier capture is named, not silent.
hurl'svariable:assigns into one shared set, so the bound value replaces
the captured one from that entry on — sometimes intended, so it is a warning:
proef::lower::bind_shadows_capture. A secret bind cannot shadow (it skips
the[Options]path) and draws no warning. -
A failed
[run] setupreaches JUnit, the GitHub summary, and PR
annotations. The abort used to return before the CI-report block, so a job
gating on--junitsaw no file at all — indistinguishable from proef never
running. The reports now carry the setup scenario itself (suite named by the
setup feature file); exit codes are untouched (ADR-0014), and nothing is
fabricated for the pool that never ran.
Internal
- The machine body has one exit.
execute's six terminating paths each
pasted the same empty-body emission; they now return through a single
funnel with the oneemit_machine_bodycall after it, so a new path
cannot forget the contract — and the empty-selection body takes its exit
code from the refusal itself instead of restating it. Post-merge cleanup
pass over the deep-audit cycle; behavior pinned by the existing path tests. - The probe and bake share the whole
variable:line.template_reads
re-spelled the injected[Options] variable:line by hand around the
shared escaper;proef_core::lower::variable_option_linenow builds it for
both, andquote_optionreturns to being private (library-surface swap;
unreleased either way). The section-end flush inbake_entry_optionsalso
drops its fence-branch duplicate — one check covers all shapes — and the
console collapse builds its annotated message without cloning the
diagnostic on the common single-site path. - The canary stopped trusting the index's tail twice over: it skips
prerelease versions (the sparse index is publish-ordered, so a9.0.0-beta
would have become "latest"), and refuses a backport older than the pin by
semver ordering (an8.0.2published after9.0.0would have produced a
green about a downgrade). deny.toml's advisory ignores were dead and are gone. The quick-xml
pair was ignored under "the patched release is unreachable" — the quick-junit
0.7 bump made it reachable and the workspace has been on the patched line;
the stale ignores would also have silenced any new advisory against it.
quick-xmlitself re-pinned to quick-junit 0.7's in-tree copy (=0.41.0,
one lock generation);lsp-serverrides to 0.10,tomlto 1.x.
Documentation
- The corpus tells the truth again, audited claim-by-claim: CONFIG
documents the[env.<name>.run]jobs-only rule a reader used to discover as
a parse error; GETTING-STARTED can produce its own output (it now states the
fixture token its §5 requires, and its reproduce command names the
--secretthe replay needs); EVENTS carries the provenance, totals, and
field facts consumers implement against; TESTING-STRATEGY describes the CI
that exists; RELEASING's gate list predicts CI; TROUBLESHOOTING's exit-1 row
covers the--checkfamily. The#Nin an outline instance is documented
as positional, with the column-placeholder naming that keeps identity stable
across--shardand JUnit history.