Repository navigation
Releases
v0.5.2
Engine: libhegel 0.51.5 (was 0.51.1)
The libhegel 0.51.5 release notes
describe 0.51.5. The move also includes 0.51.2, 0.51.3, and 0.51.4, and the
libhegel changelog at 0.51.5
describes all four releases.
Each platform gem carries libhegel 0.51.5. A caller has nothing to
change. The public methods keep their signatures and return values.
In 0.51.4, the engine shrinks two kinds of recursive values better. A
recursive value whose branches can hold one child can now shrink to a
branch of one child. A value that the engine found after it abandoned
earlier attempts to draw it now shrinks fully. A failure report can so
show a smaller counterexample than before.
In 0.51.5, the engine adds two functions, hegel_pause_faults and
hegel_unpause_faults. These bindings do not call them, and this release
does not add a way to pause faults. 0.51.2 and 0.51.3 change only the
internals of the engine.
v0.5.1
Engine: libhegel 0.51.1 (was 0.50.0)
The libhegel 0.51.1 release notes
describe 0.51.1. The move also includes 0.51.0, and the
libhegel changelog at 0.51.1
describes both releases.
Each platform gem carries libhegel 0.51.1. A caller has nothing to
change. The public methods keep their signatures and return values.
In 0.51.0, the engine lets a state machine redraw its configuration
between rounds, and to do this it changes one function that these
bindings call: hegel_state_machine_next_group takes a new out-parameter
for the concurrency level of the round. These bindings pass it.
Hegel::Stateful.run and Hegel::StateMachine keep a sequential
machine, so the level is always 1. At the start of each round, the engine
now redraws, with probability 1/50, which rules the machine enables and
how it distributes drawn values, as the engine release notes say. A long
run can so explore more than one configuration.
In 0.51.1, a run of one test case (test_cases: 1) inside Antithesis
(ANTITHESIS_OUTPUT_DIR is set) with no database_key does not record
the choices of its test case. Its failure is reported from its one
execution: Hegel.test raises the exception of the body, but the engine
does not shrink or replay the failure, the report shows no drawn values,
and it has no reproduce blob. A run outside Antithesis, or with a
database_key, does not change.
v0.5.0
Engine: libhegel 0.50.0 (was 0.49.0)
The libhegel 0.50.0 release notes
describe 0.50.0.
Each platform gem carries libhegel 0.50.0. The engine changes how a caller
declares a state machine: hegel_new_state_machine now opens an empty
machine, and new calls declare each rule and each invariant and then start
the machine. These bindings make the new calls. A caller of
Hegel::Stateful.run and Hegel::StateMachine has nothing to change, and
gets the same rules, weights, and invariants as before.
Changed
Hegel::TestCase#new_state_machinetakes only the step count. The new
methods#state_machine_add_rule,#state_machine_add_invariant, and
#state_machine_startdeclare the rules and invariants and start the
machine, in the same order as the engine calls.Hegel::Stateful.run
uses them. A caller who called#new_state_machinedirectly must change
to the new methods.
v0.4.1
Engine: libhegel 0.49.0 (was 0.48.0)
The libhegel 0.49.0 release notes
describe 0.49.0.
Each platform gem carries libhegel 0.49.0. The engine adds functions and
constants to its C interface, and changes no function that these bindings
call. A caller has nothing to change in code. The engine move changes these
results for a caller:
- The engine reads new environment variables over the settings profile:
HEGEL_VERBOSITY,HEGEL_BACKEND,HEGEL_PHASES, and
HEGEL_SUPPRESS_HEALTH_CHECK. Each one applies to a run that does not set
the matchingHegel.testkeyword.HEGEL_UNBOUNDED_CHOICESsets a setting
that has noHegel.testkeyword. A malformed value makesHegel.test
raiseHegel::ErrorwithHEGEL_E_INVALID_ARGand a message that names
the variable. In 0.48.0, the engine ignored these variables.HEGEL_REPORT_MULTIPLE_FAILUREShas no
effect, becauseHegel.testalways setsreport_multiple_failures. - The engine reads a
max_sizeof2**64 - 2as "no maximum given". The gem
now sends amax_sizeat or above2**64 - 2as2**64 - 1, no upper
bound, sotext,binary,arrays,sets,hashes, and the other sized
generators keep the meaning the caller wrote. Amax_sizeabove
2**64 - 1also means no upper bound now. Before, it raisedRangeError.
See ADR 0037.
Changed
- The string generators follow the Unicode version Ruby was built with,
RbConfig::CONFIG["UNICODE_VERSION"], through libhegel's
hegel_set_unicode_version. A drawn character falls in the General
Category Ruby gives it, andfrom_regexreads\w,\d, and\sas
Ruby does. Before, they followed the engine's newest tables, Unicode 17.0.0.
On Ruby 3.4, which follows Unicode 15.0.0, a character assigned after 15.0.0
is now drawn as unassigned. See
ADR 0036.
v0.4.0
Engine: libhegel 0.48.0 (was 0.47.2)
The libhegel 0.48.0 release notes
describe 0.48.0.
Each platform gem carries libhegel 0.48.0. The engine adds one function to
its C interface, hegel_set_unicode_version, and changes no other function.
These bindings do not call it yet, so the string generators continue to use
the newest Unicode tables of the engine, as before. A caller has nothing to
change in code. The engine move changes these results for a caller:
- With the default
report_multiple_failures: false, a run can find a
different, smaller failure while it shrinks the first one. The run now
shrinks that failure too before it reports it. In 0.47.2, the run reported
that failure without shrinking it. from_regexaccepts the end-of-string anchor\z. In 0.47.2, a pattern
with\zraisedHegel::ErrorwithHEGEL_E_INVALID_ARG.floats(min_value: Float::INFINITY, allow_infinity: false)and
floats(max_value: -Float::INFINITY, allow_infinity: false)leave no value
to draw. The draw now raisesHegel::ErrorwithHEGEL_E_INVALID_ARGand
a message that contains "leaves no value". In 0.47.2, the draw raised
Hegel::ErrorwithHEGEL_E_INTERNAL.
Profiles
Hegel.test takes profile:, the name of a settings profile to start the
run from: a shipped one, one in hegel.toml, or one you register. Left
nil, the run starts from the default profile as before, and a keyword
passed alongside still beats the profile.
Hegel::Settings.register_profile(name, profile: nil, **keywords)
registers a profile for the whole process, and
Hegel::Settings.set_default_profile(name) makes a profile the default
for every run; "" clears it. They follow hegel-rust's
Settings::register_profile and set_default_profile. An unknown or
reserved name raises Hegel::Error with the engine's message. See
ADR 0028.
Settings
Hegel.test takes five more keywords, with the names and meanings of
hegel-rust's Settings methods. Each one left nil takes its value from
the engine's settings profile, like test_cases: and seed:.
stop_on_first_failure: falsekeeps generating after a failure and
spends the wholetest_casesbudget before reporting.print_blob: falseleaves thereproduce_failure:line out of the
failure report. Leftnil, a profile orHEGEL_PRINT_BLOBdecides.show_statistics: truemakes the engine print, at the end of the run,
whattc.eventandtc.event_valuerecorded.nondeterminism_strictness:takes:quiet,:warn, or:error.
:errorends a run whose test behaves differently on a replay with
Hegel::Error.backend:takes:defaultor:urandom.
Test location
Every Hegel.test run now reports where its test lives to libhegel, with
hegel_settings_set_test_location. Under Antithesis the engine writes one
always assertion, ":: passes properties", for each
location; elsewhere it uses nothing, and no keyword changes. The file is
relative to the working directory when it lies under it, and the class and
function come from the frame that called Hegel.test. A call inside a
block takes the name of the method holding the block, and a call outside
any method, such as an RSpec example, takes the file without .rb as the
class and line <n> as the function, so each example in a file has its
own name.
Test cases
tc.event(label) and tc.event_value(label, value) record what a test
reached, for the statistics that show_statistics: true prints. They follow
hegel-rust's TestCase#event and TestCase#event_value.
Generators
date_strings,time_strings, anddatetime_stringsgenerate ISO 8601
date, time, and datetime Strings, with no options, as hegel-rust does.
A time keeps nanoseconds: nine fractional digits appear when the
nanosecond is not zero.permutations,subsequences, andsamplesdraw reorderings, ordered
subsets, and samples of a given sequence, as hegel-rust does. Its
with_replacementandwithout_replacementsetters are one keyword here,
samples(replacement: true), which is the default.durations(min_value: 0, max_value: 2**64 - 1)draws a duration as an
Integercount of nanoseconds, as hegel-rust'sdurationsdoes for a
Duration. It follows ADR 0030.arrays(elements, unique: true)draws an array with no two equal entries,
as hegel-rust'svecs(...).unique(true)does. An element equal to one
already drawn is rejected and drawn again, somin_sizestill counts
distinct entries.floats(allow_subnormal: nil)allows or rules out subnormal values, as
hegel-rust'sallow_subnormaldoes. Leftnil, it allows them whenever
the bounds admit one. Afloatscall whose bounds admit none, such as
floats(min_value: 1.0, max_value: 2.0), now tells the engine so.recursive(leaf, max_depth: 32, max_leaves: 100) { |subtrees| ... }
draws a recursively defined value, such as a tree, through the engine's
recursion calls, as hegel-rust'srecursivedoes. The engine decides each
leaf-or-branch choice and owns both budgets. An attempt that outgrows
max_leavesis drawn again from the root, and the spans it opened are
discarded, not kept. See
ADR 0032.
Text generators
text and characters take categories:, exclude_categories:,
include_characters:, and exclude_characters:, with the names hegel-rust
gives them. text also takes alphabet:, a String of the permitted
characters, and raises Hegel::Error at draw
time when it is combined with another character option. from_regex takes
alphabet:, a characters generator that limits every character of the
string, padding included. An unknown codec: now raises "text: invalid codec: ..." (or characters:) before the engine is called, and a
surrogate category in categories: raises too.
Grammar strings
from_lark(grammar) draws strings that a grammar written in the syntax of
the Lark
parsing library accepts, as hegel-rust's from_lark does. It takes
start:, explicit: (a Hash from terminal name to a generator of the
values drawn for it), alphabet: (a characters generator),
max_size:, max_depth:, and import_path:. An invalid option raises
Hegel::Error at draw time, with hegel-rust's message where it has one.
A grammar is compiled on its first draw in a run and freed when the run
ends.
Added
rule(name, weight: 1.0)sets how often the engine picks a rule relative to
the machine's other rules, as#[rule(weight = w)]does in hegel-rust. A
weight that is not finite and positive raisesHegel::Errorwhen the machine
runs.Hegel::StateMachine.rule_definitionsnow maps each name to a
Hegel::StateMachine::Rulewithblockandweight, where it mapped to the
block alone.
Fixed
- A filter on
Pool#values_consumedno longer loses the values it rejects. They
go back into the pool throughhegel_pool_restoreonce the draw is over, and
only the accepted value is consumed, as in hegel-rust.
Changed
Three defaults now match hegel-rust, which defines the interface. The results
change for a caller who relied on the old default.
-
from_regexdefaults tofullmatch: true, where it wasfalse. The whole
drawn string matches the pattern, instead of merely containing a match. To
keep the old behaviour, passfullmatch: false. -
textdefaultsmax_sizeto 100, where it was unbounded. Whenmin_size
is above 100, the default ismin_size + 100. Strings drawn without a
max_sizeare shorter than before. To draw longer strings, pass the
largermax_sizeyou need. -
binarydefaultsmax_sizethe same way, to 100 ormin_size + 100. -
A filter over a filter now spends 3 attempts between the two predicates, as
hegel-rust does, where each level used to spend 3 of its own. -
Hegel::Generator#do_draw_filtered(tc, predicate)is the draw behind
#filter. A generator whose draws have effects that a rejected value must
not keep overrides it.
Repeat a test body
tc.repeat { ... } runs its block again and again, for as long as the
engine keeps the loop going, the way hegel-rust's TestCase::repeat does.
The engine chooses how many times it runs, and shrinks a failure toward
fewer iterations. Each iteration shows in the failure report as a
// Repetition #N line, with what the iteration recorded indented under
it. A tc.assume or tc.reject inside the block skips that iteration
instead of discarding the whole test case.
The loop does not return when it ends. It raises Hegel::LoopDone, which
Hegel.test counts as a passing test case. It descends from Exception,
like the other control exceptions, so a rescue => e in the body does not
catch it. A break in the block ends the loop and returns from
tc.repeat. See
ADR 0034.
v0.3.4
Engine: libhegel 0.47.2 (was 0.47.1)
The libhegel 0.47.2 release notes
describe 0.47.2. It is the first engine release from the hegeldev/hegel
repository.
Each platform gem carries libhegel 0.47.2. The engine does not change its
C interface. A caller has nothing to change in code, and gets the same
results. The engine changes one text: the message of an internal engine
error now gives https://github.com/hegeldev/hegel/issues as the place to
report the problem, not the hegel-rust issue tracker.
Development
rake libhegel:fetchandrake libhegel:fetch_alldownload libhegel
0.47.2 and later from the hegeldev/hegel releases, and earlier versions
from the hegeldev/hegel-rust releases.
v0.3.1
Engine: libhegel 0.45.7 (was 0.45.4)
The libhegel 0.45.7 release notes
describe 0.45.7 alone. The
engine changelog
describes each engine release after 0.45.4, up to 0.45.7.
Each platform gem carries libhegel 0.45.7. The engine only adds to its C
interface, and these bindings do not use the addition. A caller has nothing
to change in code. The engine changes these messages for a caller:
from_regexwith a pattern that no string matches, such as"(?!)",
raisesHegel::Errorwith a message that names the pattern and says
that itmatches no string. Before, the run raisedHegel::Errorwith
a message that starts withUnsatisfiable.- The FilterTooMuch health check message says how to suppress the check
with thesuppress_health_checksetting, and it says "rejected by
assume() or a filter". Before, it showed Rust syntax. The
FailedHealthCheck:andUnsatisfiableprefixes do not change.
v0.3.0
Engine: libhegel 0.45.4 (was 0.45.1)
The libhegel 0.45.4 release notes
describe 0.45.4 alone. The
engine changelog
describes each engine release after 0.45.1, up to 0.45.4.
Each platform gem carries libhegel 0.45.4. The engine only adds to its C
interface, and these bindings do not use the additions. The engine
changes these results for a caller:
- A run that generates no valid test case now raises
Hegel::Errorwith a
message that starts withUnsatisfiable, also when
suppress_health_check:holds:filter_too_muchor
:test_cases_too_large. Before, such a run passed. A test that rejects
every case and expects to pass must now draw at least one valid case. from_regexrecords each drawn string as one choice that the pattern
constrains. A failure shrinks inside the pattern, so
from_regex("(?:bbbb|a)", fullmatch: true)now shrinks to"a", where
it stopped at"bbbb"before. The example database drops a failure that
an earlier engine saved for afrom_regexdraw, so the first run after
the update does not replay it.from_regexrejects a look-behind of variable width, such as
(?<=a*)b, withHegel::Error, as Python'sredoes.\w,\d,\s, and case-insensitive matching infrom_regexfollow
Unicode 17.0. Before, they followed Unicode 15.1.- A run stops when its test-case budget or its budget of invalid cases is
spent. Before, it could also stop after ten rejected cases in a row that
it had seen before.
v0.2.1
Each platform gem carries libhegel 0.45.1, up from 0.45.0. A caller has
nothing to change. The engine now reads a NULL string buffer with length
zero as the empty string, and it rejects a NULL buffer with a nonzero
length. These bindings send neither case, so a drawn value does not
change: from_regex("", fullmatch: true) draws "" on both engines.
v0.2.0
The floor is Ruby 3.4. A project on Ruby 3.3 stays on 0.1.1. See
ADR 0017.
Each platform gem carries libhegel 0.45.0, up from 0.32.5. The engine
changed its C interface in several places between the two, and these
bindings follow each change.
Changes a caller has to make
Hegel.test no longer takes stateful_step_count:. Pass step_count: to
Hegel::Stateful.run instead; it defaults to 50, as hegel-rust, hegel-go,
and hegel-java do. The engine no longer has a default of its own.
An invariant no longer runs after every rule. It runs before the first
rule and after the last, and between rules when the engine samples it,
about once in a test case that runs every step. Declare it with
invariant :name, always_run: true to keep the check after every rule.
See ADR 0021.
Failure reports
The report comes from the engine's own final replay of each failure, which
it runs inside the loop before the run ends. The body is no longer run
again after the run, so a failing run calls it once fewer per failure. A
tc.note block runs on each case the engine stamps for capture, which can
be more than one per failure. See
ADR 0019.
A body that fails only some of the time is now reported as an ordinary
failure, with a note: line from the engine that says how often it
reproduced. Before, such a body ended the run in Hegel::Error. A failure the engine could not confirm comes
without a blob, and its report leaves out the reproduction line.
reproduce_failure: now replays its blob as a run, until a replay fails,
within the engine's own budget. A blob from a nondeterministic failure
reproduces this way too. When no replay fails, Hegel.test raises
Hegel::Error and names both causes: a fixed bug, or a body whose failure
did not recur.
A body that rejects its input with tc.assume or tc.reject before it
draws anything now raises Hegel::Error ("Unsatisfiable"). Before, the
run stopped after that one case and passed.
Settings
A keyword left nil now takes its value from the engine's settings
profile: libhegel's default, a hegel.toml, or a HEGEL_* environment
variable such as HEGEL_TEST_CASES. On a CI server the engine picks its
ci profile, which derandomizes the run and turns the example database
off, so a run with database_key: and no database: stores nothing there.
A keyword you pass still wins. See
ADR 0023.
Generated values
The engine draws integers and floats from new distributions, and shrinks
further in several cases, so a seeded run draws different values
than it did on 0.32.5. from_regex follows Python's re more closely
under (?i) and (?a), and its pattern may now hold a NUL character.
times and datetimes keep microsecond precision. The engine now draws
nanoseconds, and the binding rounds each one down. See
ADR 0022.
A datetimes bound with a fraction of a microsecond could produce a value
outside it, because only the bound's whole microseconds reached the
engine. The lower bound now rounds up and the upper rounds down.
Other changes
HEGEL_LIBHEGEL_PATH pointing at an engine that lacks a function these
bindings call raises Hegel::Error that names the function. Before, FFI
raised a TypeError that named nothing.
rake libhegel:fetch downloads from hegel-rust's libhegel-v<version>
release tags, which hegel-rust uses for engine releases since 0.42.1.