Skip to content

Releases

v0.5.2

Choose a tag to compare

@github-actions github-actions released this 14 Nov 03:29
96af89a

Engine: libhegel 0.51.5 (was 0.51.1)

The libhegel 0.51.5 release notes
describe 0.51.5. The move also includes 0.51.2, 0.51.3, and 0.51.4, and the
libhegel changelog at 0.51.5
describes all four releases.

Each platform gem carries libhegel 0.51.5. A caller has nothing to
change. The public methods keep their signatures and return values.

In 0.51.4, the engine shrinks two kinds of recursive values better. A
recursive value whose branches can hold one child can now shrink to a
branch of one child. A value that the engine found after it abandoned
earlier attempts to draw it now shrinks fully. A failure report can so
show a smaller counterexample than before.

In 0.51.5, the engine adds two functions, hegel_pause_faults and
hegel_unpause_faults. These bindings do not call them, and this release
does not add a way to pause faults. 0.51.2 and 0.51.3 change only the
internals of the engine.

v0.5.1

Choose a tag to compare

@github-actions github-actions released this 12 Nov 03:59
3aa923c

Engine: libhegel 0.51.1 (was 0.50.0)

The libhegel 0.51.1 release notes
describe 0.51.1. The move also includes 0.51.0, and the
libhegel changelog at 0.51.1
describes both releases.

Each platform gem carries libhegel 0.51.1. A caller has nothing to
change. The public methods keep their signatures and return values.

In 0.51.0, the engine lets a state machine redraw its configuration
between rounds, and to do this it changes one function that these
bindings call: hegel_state_machine_next_group takes a new out-parameter
for the concurrency level of the round. These bindings pass it.
Hegel::Stateful.run and Hegel::StateMachine keep a sequential
machine, so the level is always 1. At the start of each round, the engine
now redraws, with probability 1/50, which rules the machine enables and
how it distributes drawn values, as the engine release notes say. A long
run can so explore more than one configuration.

In 0.51.1, a run of one test case (test_cases: 1) inside Antithesis
(ANTITHESIS_OUTPUT_DIR is set) with no database_key does not record
the choices of its test case. Its failure is reported from its one
execution: Hegel.test raises the exception of the body, but the engine
does not shrink or replay the failure, the report shows no drawn values,
and it has no reproduce blob. A run outside Antithesis, or with a
database_key, does not change.

v0.5.0

Choose a tag to compare

@github-actions github-actions released this 04 Nov 01:01
94b801c

Engine: libhegel 0.50.0 (was 0.49.0)

The libhegel 0.50.0 release notes
describe 0.50.0.

Each platform gem carries libhegel 0.50.0. The engine changes how a caller
declares a state machine: hegel_new_state_machine now opens an empty
machine, and new calls declare each rule and each invariant and then start
the machine. These bindings make the new calls. A caller of
Hegel::Stateful.run and Hegel::StateMachine has nothing to change, and
gets the same rules, weights, and invariants as before.

Changed

  • Hegel::TestCase#new_state_machine takes only the step count. The new
    methods #state_machine_add_rule, #state_machine_add_invariant, and
    #state_machine_start declare the rules and invariants and start the
    machine, in the same order as the engine calls. Hegel::Stateful.run
    uses them. A caller who called #new_state_machine directly must change
    to the new methods.

v0.4.1

Choose a tag to compare

@github-actions github-actions released this 02 Nov 05:08
9607c85

Engine: libhegel 0.49.0 (was 0.48.0)

The libhegel 0.49.0 release notes
describe 0.49.0.

Each platform gem carries libhegel 0.49.0. The engine adds functions and
constants to its C interface, and changes no function that these bindings
call. A caller has nothing to change in code. The engine move changes these
results for a caller:

  • The engine reads new environment variables over the settings profile:
    HEGEL_VERBOSITY, HEGEL_BACKEND, HEGEL_PHASES, and
    HEGEL_SUPPRESS_HEALTH_CHECK. Each one applies to a run that does not set
    the matching Hegel.test keyword. HEGEL_UNBOUNDED_CHOICES sets a setting
    that has no Hegel.test keyword. A malformed value makes Hegel.test
    raise Hegel::Error with HEGEL_E_INVALID_ARG and a message that names
    the variable. In 0.48.0, the engine ignored these variables. HEGEL_REPORT_MULTIPLE_FAILURES has no
    effect, because Hegel.test always sets report_multiple_failures.
  • The engine reads a max_size of 2**64 - 2 as "no maximum given". The gem
    now sends a max_size at or above 2**64 - 2 as 2**64 - 1, no upper
    bound, so text, binary, arrays, sets, hashes, and the other sized
    generators keep the meaning the caller wrote. A max_size above
    2**64 - 1 also means no upper bound now. Before, it raised RangeError.
    See ADR 0037.

Changed

  • The string generators follow the Unicode version Ruby was built with,
    RbConfig::CONFIG["UNICODE_VERSION"], through libhegel's
    hegel_set_unicode_version. A drawn character falls in the General
    Category Ruby gives it, and from_regex reads \w, \d, and \s as
    Ruby does. Before, they followed the engine's newest tables, Unicode 17.0.0.
    On Ruby 3.4, which follows Unicode 15.0.0, a character assigned after 15.0.0
    is now drawn as unassigned. See
    ADR 0036.

v0.4.0

Choose a tag to compare

@github-actions github-actions released this 21 Oct 20:32
e4ea0ca

Engine: libhegel 0.48.0 (was 0.47.2)

The libhegel 0.48.0 release notes
describe 0.48.0.

Each platform gem carries libhegel 0.48.0. The engine adds one function to
its C interface, hegel_set_unicode_version, and changes no other function.
These bindings do not call it yet, so the string generators continue to use
the newest Unicode tables of the engine, as before. A caller has nothing to
change in code. The engine move changes these results for a caller:

  • With the default report_multiple_failures: false, a run can find a
    different, smaller failure while it shrinks the first one. The run now
    shrinks that failure too before it reports it. In 0.47.2, the run reported
    that failure without shrinking it.
  • from_regex accepts the end-of-string anchor \z. In 0.47.2, a pattern
    with \z raised Hegel::Error with HEGEL_E_INVALID_ARG.
  • floats(min_value: Float::INFINITY, allow_infinity: false) and
    floats(max_value: -Float::INFINITY, allow_infinity: false) leave no value
    to draw. The draw now raises Hegel::Error with HEGEL_E_INVALID_ARG and
    a message that contains "leaves no value". In 0.47.2, the draw raised
    Hegel::Error with HEGEL_E_INTERNAL.

Profiles

Hegel.test takes profile:, the name of a settings profile to start the
run from: a shipped one, one in hegel.toml, or one you register. Left
nil, the run starts from the default profile as before, and a keyword
passed alongside still beats the profile.

Hegel::Settings.register_profile(name, profile: nil, **keywords)
registers a profile for the whole process, and
Hegel::Settings.set_default_profile(name) makes a profile the default
for every run; "" clears it. They follow hegel-rust's
Settings::register_profile and set_default_profile. An unknown or
reserved name raises Hegel::Error with the engine's message. See
ADR 0028.

Settings

Hegel.test takes five more keywords, with the names and meanings of
hegel-rust's Settings methods. Each one left nil takes its value from
the engine's settings profile, like test_cases: and seed:.

  • stop_on_first_failure: false keeps generating after a failure and
    spends the whole test_cases budget before reporting.
  • print_blob: false leaves the reproduce_failure: line out of the
    failure report. Left nil, a profile or HEGEL_PRINT_BLOB decides.
  • show_statistics: true makes the engine print, at the end of the run,
    what tc.event and tc.event_value recorded.
  • nondeterminism_strictness: takes :quiet, :warn, or :error.
    :error ends a run whose test behaves differently on a replay with
    Hegel::Error.
  • backend: takes :default or :urandom.

Test location

Every Hegel.test run now reports where its test lives to libhegel, with
hegel_settings_set_test_location. Under Antithesis the engine writes one
always assertion, ":: passes properties", for each
location; elsewhere it uses nothing, and no keyword changes. The file is
relative to the working directory when it lies under it, and the class and
function come from the frame that called Hegel.test. A call inside a
block takes the name of the method holding the block, and a call outside
any method, such as an RSpec example, takes the file without .rb as the
class and line <n> as the function, so each example in a file has its
own name.

Test cases

tc.event(label) and tc.event_value(label, value) record what a test
reached, for the statistics that show_statistics: true prints. They follow
hegel-rust's TestCase#event and TestCase#event_value.

Generators

  • date_strings, time_strings, and datetime_strings generate ISO 8601
    date, time, and datetime Strings, with no options, as hegel-rust does.
    A time keeps nanoseconds: nine fractional digits appear when the
    nanosecond is not zero.
  • permutations, subsequences, and samples draw reorderings, ordered
    subsets, and samples of a given sequence, as hegel-rust does. Its
    with_replacement and without_replacement setters are one keyword here,
    samples(replacement: true), which is the default.
  • durations(min_value: 0, max_value: 2**64 - 1) draws a duration as an
    Integer count of nanoseconds, as hegel-rust's durations does for a
    Duration. It follows ADR 0030.
  • arrays(elements, unique: true) draws an array with no two equal entries,
    as hegel-rust's vecs(...).unique(true) does. An element equal to one
    already drawn is rejected and drawn again, so min_size still counts
    distinct entries.
  • floats(allow_subnormal: nil) allows or rules out subnormal values, as
    hegel-rust's allow_subnormal does. Left nil, it allows them whenever
    the bounds admit one. A floats call whose bounds admit none, such as
    floats(min_value: 1.0, max_value: 2.0), now tells the engine so.
  • recursive(leaf, max_depth: 32, max_leaves: 100) { |subtrees| ... }
    draws a recursively defined value, such as a tree, through the engine's
    recursion calls, as hegel-rust's recursive does. The engine decides each
    leaf-or-branch choice and owns both budgets. An attempt that outgrows
    max_leaves is drawn again from the root, and the spans it opened are
    discarded, not kept. See
    ADR 0032.

Text generators

text and characters take categories:, exclude_categories:,
include_characters:, and exclude_characters:, with the names hegel-rust
gives them. text also takes alphabet:, a String of the permitted
characters, and raises Hegel::Error at draw
time when it is combined with another character option. from_regex takes
alphabet:, a characters generator that limits every character of the
string, padding included. An unknown codec: now raises "text: invalid codec: ..." (or characters:) before the engine is called, and a
surrogate category in categories: raises too.

Grammar strings

from_lark(grammar) draws strings that a grammar written in the syntax of
the Lark
parsing library accepts, as hegel-rust's from_lark does. It takes
start:, explicit: (a Hash from terminal name to a generator of the
values drawn for it), alphabet: (a characters generator),
max_size:, max_depth:, and import_path:. An invalid option raises
Hegel::Error at draw time, with hegel-rust's message where it has one.
A grammar is compiled on its first draw in a run and freed when the run
ends.

Added

  • rule(name, weight: 1.0) sets how often the engine picks a rule relative to
    the machine's other rules, as #[rule(weight = w)] does in hegel-rust. A
    weight that is not finite and positive raises Hegel::Error when the machine
    runs. Hegel::StateMachine.rule_definitions now maps each name to a
    Hegel::StateMachine::Rule with block and weight, where it mapped to the
    block alone.

Fixed

  • A filter on Pool#values_consumed no longer loses the values it rejects. They
    go back into the pool through hegel_pool_restore once the draw is over, and
    only the accepted value is consumed, as in hegel-rust.

Changed

Three defaults now match hegel-rust, which defines the interface. The results
change for a caller who relied on the old default.

  • from_regex defaults to fullmatch: true, where it was false. The whole
    drawn string matches the pattern, instead of merely containing a match. To
    keep the old behaviour, pass fullmatch: false.

  • text defaults max_size to 100, where it was unbounded. When min_size
    is above 100, the default is min_size + 100. Strings drawn without a
    max_size are shorter than before. To draw longer strings, pass the
    larger max_size you need.

  • binary defaults max_size the same way, to 100 or min_size + 100.

  • A filter over a filter now spends 3 attempts between the two predicates, as
    hegel-rust does, where each level used to spend 3 of its own.

  • Hegel::Generator#do_draw_filtered(tc, predicate) is the draw behind
    #filter. A generator whose draws have effects that a rejected value must
    not keep overrides it.

Repeat a test body

tc.repeat { ... } runs its block again and again, for as long as the
engine keeps the loop going, the way hegel-rust's TestCase::repeat does.
The engine chooses how many times it runs, and shrinks a failure toward
fewer iterations. Each iteration shows in the failure report as a
// Repetition #N line, with what the iteration recorded indented under
it. A tc.assume or tc.reject inside the block skips that iteration
instead of discarding the whole test case.

The loop does not return when it ends. It raises Hegel::LoopDone, which
Hegel.test counts as a passing test case. It descends from Exception,
like the other control exceptions, so a rescue => e in the body does not
catch it. A break in the block ends the loop and returns from
tc.repeat. See
ADR 0034.

v0.3.4

Choose a tag to compare

@github-actions github-actions released this 20 Oct 11:23
db5f6d5

Engine: libhegel 0.47.2 (was 0.47.1)

The libhegel 0.47.2 release notes
describe 0.47.2. It is the first engine release from the hegeldev/hegel
repository.

Each platform gem carries libhegel 0.47.2. The engine does not change its
C interface. A caller has nothing to change in code, and gets the same
results. The engine changes one text: the message of an internal engine
error now gives https://github.com/hegeldev/hegel/issues as the place to
report the problem, not the hegel-rust issue tracker.

Development

  • rake libhegel:fetch and rake libhegel:fetch_all download libhegel
    0.47.2 and later from the hegeldev/hegel releases, and earlier versions
    from the hegeldev/hegel-rust releases.

v0.3.1

Choose a tag to compare

@github-actions github-actions released this 15 Oct 02:44
39b6519

Engine: libhegel 0.45.7 (was 0.45.4)

The libhegel 0.45.7 release notes
describe 0.45.7 alone. The
engine changelog
describes each engine release after 0.45.4, up to 0.45.7.

Each platform gem carries libhegel 0.45.7. The engine only adds to its C
interface, and these bindings do not use the addition. A caller has nothing
to change in code. The engine changes these messages for a caller:

  • from_regex with a pattern that no string matches, such as "(?!)",
    raises Hegel::Error with a message that names the pattern and says
    that it matches no string. Before, the run raised Hegel::Error with
    a message that starts with Unsatisfiable.
  • The FilterTooMuch health check message says how to suppress the check
    with the suppress_health_check setting, and it says "rejected by
    assume() or a filter". Before, it showed Rust syntax. The
    FailedHealthCheck: and Unsatisfiable prefixes do not change.

v0.3.0

Choose a tag to compare

@github-actions github-actions released this 14 Oct 03:21
9fe3e4f

Engine: libhegel 0.45.4 (was 0.45.1)

The libhegel 0.45.4 release notes
describe 0.45.4 alone. The
engine changelog
describes each engine release after 0.45.1, up to 0.45.4.

Each platform gem carries libhegel 0.45.4. The engine only adds to its C
interface, and these bindings do not use the additions. The engine
changes these results for a caller:

  • A run that generates no valid test case now raises Hegel::Error with a
    message that starts with Unsatisfiable, also when
    suppress_health_check: holds :filter_too_much or
    :test_cases_too_large. Before, such a run passed. A test that rejects
    every case and expects to pass must now draw at least one valid case.
  • from_regex records each drawn string as one choice that the pattern
    constrains. A failure shrinks inside the pattern, so
    from_regex("(?:bbbb|a)", fullmatch: true) now shrinks to "a", where
    it stopped at "bbbb" before. The example database drops a failure that
    an earlier engine saved for a from_regex draw, so the first run after
    the update does not replay it.
  • from_regex rejects a look-behind of variable width, such as
    (?<=a*)b, with Hegel::Error, as Python's re does.
  • \w, \d, \s, and case-insensitive matching in from_regex follow
    Unicode 17.0. Before, they followed Unicode 15.1.
  • A run stops when its test-case budget or its budget of invalid cases is
    spent. Before, it could also stop after ten rejected cases in a row that
    it had seen before.

v0.2.1

Choose a tag to compare

@github-actions github-actions released this 10 Oct 03:27
fa1f22f

Each platform gem carries libhegel 0.45.1, up from 0.45.0. A caller has
nothing to change. The engine now reads a NULL string buffer with length
zero as the empty string, and it rejects a NULL buffer with a nonzero
length. These bindings send neither case, so a drawn value does not
change: from_regex("", fullmatch: true) draws "" on both engines.

v0.2.0

Choose a tag to compare

@github-actions github-actions released this 07 Oct 13:30

The floor is Ruby 3.4. A project on Ruby 3.3 stays on 0.1.1. See
ADR 0017.

Each platform gem carries libhegel 0.45.0, up from 0.32.5. The engine
changed its C interface in several places between the two, and these
bindings follow each change.

Changes a caller has to make

Hegel.test no longer takes stateful_step_count:. Pass step_count: to
Hegel::Stateful.run instead; it defaults to 50, as hegel-rust, hegel-go,
and hegel-java do. The engine no longer has a default of its own.

An invariant no longer runs after every rule. It runs before the first
rule and after the last, and between rules when the engine samples it,
about once in a test case that runs every step. Declare it with
invariant :name, always_run: true to keep the check after every rule.
See ADR 0021.

Failure reports

The report comes from the engine's own final replay of each failure, which
it runs inside the loop before the run ends. The body is no longer run
again after the run, so a failing run calls it once fewer per failure. A
tc.note block runs on each case the engine stamps for capture, which can
be more than one per failure. See
ADR 0019.

A body that fails only some of the time is now reported as an ordinary
failure, with a note: line from the engine that says how often it
reproduced. Before, such a body ended the run in Hegel::Error. A failure the engine could not confirm comes
without a blob, and its report leaves out the reproduction line.

reproduce_failure: now replays its blob as a run, until a replay fails,
within the engine's own budget. A blob from a nondeterministic failure
reproduces this way too. When no replay fails, Hegel.test raises
Hegel::Error and names both causes: a fixed bug, or a body whose failure
did not recur.

A body that rejects its input with tc.assume or tc.reject before it
draws anything now raises Hegel::Error ("Unsatisfiable"). Before, the
run stopped after that one case and passed.

Settings

A keyword left nil now takes its value from the engine's settings
profile: libhegel's default, a hegel.toml, or a HEGEL_* environment
variable such as HEGEL_TEST_CASES. On a CI server the engine picks its
ci profile, which derandomizes the run and turns the example database
off, so a run with database_key: and no database: stores nothing there.
A keyword you pass still wins. See
ADR 0023.

Generated values

The engine draws integers and floats from new distributions, and shrinks
further in several cases, so a seeded run draws different values
than it did on 0.32.5. from_regex follows Python's re more closely
under (?i) and (?a), and its pattern may now hold a NUL character.

times and datetimes keep microsecond precision. The engine now draws
nanoseconds, and the binding rounds each one down. See
ADR 0022.

A datetimes bound with a fraction of a microsecond could produce a value
outside it, because only the bound's whole microseconds reached the
engine. The lower bound now rounds up and the upper rounds down.

Other changes

HEGEL_LIBHEGEL_PATH pointing at an engine that lacks a function these
bindings call raises Hegel::Error that names the function. Before, FFI
raised a TypeError that named nothing.

rake libhegel:fetch downloads from hegel-rust's libhegel-v<version>
release tags, which hegel-rust uses for engine releases since 0.42.1.