Skip to content

Releases

v1.9.0

Choose a tag to compare

@github-actions github-actions released this 16 Nov 23:53
2e87678

Added

  • New opt-in operator rescue_body mutates rescue clauses (#418).
    It replaces the whole body of each rescue clause with nil, including a one-statement body.
    It also turns x rescue y into x rescue nil.
    A re-raise or a retry body is mutated too, because nil swallows the error.
    An empty body, a body that is already nil, (nil) or (), and an ensure body give no mutant.
    A body whose value is discarded and that holds only literals and variable reads is declined, and counts in summary.declined_by_operator.
    See docs/equivalent-skips.md, "Rescue body".
    Before this, no operator mutated a one-statement rescue body.
    The default operator set does not change.
  • return_nil reaches a method with a def-level rescue, else or ensure (#418).
    It replaces the final expression of the body, or of the else clause when there is one, with nil.
    It does the same for each rescue clause. The ensure value is not a return value, so it is left alone.
    Before this, return_nil gave such a method no final-expression mutant.
    A run with return_nil can now report a different number of mutants and new survivors. With --baseline, refresh the baseline.
    With both operators, a one-statement rescue body gives the same edit twice. The run keeps the return_nil mutant, lists the rescue_body id in its copies, and counts that copy in dropped.
    A final explicit begin ... end block gets the same mutants as the def-level form, so RuboCop's Style/RedundantBegin autocorrect does not change them (#470).
  • run says when an ignore: entry matches no mutant (#377).
    The warning names the id and says the list covers only that run's sources and operators.
    The warning does not change the exit code.
    --format json lists the ids in summary.unmatched_ignores, in file order, each id once.
    --dry-run prints the same warning, and --dry-run --format json lists the same ids.
    --since does not warn about an entry whose mutant is outside the diff: matching happens before the diff narrows the run.
    A run of some files warns about entries those files and operators do not match. The warning says so.
    A source you passed that has no paired test still counts: its entries are not reported as unmatched, and run agrees with run --dry-run of the same files.
    --only skips the check. The list is then empty, which does not mean every entry matched.
    A dropped repeat of an edit counts as matched.
    An old-format id that still matches is not in this list. The old-id warning still names it.
    mutineer migrate --dry-run over every source lists entries that match nothing and exits 1.
    That exit code is the CI check, in 1.x and after 2.0.
    The key is part of schema 1.10.
  • A report names the dropped copy of an edit (#374).
    When two operators, or one operator twice, make the same change on one
    line, one copy runs and the other is dropped. The JSON report now lists
    each dropped id on the kept mutant as copies, on survivors, ignored
    mutants, and the other mutant rows. summary.dropped counts them. They
    stay out of total and out of the score. mutineer triage prints an
    ignore: entry for the survivor and for each of those ids, so one round
    writes every entry. An entry still ignores only its own id. The human
    report, the HTML report, the dry run, and the Action summary show the
    count. A dropped copy of an ignored mutant is listed on the kept mutant,
    so that entry no longer looks unused. Both keys are part of schema 1.10.
  • A source that cannot load alone points at boot: (#376).
    When a source raises NameError (including NoMethodError) while it
    loads, and the run has no boot: file, stderr adds one line. The names
    it needs come from boot: or --boot, which loads before the sources.
    Listing the entry file as an earlier source also loads it first.
    In that run, require: files still load after the sources, so a source
    cannot use them while it loads. With a boot file they load after the
    boot file, and the parent does not require the sources.
    --test-command does not load require: files. The hint line is absent
    for any other load error, and for a NameError in the boot file or a
    require: file. A dry run does not load the source, so it does not print
    the line. --daemon and --test-command do not require sources in the
    tool process, so they do not print it either. --help describes --boot
    as an app entry point or a gem's lib/<name>.rb, and no longer limits it
    to Rails apps.
  • A run counts sources that have no paired test (#381).
    The report names each skipped source and how many mutants it holds.
    JSON adds summary.unpaired_sources, summary.unpaired_mutants, and unpaired_sources[].
    The human report, the HTML report, and the dry run show the same counts.
    Unpaired mutants are excluded from the score and cannot fail --threshold.
    They join the unrun-share stderr warning, except under --fail-fast.
    That warning does not change the exit code.
    Schema version is 1.10.
    Compare it as two integers, so 1.10 follows 1.9.
    On Fizzy at d5b7b749d, 106 sources and 312 mutants were unpaired.
    The paired run total was 759, so the share was 29.1%.
    On Fizzy at f220e4832, 92 sources and 149 mutants were unpaired.
    The paired total was 922, so the raw share was 13.9%.
    After the 10 ignored mutants leave the denominator, the share is 14.0%.
    Mutineer now parses those sources.
    A disable-line warning or a git-diff warning can name them.
    The Action step summary does not show these counts (#481).
    A dry run with --rails or --boot and no --test still exits 2 (#410).
    #349 measured the unrun share (of attempted plus unrun) on Fizzy at f220e4832, with Mutineer at 46ee2bb3.
    A full daemon run with the default operators had 840 attempted mutants, 1 unplaceable, 0 skipped (invalid) and 149 unpaired.
    Without the unpaired mutants, the share was 1 of 841 (0.12%).
    With them, it was 150 of 990 (15.2%), so the warning fired.
    With all 21 operators, the four paired files that can hold an unrun mutant had 24 under redefine and 20 under reload.
    Against the default run's 840 attempted mutants, that is at most 2.8%.
    The 10% limit stays.
    A share above it that comes from unpaired mutants is a true count, not noise.
    The 2 skipped (invalid) mutants in the all-operator runs came from a literal_mutation bug that this release fixes (#469).
  • A concern pairs with the test that mirrors its namespace (#379).
    app/models/concerns/storage/totaled.rb can pair with test/models/storage/totaled_test.rb.
    That path matches Storage::Totaled.
    The concerns/ path still wins when it has a test.
    Existing pairs stay the same.
    Before this, the concern was skipped and added no score.
    The skip line was no test found by convention.
    A concern that was skipped can now run.
    That can change a score or a gate.
    --baseline treats new survivors as new.
    Refresh the baseline.
    See docs/ci.md, "Refresh the baseline".
    A failed namespace load can move a --threshold run from exit 0 to exit 1.
    Those mutants move from no_coverage to uncapturable.
    A nil score with a broken mutant then fails the gate.
    The share of mutants with no verdict also rises.
    With use-bundler: false, an Action with an empty version installs the latest gem.
    Pin version, or refresh the baseline, before the next run.
    With use-bundler: true, the Gemfile picks the version.
    A failed namespace test does not mark a concern that has a test at its concerns/ path.
    A namespace split name pairs under #408.
  • A concern pairs with a split test at its namespace path (#408).
    app/models/concerns/storage/totaled.rb can pair with test/models/storage/totaled_bytes_test.rb.
    The concerns/ path still wins when it has a test that pairs, exact or split.
    A split owned by a longer source does not count.
    A longer source keeps its file when that source pairs with it.
    That includes a longer concern with no test of its own, and a model or lib/ file of the longer name.
    A model keeps a split file it already pairs with. The concern may use that same file. The model does not lose it.
    A shorter concern keeps a namespace split when the longer concern already has a test at the concerns/ path.
    The longer concern does not add that file.
    A namespace spec stays the only file. A failed split beside it does not mark the concern.
    Before this, a namespace split name left the concern skipped, with no score.
    A concern that was skipped can now run. That can change a score or a gate.
    --baseline treats new survivors as new. Refresh the baseline.
    See docs/ci.md, "Refresh the baseline".
    A failed namespace split moves those mutants from no_coverage to uncapturable.
    On the Fizzy tree measured for #379, no unpaired concern had a namespace split name.
    One of 352 app/ sources paired with a split file.
  • Mutineer now reports the methods that got no mutants (#386). A
    method whose body is one if with a rescue gets no mutant from the
    default operators, and the run did not say so. The dry run summary now
    ends with no mutants in N of M methods, and --dry-run --verbose
    prints one no mutants: line per method. The human report has a
    No mutants: row. The JSON report adds summary.subjects,
    summary.unmutated_subjects and a top-level unmutated_subjects[]
    list, and schema_version is now 1.10. A program that calls
    Reporter#report without the method counts gets null for all three,
    not 0; the CLI always fills them. An empty run says how many
    methods it found and points to --list-operators. Scores, exit codes,
    the mutants that run and the default operators do not change. The
    README names condition_true and condition_false for these methods.
    For a one-statement rescue body, use the new opt-in `rescue_...
Read more

v1.8.2

Choose a tag to compare

@github-actions github-actions released this 10 Nov 13:08
354c77d

Fixed

  • Redaction no longer hides a password inside other words (#341). A
    form of the database password (raw, decoded or escaped) longer than 3
    characters, made only of ASCII letters, digits and underscores, is now
    hidden only as a whole word, like a short one. With
    password test, the backtrace path user_test.rb stays readable. Such a
    password inside a longer word is no longer hidden. The word test on its
    own is still hidden, also right after an escape such as %40 or \n, a
    colour code, or a -p flag. That escape rule now covers short forms
    too. A longer form with any
    other character is still hidden wherever it appears.
  • Redaction keeps the host after a password with an @ (#341). In
    postgres://u:p@ss@h/db and p@ss, only p@ss is hidden now. Before, the
    host and the rest of the line were hidden too. One known limit: in a URL
    other than DATABASE_URL, a password that holds an @, a host, a /, :,
    ? or #, and a space, shows the part after that @ when the URL's own
    host has nothing after it, or has an @ in its path or query. #341 tracks
    it.
  • Daemon shutdown no longer waits a second (#341). A process the app
    started could keep the daemon's error output open. Each quit or respawn of
    --daemon then waited one second. Mutineer now stops waiting once the
    daemon has exited and its output is read.
  • One line that Prism 2 rejects no longer skips every mutant in the file
    (#392).
    Prism 2.0 and later report an operator write to a bare constant
    inside a method (T += 1, T ||= 1, T &&= 1) as a dynamic constant
    assignment. Ruby's parse.y does the same, but Ruby 3.4 and 4.0 load such
    a file with their own Prism 1. Mutineer kept a mutant only when the mutated
    file parsed with no errors, so on Prism 2 every mutant of that file was
    counted as skipped (invalid). When Prism reports an error, Mutineer now
    asks the running Ruby to compile the mutant, and skips it only when Ruby
    cannot. This holds in the dry run, in process, with --test-command, on
    --daemon, and for the redefine strategy's method snippet. A project gets
    Prism 2 when its bundle loads it, for example
    through minitest 6.1. On Prism 2, the mutants of such a file that were
    skipped now get a verdict, so the score, the counts, and the JSON report
    can change for those projects. Prism 1 runs do not change.
  • --test-command warns about a random test order (#338). Mutineer
    sets no seed for your command, so the clean run and each mutant run can
    run the tests in different orders. An order-dependent suite can then fail
    a mutant run for a reason the mutant did not cause, and that mutant
    scores a false killed. When the clean run prints a random seed
    (Minitest Run options: --seed N, RSpec Randomized with seed N) and
    Mutineer finds no fixed seed, it now prints one warning with the fix for
    that runner: put SEED=1 before the mutineer command for Minitest, or
    add --order defined to an RSpec command and keep --order rand out of
    SPEC_OPTS, which RSpec reads after the command. The docs for
    --test-command say the same. Mutineer does not set the seed itself, so scores do not
    change. It cannot see a seed set in a wrapper script or .rspec, or a
    Minitest test_order, so the warning can print when the order is
    already fixed. #368 tracks these limits.
  • Reload places a hook-block mutant before the rest of its file runs
    (#348).
    Under --strategy reload, a placeable mutated method from a
    concern's included or prepended block reached the classes that include
    or prepend the concern only after the whole mutant file loaded. Methods
    that reload cannot place still score unplaceable. A later line in the same
    file that called the method while the file loaded, such as
    TOTAL = Host.new.doubled(2), still used the original method. Mutineer
    now places the mutated method when the load reaches the hook call, before
    the rest of the file runs, in-process and on --daemon. Such a mutant can
    now be killed where it used to survive or score ran_at_load, so the
    score of an affected file can change. A mutant that raises in such a
    load-time call now scores error. The alias and kept-method checks
    still score error. The kept-method check runs after the load, so a
    method handle the file holds only while it loads does not count.
  • The Action creates the cache folder when output points into it
    (#340).
    On a fresh checkout, output: .mutineer/pr.json exited 2,
    because .mutineer/ did not exist yet. The CLI already created that
    folder (#270), but the Action checks the path itself before it runs the
    CLI. The Action now creates the cache folder when it is the missing parent
    of output: .mutineer by default, the cache_dir from .mutineer.yml,
    or a --cache-dir in extra-args (or a prefix of it down to --cac, such as
    --cache). Any other missing folder still exits 2.
  • Reload runs a Rails class body a second time without errors (#378,
    #384).
    Reload loads the mutant file over the class that is already
    defined, so its body runs again. A model that declares an enum, and a
    class that calls skip_callback (a controller's skip_before_action or
    allow_unauthenticated_access calls it), raised on that second run, and
    every mutant in the file scored error. While the mutant file loads, an
    enum the class had before the load now skips Rails' conflict check, and
    skip_callback does not raise for a callback the first run already
    removed: one named by a method the class has, such as a callback from the
    parent class or an included concern. A skip that names no such method or
    parent callback still raises. Both checks run as usual while the tests
    run. This holds on --daemon and with --strategy reload.
  • Reload no longer runs validators and block callbacks twice (#390).
    The second run of a class body added a second copy of each validates,
    each validate block, and each callback written as a block or a lambda,
    such as before_save do ... end. Rails removes a repeated callback only
    when it names a method. A test that counts errors or side effects then
    failed for every mutant in the file, so those mutants scored killed
    falsely. After the mutant file loads, Mutineer now keeps one copy of each:
    the new one, in the place of the old one, so the order of callbacks does
    not change. Entries from an included concern stay, because the reload does
    not run the concern's block again. When Mutineer cannot tell which old
    entry a new one repeats, it keeps both, as before (#413). This holds on
    --daemon and with --strategy reload. With eager loading on, a mutant
    of a parent class file still adds a callback back to each subclass that
    skipped it (#414).
    Score change: a Rails model with such a test can now show survivors
    that were false kills before, so its score can go down.
  • Concern mutants load when the test app loads lazily (#383). With
    eager_load off (the Rails test default), the mutant's load was the first
    use of the concern, so Rails loaded the original file inside it. The
    original included block then came back, and the mutant's block raised
    MultipleIncludedBlocks. Mutineer now loads the original file first, through
    the app's autoloader, so the mutant's block replaces it.
  • The pending-migration check no longer trips shared_databases (#385).
    rails/test_help checks every database for pending migrations when the
    tests load, shared ones too. A parallel --daemon run then reported the
    shared database as used and scored every mutant error. Mutant forks of a
    run with shared_databases now turn that check off. The clean check, which
    runs before any mutant, still makes it.
  • No Rails deprecation warning from concern mutants (#387). Placing a
    mutant inside an included block looked at every module. Rails 8.2 keeps a
    deprecated constant as a module that warns on every call, so each such
    mutant printed LoadInterlockAwareMonitor is deprecated. Mutineer now skips
    that module.

v1.8.1

Choose a tag to compare

@github-actions github-actions released this 02 Nov 17:52
59a93ca

Fixed

  • No "already initialized constant" warnings under reload (#364). Each
    mutant loads its whole file again, so every constant in the file was
    assigned twice. Ruby then printed two warning lines per constant, per
    mutant, on stderr. Mutineer now turns warnings off while it loads the
    mutant file, in-process and on --daemon, and turns them back on before
    the tests run. Warnings that the tests raise still print. Other warnings
    raised while the mutant file loads, such as an unused variable under -w,
    no longer print either. Scores and the JSON report do not change.
  • The "not green" message says it once (#360). When the clean run of a
    file's tests failed, the message said "the unmutated suite is not green"
    twice. It now says it once, then names the test files after Failed:.
    A failure that only the check of all test files together finds is listed
    as Failed: combined suite.
    Under Minitest, the seed hint follows them.
    The message itself still does not name the failing tests or the first
    error; #360 tracks that. On the default in-process path, the failing
    test's own report still prints on stderr before the message.
  • An unexpected error no longer prints an unredacted message (#341).
    An error that no other handler expected went to Ruby's default printer,
    which did not hide a database password in its message. The CLI now prints
    that error with the password hidden, with its backtrace, and exits 1, as
    before. No known path raised such an error with the password.
  • An ambiguous or malformed option is a usage error (exit 2). An option
    that matches more than one flag, such as --ver (--verbose or
    --version), or a value given to a flag that takes none, such as
    --verbose=1, crashed with a backtrace and exit 1. It now prints one line
    and exits 2, like every other bad option. This holds for init and
    triage too.

v1.8.0

Choose a tag to compare

@github-actions github-actions released this 29 Oct 19:30
37816d4

Highlights

Read these before you upgrade. Details are in the sections below.

  • Scores can change after the upgrade. Under --strategy reload, most methods
    in a concern's included or prepended block are now scored (#297). Runs
    that list call_to_receiver get more mutants (#274). Runs that list
    range get new mutants for endless ranges (#299). A mutant that two
    operators make now counts once (#275). These can change counts, scores
    and survivors. If you gate on --baseline, write a new full baseline.
  • Delete ignore: entries for range mutants (#299). A new range
    mutant can take an old id, so an old entry can hide a real survivor.
  • Fewer false kills. Under --test-command, a non-zero exit other than 1
    scores error, and so does an exit 1 with no failing test in the summary
    line (#292). An Active Record connection or lock error inside a Minitest
    test scores error too (#296). Before, these scored killed.
  • Exit codes match STABILITY.md (#288). A boot, source or require:
    file that raises while it loads exits 1. A missing boot or require:
    file exits 2.
  • --threshold compares the exact score (#280). A run with 1999 of
    2000 mutants killed shows 100.0% but no longer passes --threshold 100.
  • New report data. The JSON report adds summary.declined and
    summary.declined_by_operator (schema 1.9), and stderr warns when many
    mutants could not run (#291).
  • Parallel --daemon with unused test databases (#316). List them under
    shared_databases to run more than one worker. Needs Rails 7.1 or later.
  • The gem ships the agent skill (#315).
  • Releases wait for a maintainer. Nothing publishes on a schedule or on
    a tag push.

Added

  • shared_databases config key (#316). An app with a second test
    database ran one worker under --daemon. List the databases your tests
    never use (shared_databases: [cable]), and the workers share them and run
    in parallel. After each mutant, a parallel worker checks those databases.
    The mutant scores error when a pool for one is still connected after the
    tests, or ran SQL during them. A replica in the same connects_to counts
    as its database. Listing the primary database stops the run with exit 2.
    A name that matches no test database warns and is not checked. The check
    needs Rails 7.1 or later. Three safety changes come with it for
    --rails --daemon: a test database with database_tasks: false now
    counts toward the one-worker limit, and the run keeps one worker when the
    app's databases cannot be listed or the first daemon does not boot
    (before, each case could start several workers). See
    More than one database
    for the transactional-tests caveat.
  • Counts of mutants that were not scored (#291). Operators now count the
    candidates they decline: a heredoc they cannot mutate safely, or an
    equivalent skip such as an endless range or a skipped chain-link name. The
    dry run prints the count. The human report prints it when the run has
    mutants; when every candidate was declined, the empty-run warning names the
    count instead (that warning is off under --allow-empty). The JSON report
    adds summary.declined and summary.declined_by_operator (schema 1.9).
    A new stderr warning fires when unplaceable and skipped-invalid mutants are
    more than one and more than 10% of the mutants a run tried to score. The
    warning does not change the exit code. Measured on commit caa9e1a (this
    change merged with main) with every operator: the dry run on lib/
    declines 195 candidates beside 17,374 mutations (call_to_receiver 122,
    chain_link 45, range 12, statement_removal 10, return_nil 6); the dry run
    on the Rails fixture app declines 0 beside 69 mutations. A
    --strategy redefine --rails run on the fixture app models (order,
    price_list, billing/fee) has 0 unplaceable and 0 skipped-invalid of 12
    attempted, so no warning. The fixture has no concerns, so it cannot tune
    the 10% limit.
  • Included and prepended block methods under reload. After you upgrade,
    follow the full baseline refresh recipe
    with --no-since --format json --output .mutineer/refreshed-baseline.json and your
    existing source and test options. Review the report and replace the stored
    baseline. This applies only to runs that use --strategy reload: under
    redefine these methods stay unplaceable. GitHub Action users on reload,
    the Action's default (a Rails app without a daemon uses redefine), must
    replace the file passed to the baseline input too. A check against an older
    report lists the new survivors as regressions.
    Under --strategy reload, in-process and on the daemon, a method in a
    concern's included or prepended block is now mutated and scored (#297).
    Mutineer loads the mutated method into each class that already has it. A
    prepended wrapper that calls super reaches that mutated method too. A
    concern included into an object's singleton class is supported. A class
    that includes the concern later gets it from the reloaded block. The
    rest of the block does not run again, so validators and callbacks are not
    added twice. These mutants used to be unplaceable, so a reload score can go
    down and a --threshold gate can start to fail. Some methods stay
    unplaceable: a method in a class, module or Struct.new block inside the
    block, a method inside another method or block, a method with an explicit
    receiver other than self (such as def base.tag in included do |base|),
    a hook-block method whose source line conflicts with another def or an
    explicit Ruby method creator (the full rule is in docs/json-schema.md),
    every method in a block that uses alias or alias_method, a method the
    block itself calls, and every method in a block whose included do call is
    not a plain line of the module
    body (for example inside if, unless, or x = included do). The full list
    is in docs/json-schema.md. If a class aliases a method that Mutineer does
    run (one not in that list) outside the block, the mutant scores error,
    because the alias keeps the original method. This also applies when a
    wrapper replaces the original name and calls the saved alias. A retained
    Method or UnboundMethod for the original also makes the mutant score
    error: a wrapper can call that handle without testing the mutated method.
    --strategy redefine still
    reports all of these methods unplaceable.
  • The gem ships the agent skill. skills/mutineer/SKILL.md is now
    packaged in the gem, and the gemspec opts in as a
    Rails Hyperdrive
    companion (hyperdrive_targets: "*", hyperdrive_artifacts: "skill"). In a
    Rails app that bundles mutineer, bin/rails hyperdrive:init installs the
    skill to .claude/skills/mutineer/SKILL.md. A test in the default suite
    fails if the gemspec stops shipping the skill or drops the opt-in (#315).

Fixed

  • --daemon stops on MySQL older than 8.0 (#298). The support matrix said
    MySQL 5.7 is not supported, but --daemon ran on it. Mutineer now reads the
    server version before it copies any worker database. MySQL older than 8.0
    stops the run with could not provision <base>: MySQL <version> is not supported. Pass --no-daemon (or set daemon: false) to stay serial.
    MariaDB, MySQL 9 or later, and a server version Mutineer cannot read print
    one warning and run.
  • An in-test database error scores error, not killed (#296). When
    the first failing test of a Minitest mutant run failed with an Active
    Record connection or lock error (ConnectionNotEstablished, including pool
    timeouts, ConnectionFailed, Deadlocked, LockWaitTimeout), the mutant
    now scores error on --daemon and in-process. This includes an error in
    setup. Errors are out of the score, so a lost database connection during
    a run no longer raises the score. A stderr line names the error. A mutant
    that really causes a deadlock, or that breaks connection-handling code, now
    also scores error. More error verdicts can fail a --threshold run
    through the existing no-verdict gate. Other database errors, such as a bad
    SQL statement, still score killed. So does ConnectionNotDefined, which
    Rails 8 raises for an unknown role or shard. Rails 7.2 raises a plain
    ConnectionNotEstablished there, so such a mutant scores error on 7.2.
    With --matrix, a database-failed test is never a killer and leaves its
    row incomplete, even if an earlier assertion already killed the mutant.
  • An old temp file of the other mode no longer stops the boot (#287).
    Before the app boots, each mode sweeps the source directories. In 1.7.0
    an in-process run removed only old mutineer_mutant*.rb files and the
    daemon removed only old mutineer_daemon*.rb files, so a file the other
    mode left made Zeitwerk raise. Now both boot sweeps remove an orphan of
    either mode, and the daemon tells Zeitwerk to skip both names. You no
    longer delete such a file by hand. A file of your own whose name only
    starts with mutineer_mutant or mutineer_daemon, such as
    mutineer_mutant_helper.rb, is no longer removed: a sweep now matches
    only the temp file shape. An unlocked old file is removed even when its
    creator's pid has been reused; a held file lock still protects it.
    Do not run Mutineer older than 1.7.0
    at the same time in the same checkout: it still writes temp files
    without the dot, and a newer run's mutants error when Zeitwerk finds
    one.
  • A second in-process run no longer deletes a live mutant file (#295).
    Each in-process run sweeps the source directories before its boot and
    after its run. That sweep treated every .mutineer_mutant*.rb file a...
Read more

v1.7.0

Choose a tag to compare

@github-actions github-actions released this 23 Oct 23:18
ef5f17f

Added

  • Call-to-receiver operator (Tier-2, opt-in via --operators):
    call_to_receiver replaces the last call of a chain with its receiver, with
    the arguments and block of that call (user.full_name -> user). The mutant
    survives when no test checks what that last call does. It does not overlap
    with chain_link: chain_link drops a call in the middle of a chain, and
    call_to_receiver drops only the last call. The two give the same source
    only when the last call repeats the call before it (node.name.name). It
    skips the same names as chain_link, and it skips operator calls and
    setters.
  • Reasons on ignored mutants. Text after -- on
    # mutineer:disable-line, and a .mutineer.yml ignore: mapping
    (id plus reason), is stored on the ignored mutant. The JSON report
    (schema_version 1.8) and the HTML report show it. A mapping with no
    id, or with an unknown key, warns. Mutineer 1.7 is the first version
    that reads a mapping. An older version matches no mutant to a mapping
    and does not warn, so the mutant it names is not ignored. Upgrade
    every gate that reads the file before you add a mapping.
    mutineer migrate rewrites the old id inside a mapping and keeps its
    reason.
  • mutineer triage. It prints ignore: mapping entries for survivors
    in a JSON report. --reason is required. Pass --all or --id. It
    does not edit files.
  • Equivalent-skip page. The docs list every built-in chain-link skip
    and the endless-range skip, and why each one is equivalent.
  • Postgres worker databases for --rails --daemon. Each worker gets
    <base>-mutineer-<N>, copied from the test database before mutants run.
    A second run cannot drop a database another live run holds. An app with
    more than one database config runs one worker and says so once.
    The worker databases stay after the run.
  • MySQL 8 worker databases for Minitest on --rails --daemon. Each worker gets
    <base>-mutineer-<N> on mysql2 or trilogy, with the base shortened to fit
    MySQL's 64-byte name limit. Mutineer creates the
    database, loads schema.rb or structure.sql, and copies rows once at
    the start. Triggers from the schema do not fire on the copied rows.
    Mutineer creates them again after the copy.
    A second run cannot drop a database another live run holds.
    That work uses the TLS and timeout settings from the database config.
    MariaDB is not tested. MySQL 5.7 is not supported. The worker databases
    stay after the run.
  • mutineer migrate. It rewrites old-format ids in .mutineer.yml
    ignore: to the current ids. It keeps comments and every other line.
    One old id that matches several mutants becomes every new id. An id that
    matches nothing stays, and the command exits 1. A block scalar under
    ignore: is not supported: the command exits 2 and does not change the
    file. --dry-run prints each change and writes nothing. It does not
    rewrite a baseline. Generate a new baseline with --format json. An
    old-id warning names mutineer migrate and says that this becomes an
    error in Mutineer 2.0. The baseline warning says to generate a new
    baseline with --format json.
  • mutineer init. It writes a commented .mutineer.yml with the default
    operators. since stays commented, so the first run is a full scan.
    init --rails also sets rails: true and prints the first run command.
    An existing file is kept unless you pass --force.
  • Rails rollout guide. The page walks from one model, to a report-only CI
    job, to a --since and --baseline pull request gate, to --threshold
    on chosen folders. It has a GitHub Actions recipe and a GitLab CI recipe.
  • A mutation-testing primer in the README. "What is mutation testing?"
    walks through one small example, from a surviving mutant to the test that
    kills it, and says what each report row means and what to do with a
    survivor. The usage block lists all four commands.
  • A production-ready section and a stability contract. The README opens
    with a support matrix for Ruby, database, framework, and parallelism.
    STABILITY.md lists what a release keeps and what 2.0 will change.
    SECURITY.md, CONTRIBUTING.md, and GitHub issue forms are new.
    The docs home page takes the same section from the README at site build.
  • --allow-empty and allow_empty:. A run with no mutants now warns
    that Mutineer 2.0 will fail it, unless --allow-empty marks it as expected.
    A --since run whose changes hold no mutants does not warn: it says the
    changes hold nothing to test. A --since run with no mutants even before
    scoping still warns. The exit code does not change in 1.x.
  • --no-daemon. It beats a .mutineer.yml daemon: true for one run,
    like --no-fail-fast. A Minitest --rails run without a daemon value now
    prints a notice that 2.0 uses --daemon by default for --rails;
    --no-daemon (or daemon: false) keeps the run serial and stops the
    notice.
  • "Did you mean" hints. Unknown names in --operators, in
    .mutineer.yml operators: and keys, and in # mutineer:disable-line
    markers get the closest valid name. The .mutineer.yml warnings also say
    that 2.0 makes them errors.
  • --coverage-from FILE lets parallel shards share one coverage capture.
    A --rails/--boot run reads the coverage.json of an earlier run with
    the same tests over these sources or more. It skips the capture. Each test
    that covers one of its sources still runs alone, as on a cache hit, because
    a mutant runs only those tests. A test that covers none of them does not,
    so a shard runs far fewer tests alone than a full suite. All tests then run
    once together, and the run saves its own cache.
    Boot-mode caches now also record each source's fingerprint, the ownership
    list and a digest of the other inputs. A capture that does not fit is an
    error (exit 2), never a silent recapture. So is one where loading its other
    sources ran these sources' code, which happens under lazy loading with
    --strategy redefine. Without boot mode, and with --daemon,
    --test-command or --dry-run, the flag warns that it has no effect.

Changed

  • Rails test workers stay off during a mutant run. Mutineer sets
    PARALLEL_WORKERS to 1 for in-process, --daemon, and --test-command
    runs. A value you already set is replaced, and Mutineer prints one line.
    The coverage cache stores the PARALLEL_WORKERS value, so a cache from an
    older version rebuilds once. --test-command also sets DISABLE_SPRING=1.
    The README explains parallelize, Bootsnap, and Redis after a fork.
  • --test-command exit codes 126 and 127 are scored error, not
    killed.
    A wrapper that could not start the suite ran no test. Under
    --threshold, such a run can now score lower or fail the no-verdict limit,
    where before it passed on false kills.
  • A run with errored mutants and no --threshold says so. Errors are not
    in the score, so they do not fail the run; the message says what
    --threshold would enforce.
  • --since also narrows suppressed mutants. A scoped report lists only
    ignored mutants on changed lines, as --dry-run --since already did.
  • The empty-run message moved from the human report to one stderr warning
    for every format.
    "No mutations generated" is gone. An empty --since
    run prints "nothing to test in the changes since REF" instead.

Fixed

  • Daemon errors no longer show the database password. Rails parses
    DATABASE_URL itself, and a malformed URL makes it raise an error that
    quotes the whole URL. The daemon sent that error to the terminal when
    the app failed to boot, and the same could happen when it set up worker
    databases or the coverage map, and in a --verbose coverage warning.
    Mutineer now replaces the password in any URL, and the DATABASE_URL
    password anywhere else (escaped or cut short included), with
    [REDACTED] before it reports the error.
  • --strategy reload runs concern files with an included or prepended
    block (#256).
    Every mutant in such a file was errored: ActiveSupport
    raised MultipleIncludedBlocks (or MultiplePrependBlocks) because the
    mutant loads from a temp file with another path. Before each reload mutant
    loads, in-process and on the daemon, the forked child now forgets the blocks
    that the source file stored, so methods in the concern body and in
    class_methods are killed or survive as usual. A block the mutant's load
    does not store again (a concern under an unless defined? guard) is put
    back. A method inside an
    included or prepended block is unplaceable under reload too: the block
    does not run again in a class that already includes or prepends the
    concern, so the mutant would falsely survive. That includes a method of a
    class defined inside the block. Use --test-command to run those mutants.
  • No heredoc opener without its body. A mutation that kept a heredoc
    opener but replaced the lines of its body gave source that does not parse,
    such as operand_removal on foo(<<~EOS) || with the body and bar on
    the next lines. The mutant showed only as skipped (invalid). Now the
    operator does not emit it.
  • An RSpec run that fails only on an error outside examples scores the
    mutant errored, not killed.
    A spec file that raised while loading, or
    a failing before(:suite) hook, made RSpec exit 1, and every mutant was
    scored killed. On a Rails app, rails_helper called SimpleCov.start
    after --rails had suspended Coverage (#228), so each spec file raised as
    it loaded, and 18 of 18 mutants were scored killed; 3 of them survive.
    Now the runner returns 2 when RSpec recorded an error outside examples and
    no example failed, so the mutant is errored: out of the score and in
    no_verdict, so that run now fails --threshold instead of passing at
    100%. A failed example still kil...
Read more

v1.6.0

Choose a tag to compare

@github-actions github-actions released this 08 Oct 07:56
a766c96

Added

  • An unplaceable mutant status. Under --strategy redefine, a mutant in
    a method whose class or module has no constant name is not run. It is left
    out of the score and, unlike uncapturable, out of no_verdict, so it does
    not fail --threshold. The human report gives it an Unplaceable: row, the
    HTML report a count, and the JSON report summary.unplaceable and an
    unplaceable[] list; schema_version is 1.7.
  • The agent skill installs with gh skill and npx skills (#125). It
    moved from docs/skill.md to skills/mutineer/SKILL.md, the layout of the
    Agent Skills specification, so gh skill install davidteren/mutineer mutineer finds it and npx skills add davidteren/mutineer --skill mutineer
    installs one SKILL.md instead of the whole docs/ site. The site build
    copies that file to skill.md, so its URL still works. The skill's
    description now says when to use it, and the README shows how to install it.
  • A ran_at_load mutant status (#187). A mutant on a method-body line
    that ran while the app booted or its class loaded gets this status when it
    survives, or when only the load ran the line. A kill stays a kill. It is
    left out of the score and out of no_verdict, so it does not fail
    --threshold. The human report gives it a Ran at load: row that points
    to --test-command to verify these mutants, the HTML report a count, and
    the JSON report summary.ran_at_load and a ran_at_load[] list, within
    schema 1.7.

Changed

  • Mutants no longer run under Coverage with --boot or --rails (#228). Coverage
    starts before the app boots so the coverage map can be built, and it used to
    keep running in every mutant fork after that, though nothing read it. It is
    now suspended once the map is built, from a fresh capture or from the cache.
    Coverage that a host process started before the run is left running.
    On a small app whose test calls tiny methods in a loop, the CPU time of a
    whole in-process --boot run fell from 0.76s to 0.57s (median of five). On
    the Rails fixture app the run is mostly boot, and the change is within noise
    (1.61s before, 1.60s after). --daemon runs its mutants in worker daemons
    that never start Coverage, so its time does not change (3.48s before, 3.44s
    after); the map-building daemon now suspends Coverage after the map too.
  • A mutant runs its paired test files first, then the cheapest (#203). The
    covering test files used to run in the order the coverage map stored them,
    so a mutant that one fast file kills could still score timeout when slow
    files ran first. Now the files that pairing finds for the source run first,
    then the rest from fastest to slowest by the time each file took during
    coverage capture, less the fastest file's time (the startup cost each
    capture pays). The order is fixed for a given cache. Costs compare in
    doubling buckets, so files of close cost keep their path order when a
    rebuilt cache measures them again; a file near a bucket edge can still move,
    so keep .mutineer/ between CI runs for the most stable --baseline. The
    order applies to Minitest (after the seeded class shuffle, with
    parallelize_me! classes still last) and to RSpec. --daemon runs every
    covering test with no stop, so the order does not change its verdicts. The
    coverage cache now saves these timings, so a cache from an earlier version
    rebuilds once. A complete run gives the same verdicts and --matrix rows,
    but the new order can change which mutants time out, so a --baseline gate
    can see a one-time change on upgrade.
  • Job selection moved from Mutineer::Runner to a new Mutineer::JobPlan
    module (#75).
    The in-process runner, the --daemon backend and
    --dry-run share it, so each file now requires what it uses and no
    require_relative cycle exists in lib/mutineer. The moved methods are
    collect_jobs, filter_since, coverage_selection, ran_at_load?,
    load_verdict, abort_if_unclean!, test_load_roots, source_dirs,
    sweep_orphans, suppress_map, suppressed? and result_keys. These were
    internal methods, so Runner keeps no copies. Runs, reports and caches do
    not change.

Fixed

  • An endless method that only the tests call is no longer no_coverage
    under --rails, --boot or --daemon (#209).
    Ruby counts no line when
    an endless method such as def half(x) = x / 2 runs, only its def line
    when it is defined. The boot defines it before any test runs, so the forked
    coverage capture never saw that line run. Coverage now also
    counts methods (Coverage.start(lines: true, methods: true)), and a test
    that calls such a method covers its def line. Coverage caches from earlier
    versions rebuild once.

  • module_function :name in a reopened module inside class << self
    promotes the method from the earlier opening (#216).
    Ruby treats both
    openings as one module, but Mutineer matched them only within one body, so
    it named the method #<Class:App>::M#c instead of #<Class:App>::M.c.
    The two openings now match by the module's full name. A module whose name
    Mutineer cannot be sure of still matches only within its own body: a path
    such as self::X or Foo::X written directly inside class << self, a
    name under an anonymous class, or a module in a class << self inside a
    method or a class_eval block. A module self::X in a block, or a
    self::X = ... in a class_eval block, matches only openings in the same
    block. A self::X like these now also stays apart from a
    module of the same name outside class << self, where a module_function
    in that module used to promote its methods.

  • A module self::X or class self::X in a block is unplaceable under
    --strategy redefine (#229).
    In a block, self is not always the
    enclosing class: in Other.class_eval do it is Other, and in
    Foo = Class.new do it is the new class. Mutineer gave these modules a
    known owner on the enclosing namespace, so redefine loaded the mutant onto
    a module the tests never use, and the mutant could falsely survive or
    score error. Such a module, everything nested in it, and a
    self::X = ... in a block that does not build a class now have
    their owner unknown: redefine reports their mutants as unplaceable, and
    --strategy reload still runs them. In a Data.define, Struct.new,
    Class.new or Module.new block, the module is now named under the
    builder's constant (Outer::Foo::X#c, was Outer::X#c), so the subject
    names and mutant ids of those methods change. self::X directly in a class
    or module body is unchanged.

  • --daemon worker databases start as a copy of the test database
    (#222).
    Each worker slot's SQLite database used to start from
    db/schema.rb alone. Rows the daemon wrote while it booted, in
    initializers or --require files, were only in the base test database, so
    a test that read them passed in-process but failed under --daemon. On its
    first use, a worker database is now a copy of the base test database (schema
    and rows, made with the SQLite online backup API). The worker still loads
    db/schema.rb when the copy's schema differs from it (another schema
    version, or another or no schema_sha1 checksum in ar_internal_metadata),
    for example when the test database is empty, out of date, or set up with a
    plain load "db/schema.rb". That load drops the
    copied rows of the tables it defines.

  • Standalone coverage capture loads the --require files (#217). A run
    without --boot requires the sources and then each --require (or
    require:) file before it forks the mutants, but the coverage capture
    loaded only the sources. A source method that a --require file called
    while it loaded was then not seen as run at load, so its mutant was a false
    survived instead of ran_at_load. The capture and the clean-suite check
    now load the --require files after the sources. With or without
    --boot, the coverage cache rebuilds when one of them changes, is added,
    is removed, or moves in the load order.

  • --daemon loads the --require files (#220). The daemon booted the app
    but never loaded the --require (or require:) files, so a test that
    needs one failed under --daemon and passed in-process. A source method
    that such a file called while it loaded was also not seen as run at load,
    so its mutant could be a false survived instead of ran_at_load. The
    daemon now loads the files after the app boots, as an in-process run does,
    and the daemon's coverage cache rebuilds when one of them changes.

  • The release workflow publishes only stable tags (#161). Its trigger,
    v*.*.*, also matches a prerelease tag such as v1.3.1-rc1, and the only
    guard checked that the tag matched Mutineer::VERSION. A prerelease tag
    could then publish the gem and move the floating major tag (v1) that
    Action users pin. The workflow now stops with a clear error before any
    publish step unless the tag is a stable vMAJOR.MINOR.PATCH.

  • Coverage capture keeps to capture_timeout from start to finish. Under
    --boot and --daemon, a hung test file ignored the timeout and blocked
    the run; it is now skipped as "timed out", like a standalone capture. A
    process that a test started and left running could keep the result pipe
    open and make a boot-mode capture wait for it to exit. Each capture now runs
    in its own process group, and a timeout stops the whole group, not only the
    test process. A capture whose test finished just before the deadline could
    also lose its valid result and report "invalid coverage output"; it now
    gets a short grace period to read the result. (#101, #129)

  • --daemon no longer waits forever on a daemon that stops answering
    (#101). A mutant whose verdict has not arrived 30 seconds after its own
    timeout is scored error, and the daemon is killed an...

Read more

v1.5.0

Choose a tag to compare

@github-actions github-actions released this 06 Oct 17:56
9893cc7

Added

  • --timeout SECONDS and --capture-timeout SECONDS, also timeout: and
    capture_timeout: in .mutineer.yml. They set the in-process per-mutant
    time limit (default 10s) and the coverage-capture time limit (default 120s),
    which were fixed. On a large Rails suite the clean run of the unmutated
    tests took longer than 120s, so every run that offered the whole suite
    stopped as not green, and a mutant on a line many tests cover took longer
    than 10s and was scored timeout. --daemon keeps its own per-mutant
    limit but uses --capture-timeout for its coverage capture;
    --test-command uses neither. Either backend warns about a limit it was
    given and does not use.

  • --cache-dir DIR, also cache_dir: in .mutineer.yml, sets where the
    coverage cache is written (default .mutineer, as before). Two runs from the
    same project root shared .mutineer/coverage.json and could overwrite each
    other's map; each can now have its own directory. --daemon uses it too.
    A blank value (an unset CI variable) exits 2 rather than writing the cache
    into the project root.

  • The README states how timeouts affect the score (#62): a timed-out
    mutant is left out of the score, and counts as a mutant with no verdict
    under --threshold.

  • --matrix reports which tests kill each mutant (matrix: true in
    .mutineer.yml). Each mutant runs every test in its covering files instead
    of stopping at the first failure, and the child sends each test's outcome to
    the parent as it is recorded. The report names blind tests, which ran in a
    complete row and killed no mutant, and redundant tests, whose every kill
    another test also makes. Redundancy is judged one test at a time, so delete
    redundant tests one at a time. The human report lists up to 20 of each, the
    HTML report all of them, and the JSON report moves to schema 1.5 with a
    matrix block that appears only with the flag. A test is its file and its
    id (the example id under RSpec), so examples that share a description stay
    apart and an example whose generated description changes with the mutant
    stays one test. Verdicts, the score and the exit code do not change: each
    mutant gets the verdict a run without the flag gives. After a serial test
    fails, an exit, a crash or the timeout in a later test, class or group
    leaves it killed; an end in the code that run still reaches (the failing
    test's class wrapper or group hooks, the suite's own cleanup) and a failure
    in a parallelize_me! class keep the exit status. A row is complete only
    when the child's stream arrived in order and every test reported (under
    Minitest the recorder saw every test, under RSpec every planned example),
    so a caught Interrupt leaves it incomplete. The human report names up
    to 20 incomplete rows and the HTML and JSON reports all of them; the human
    and HTML reports warn not to delete a blind test until its rows are
    complete.
    Minitest and RSpec 3.3+ are supported, on the in-process backend only;
    --daemon, --test-command, --fail-fast and --dry-run exit 2 with it.
    --no-matrix and --no-fail-fast beat the matching .mutineer.yml key,
    so a file that sets one can still run with a flag that conflicts with it.

  • no_coverage[] and uncapturable[] name each mutant. Every entry in
    the JSON report now has operator, token and id, as ignored[] has, so
    mutants on one line can be told apart and an id can go in ignore:. The
    entries are sorted by (file, line, operator, id). schema_version is 1.6.

  • The site and README explain "clean-room" and compare Mutineer with
    Mutant
    , and the landing page hero has a one-minute explainer video. It
    plays at 55% volume while in view (not with reduced motion; muted until
    the first click or key where the browser requires it), pauses when it scrolls
    away or the tab is hidden, and has an Enlarge button.

Changed

  • The human report has a Timeout: row, and the score line lists timeouts
    apart from errored mutants. Before, the Errored: row and the score line
    added the two together, while the JSON report kept them apart. The HTML
    summary now shows a timeout count apart from errored too.
  • A test file given as a source after --test exits 2. --test takes one
    file, so in mutineer run app/x.rb --test spec/a_spec.rb spec/b_spec.rb the
    second spec became a source to mutate, and the run tested with one spec file
    without a word. Now a source named *_test.rb or *_spec.rb inside a
    test/ or spec/ directory, given with --test, is a usage error that says
    to repeat --test. The usage line now reads --test <test> [--test <test>...].

Fixed

  • The RSpec runner keeps what gems configured before the run. It called
    RSpec.reset, which drops the whole configuration. Under rails: true, a
    --boot file or require:, the app's gems load first, so a setting such as
    rspec-retry's verbose_retry was gone, and a support file that set it
    raised NoMethodError. The run stopped with "the unmutated suite is not
    green". The runner now calls RSpec.clear_examples, which keeps the
    configuration. Hooks that gems added now apply too: an example with
    rspec-retry's retry: 3 runs up to 3 times per mutant, as under rspec.
  • --since no longer scores a file that Git ignores (#168). A file
    outside the index is scored in full, so a new file is not missed. That
    check also caught files that Git ignores on purpose (.gitignore,
    .git/info/exclude or the global excludes file), such as generated code
    inside a directory run. Those files now get no mutants under --since,
    and a warning names each one. Other untracked files are still scored in
    full, and so is a file when Git cannot tell, such as one beyond a symlink or
    inside a submodule.
  • The human report escapes control characters from the source (#163).
    A surviving line that held a terminal control byte (for example ESC) was
    printed as that byte, so it could change what the terminal showed. The
    token, the replacement, the diff lines, method names, file paths and the
    kill matrix's test names now print each control character except tab as
    its Ruby escape (\e), and a byte that is not valid UTF-8 (a Latin-1
    source) as \xNN. The JSON and HTML
    reports already escaped text and do not change.
  • One edit on a line is scored once (#159). Some opt-in operators emitted
    two mutants that give the same source: literal_mutation on 0 (the
    "change to 1" and "add 1" rules), negation_removal on !!x (either !
    removed), and chain_link or operand_removal on a repeated part such as
    a.b.b.c or x || x. Each copy had its own id, so one surviving edit
    counted twice. Now only the first copy on a line runs. Copies that start
    on different lines (a multi-line chain) both stay, so --since never
    loses the edit. Ids are assigned before the copy is dropped, so every kept
    mutant keeps its id, and the dropped copy's id simply stops appearing.
    Default runs do not change. If you use these operators with --baseline,
    regenerate the baseline after upgrading: the score can move with no code
    change, and in a full (unscoped) comparison a dropped copy that survived
    is listed as fixed. An ignore entry for one copy's id still ignores only
    that copy; the other copy still runs.
  • # mutineer:disable-line inside a string no longer silences a line
    (#158). The marker was found with a text search, so a string such as
    "# mutineer:disable-line" ignored every mutant on its line and left
    them out of the score. Only a real # comment counts now. A heredoc or
    regex that holds the text is ignored too.
  • ./lib/x.rb and an absolute path pair with a test (#104). Auto-pairing
    found the test for lib/calc.rb but not for ./lib/calc.rb or the
    absolute path to the same file. It reported "no test found by convention"
    and exited 2. A file argument inside the project is now made relative to
    the project root first, so equivalent spellings (./, .., absolute)
    pair with the same test and run once. A symlink keeps its own name, so it
    stays a separate source from its target. Reports show that root-relative
    path. A path outside the project stays as typed.
  • A mutant on another line of a multi-line statement runs its tests
    instead of being reported as no_coverage. Ruby counts one line of a
    statement only. When the mutant's own line has no count, the runner now uses
    the tests that ran the statement that holds the mutation. A heredoc body
    belongs to its statement. This fallback skips a first or last line that also
    holds other code, such as the def line, and the body of x while c and
    x until c. It also gives no tests to a mutant in code of the statement
    that runs only sometimes: a when or in condition, a rescue class
    list, a parameter default, the rescue side of x rescue y, the value of
    ||= or &&=, the operand of defined?, the right side of a || or
    a &&, a branch of x if c, x unless c, a ternary or else, the
    pattern of v in p or v => p, an
    argument or block of x&.m(...), and the value of x&.m += v. Those
    mutants stay no_coverage, as before.
  • A survivor diff for a multi-line mutant applies with git apply (#106).
    The JSON diff header always read @@ -N +N @@, even when the mutant
    removed two lines and added one, so git apply rejected the patch as
    corrupt. The header now carries each side's line count (@@ -3,2 +3 @@).
    A one-line diff keeps its old header. A mutant that empties a line now
    shows that empty line as a + line. A side whose last line has no final
    newline gets \ No newline at end of file. Lines of a CRLF file now keep
    their \r, so the patch applies to that file.
  • Split test files pair with their source (#87). A source such as
    app/foo/bar.rb now also uses `test/foo/bar_upsert_test....
Read more

v1.4.0

Choose a tag to compare

@github-actions github-actions released this 30 Sep 15:17
c487934

Added

  • Condition-forcing operators (Tier-2, opt-in via --operators):
    condition_true and condition_false replace an if/elsif/unless,
    ternary, modifier or case/in guard condition with (true) or (false), so
    its branch always runs or never runs. A surviving mutant means no selected
    test detected the forced condition. A literal condition, also in
    parentheses, makes no mutant: forcing it changes nothing or repeats the
    boolean_literal flip. A condition that holds a heredoc makes no mutant.
    A condition that assigns a variable keeps its code, and only its value
    is forced ((m = x; true)), so later code still sees the variable.
    The never-runs side of an else-less conditional can
    be the same program as the nil that statement_removal or return_nil
    puts in its place; it is still made, so these mutants and their ids do not
    depend on which other operators run.
  • Operator-assignment operator (Tier-2, opt-in via --operators):
    operator_assignment swaps the operator of a compound assignment:
    += <-> -=, *= <-> /=, and %= and **= -> *=. It covers local,
    instance, class and global variables, constants, calls (a.b += 1) and
    index calls (a[i] += 1). The arithmetic operator never sees these
    forms, because Prism does not parse them as calls. The operator does not
    change ||=, &&=, or the bitwise and shift forms (|=, <<=).

Fixed

  • A typed --rails, --verbose or --debug beats rails: false and
    verbose: false in .mutineer.yml
    (#103). The flag was dropped, so the
    run went on without Rails boot or verbose output. The config layers now keep
    only the keys the user wrote, and "did the user write this" comes from those
    keys, not from a hand-kept list.
  • framework: rspec in .mutineer.yml survives test pairing (#103).
    Pairing re-detected the framework from the test file names, so an RSpec
    suite in test/calc_test.rb ran under Minitest.
  • A bad number is an error, not a rounded value (#105). --jobs 1.9 and
    --baseline-epsilon abc no longer become 1 and 0.0. One option schema
    parses each value once, where it enters, for both the command line and
    .mutineer.yml.
  • --since keeps a source file that git does not track yet (#156). Before,
    such a file counted as unchanged. The run scored no mutants, and a positive
    --threshold still exited 0. The file is now new in full, so all its
    mutants run.

Changed

  • These now exit 2 with a message that names the option: jobs: 1.9 or
    jobs: true in .mutineer.yml (before: 1, or a crash); a bad, negative
    or non-finite --baseline-epsilon (before: 0.0); a boolean key in
    .mutineer.yml that is not true or false, such as the string "yes"
    (before: false).
  • --jobs takes plain digits only. 0x2, +2, " 2" and 1_0 exit 2.
    Before, Integer() read them as 2, 2, 2 and 10. --threshold and
    --baseline-epsilon take a plain decimal, such as 2 or 2.5, from a
    string. 0x10, +2, 1_0 and 1e2 exit 2. Before, Float() read them
    as 16.0, 2.0, 10.0 and 100.0.
  • A string option that gets true, false or no value exits 2, such as
    only: false or baseline: in .mutineer.yml. A blank since also exits 2,
    in the file and on the command line, so an unset shell variable does not
    turn scoping off. since: false still means no scoping.
  • Config file errors start with the file and key, for example
    .mutineer.yml: threshold must be a number between 0 and 100, where they
    started with --threshold.
  • The docs site is built in CI — a Pages workflow runs rake site:build
    and deploys the result, so the YARD HTML under /api/, llms-full.txt,
    json-schema.html and sitemap.xml are no longer committed. CI checks
    that the site builds, in place of rake yard:pages:check. The site keeps
    its URLs (#153).

v1.3.0

Choose a tag to compare

@github-actions github-actions released this 29 Sep 06:19
4e8629c

Changed

  • Mutant ids now include the project-relative file path (#126). Before, two
    mutants with the same method name, operator and token in different files got
    the same id. That happened with a top-level def or block in two files, and
    with a class reopened in another file. One ignore: entry then suppressed
    both mutants, and --baseline could hide a new survivor behind an old one.
    • Every survivor id changes once in this release. This affects any external
      tool that tracks survivors by id.
    • Moving or renaming a file now changes its ids.
    • Ids follow the project root: the directory mutineer runs from, or the
      Action's working-directory. A run from a different root gives different ids.
      mutineer finds .mutineer.yml by walking up, so a run from a subdirectory
      prints one [mutineer] warning that the loaded ignore ids will not match.
    • A source outside the project root uses its absolute path, so its ids differ
      between machines.
    • Two methods with the same qualified name in one file (for example two
      top-level def index in two DSL blocks) now get different ids. The second
      and later ones hash their position among those methods; the first keeps its
      id.
  • The JSON report marks its id format (schema 1.4, additive).
    summary.id_format is 2 for ids that include the file path.
    summary.legacy_id_matches counts old-format ignore: entries (ignore) and
    survivors matched only through an old-format baseline id (baseline). The
    GitHub Action shows one warning annotation when either count is not zero.

Deprecated

  • Old-format ids in ignore: and in baselines. Matching on them is removed
    in 2.0. Replace each old ignore: entry with the new ids from the warning
    (only the intended ones when it over-matched several mutants).
    Regenerate a baseline (--format json) only after every gate that reads it
    runs this version or later (the Action's version: pin, your CI
    Gemfile.lock). An older version treats every new-format survivor as new.

Fixed

  • module_function :name and module_function def name promote only
    their own module's methods
    — a class or module in the same file with a
    method of the same name kept an instance method in Ruby, but mutineer
    named it as a class method. --strategy redefine then mutated a method the
    tests never call, so a killable mutant falsely survived, and
    --only Class#name selected nothing (#98). The affected methods now get
    their correct names, so their mutant ids change: regenerate any ignore
    entries or baseline survivors that pointed at them.
  • A root-anchored reopening names the top-level constant — a
    module ::Root or class ::Solo written inside another module now gives
    Root and Solo, not Outer::Root and Outer::Solo. This applies to every
    method in such a body, with or without module_function, so their mutant
    ids change too. --strategy redefine rebuilds such a body's scope as
    written (module Outer then class ::Solo), so a constant from Outer
    still resolves in the mutated method, as it does under reload. Before,
    the method raised NameError in the test, which counted as a false kill
    (#145).
  • Old-format ids keep working, with a warning (#126). An old-format
    ignore: entry still suppresses the mutants it matched before. The run prints
    one [mutineer] warning per entry with each new id, its file and its method.
    When the entry matched several mutants (in different files, or same-named
    methods in one file), the warning says it
    over-matched and to keep only the ids for the mutant you meant to ignore. A
    --baseline file without summary.id_format matches on new ids, or on old
    ids from the same file, so no survivor reads as new or fixed only because its
    id changed. A stored file outside the project root (a baseline written on
    another machine) matches on the old id alone. The run prints one
    [mutineer] warning to regenerate the baseline.

v1.2.0

Choose a tag to compare

@github-actions github-actions released this 28 Sep 21:04
0274938

Added

  • Operand-removal operator (Tier-2, opt-in via --operators):
    operand_removal replaces a && b with (a) and with (b), and does the
    same for ||, and and or. The mutant survives when no test needs the
    operand that the mutant removes. The operator never keeps a jump operand
    (return, break, next, redo, retry) alone, because a jump does not
    parse in a value context. It never removes an operand that holds a heredoc,
    because the heredoc body stays behind as code. It skips nested method
    definitions, because mutineer mutates each one as its own method.
  • Array-literal operator (Tier-2, opt-in via --operators):
    array_literal replaces a non-empty array literal, such as [a, b] or
    %i[a b], with []. The mutant survives when no test checks the contents
    of the array. The operator skips an implicit array (x = 1, 2), an array
    that holds a heredoc, and nested method definitions.
  • Sources also pair with Minitest's test/**/test_*.rb files — after the
    _test.rb forms, so existing projects pair as before. lib/helper.rb does
    not pair with the test/test_helper.rb support file. A failed capture of a
    test_<name>.rb file now marks <name>.rb uncapturable, as <name>_test.rb
    does (#120). Without framework: set, a source with spec/<name>_spec.rb
    and test/test_<name>.rb but no <name>_test.rb now pairs with the
    Minitest file, as the "Minitest first" order says.

Fixed

  • reload loads the mutant by an absolute path — a relative source path
    gave the mutant relative backtrace paths, so code that checks its own frames
    by absolute path failed for every mutant, a false kill (#123).
  • require "test_helper" works without RUBYOPT — a standalone run puts
    lib, then each test file's test_helper.rb directory, on the load path,
    as boot mode and rake test do. A run where no test records coverage because
    captures failed now exits 1 instead of reporting N/A (#119).
  • A disable-line marker warns about an operator it does not know — a
    reason written without -- became part of the operator name, so the marker
    suppressed nothing and said nothing (#124). A marker followed only by spaces
    or commas, such as disable-line -- why, now disables the whole line.
  • Coverage capture and the clean check run each source once — they read
    sources with load, so a test's own require ran them again: a Struct
    superclass raised superclass mismatch, and load-time code ran twice (#122).
    A mutant of such a class still errors under --strategy reload, which loads
    the mutated file again; --strategy redefine runs it.
    Code that guards itself to run once (unless defined?(X)) can now show as
    covered, so its mutants run where they were no_coverage before.
  • A red unmutated suite now shows why it failed — in a standalone run, the
    Minitest summary or RSpec output of the failing test, with its failure
    message, goes to stderr before the "not green" error. A passing run prints
    nothing extra. Boot mode (--rails, --boot) is unchanged (#121).