Repository navigation
Releases
v1.9.0
Added
- New opt-in operator
rescue_bodymutatesrescueclauses (#418).
It replaces the whole body of eachrescueclause withnil, including a one-statement body.
It also turnsx rescue yintox rescue nil.
A re-raise or aretrybody is mutated too, becausenilswallows the error.
An empty body, a body that is alreadynil,(nil)or(), and anensurebody give no mutant.
A body whose value is discarded and that holds only literals and variable reads is declined, and counts insummary.declined_by_operator.
Seedocs/equivalent-skips.md, "Rescue body".
Before this, no operator mutated a one-statementrescuebody.
The default operator set does not change. return_nilreaches a method with a def-levelrescue,elseorensure(#418).
It replaces the final expression of the body, or of theelseclause when there is one, withnil.
It does the same for eachrescueclause. Theensurevalue is not a return value, so it is left alone.
Before this,return_nilgave such a method no final-expression mutant.
A run withreturn_nilcan now report a different number of mutants and new survivors. With--baseline, refresh the baseline.
With both operators, a one-statementrescuebody gives the same edit twice. The run keeps thereturn_nilmutant, lists therescue_bodyid in itscopies, and counts that copy indropped.
A final explicitbegin ... endblock gets the same mutants as the def-level form, so RuboCop'sStyle/RedundantBeginautocorrect does not change them (#470).runsays when anignore:entry matches no mutant (#377).
The warning names the id and says the list covers only that run's sources and operators.
The warning does not change the exit code.
--format jsonlists the ids insummary.unmatched_ignores, in file order, each id once.
--dry-runprints the same warning, and--dry-run --format jsonlists the same ids.
--sincedoes not warn about an entry whose mutant is outside the diff: matching happens before the diff narrows the run.
A run of some files warns about entries those files and operators do not match. The warning says so.
A source you passed that has no paired test still counts: its entries are not reported as unmatched, andrunagrees withrun --dry-runof the same files.
--onlyskips the check. The list is then empty, which does not mean every entry matched.
A dropped repeat of an edit counts as matched.
An old-format id that still matches is not in this list. The old-id warning still names it.
mutineer migrate --dry-runover every source lists entries that match nothing and exits 1.
That exit code is the CI check, in 1.x and after 2.0.
The key is part of schema1.10.- A report names the dropped copy of an edit (#374).
When two operators, or one operator twice, make the same change on one
line, one copy runs and the other is dropped. The JSON report now lists
each dropped id on the kept mutant ascopies, on survivors, ignored
mutants, and the other mutant rows.summary.droppedcounts them. They
stay out oftotaland out of the score.mutineer triageprints an
ignore:entry for the survivor and for each of those ids, so one round
writes every entry. An entry still ignores only its own id. The human
report, the HTML report, the dry run, and the Action summary show the
count. A dropped copy of an ignored mutant is listed on the kept mutant,
so that entry no longer looks unused. Both keys are part of schema1.10. - A source that cannot load alone points at
boot:(#376).
When a source raisesNameError(includingNoMethodError) while it
loads, and the run has noboot:file, stderr adds one line. The names
it needs come fromboot:or--boot, which loads before the sources.
Listing the entry file as an earlier source also loads it first.
In that run,require:files still load after the sources, so a source
cannot use them while it loads. With a boot file they load after the
boot file, and the parent does not require the sources.
--test-commanddoes not loadrequire:files. The hint line is absent
for any other load error, and for aNameErrorin the boot file or a
require:file. A dry run does not load the source, so it does not print
the line.--daemonand--test-commanddo not require sources in the
tool process, so they do not print it either.--helpdescribes--boot
as an app entry point or a gem'slib/<name>.rb, and no longer limits it
to Rails apps. - A run counts sources that have no paired test (#381).
The report names each skipped source and how many mutants it holds.
JSON addssummary.unpaired_sources,summary.unpaired_mutants, andunpaired_sources[].
The human report, the HTML report, and the dry run show the same counts.
Unpaired mutants are excluded from the score and cannot fail--threshold.
They join the unrun-share stderr warning, except under--fail-fast.
That warning does not change the exit code.
Schema version is1.10.
Compare it as two integers, so1.10follows1.9.
On Fizzy atd5b7b749d, 106 sources and 312 mutants were unpaired.
The paired run total was 759, so the share was 29.1%.
On Fizzy atf220e4832, 92 sources and 149 mutants were unpaired.
The paired total was 922, so the raw share was 13.9%.
After the 10 ignored mutants leave the denominator, the share is 14.0%.
Mutineer now parses those sources.
A disable-line warning or a git-diff warning can name them.
The Action step summary does not show these counts (#481).
A dry run with--railsor--bootand no--teststill exits 2 (#410).
#349 measured the unrun share (of attempted plus unrun) on Fizzy atf220e4832, with Mutineer at46ee2bb3.
A full daemon run with the default operators had 840 attempted mutants, 1 unplaceable, 0 skipped (invalid) and 149 unpaired.
Without the unpaired mutants, the share was 1 of 841 (0.12%).
With them, it was 150 of 990 (15.2%), so the warning fired.
With all 21 operators, the four paired files that can hold an unrun mutant had 24 under redefine and 20 under reload.
Against the default run's 840 attempted mutants, that is at most 2.8%.
The 10% limit stays.
A share above it that comes from unpaired mutants is a true count, not noise.
The 2 skipped (invalid) mutants in the all-operator runs came from aliteral_mutationbug that this release fixes (#469). - A concern pairs with the test that mirrors its namespace (#379).
app/models/concerns/storage/totaled.rbcan pair withtest/models/storage/totaled_test.rb.
That path matchesStorage::Totaled.
Theconcerns/path still wins when it has a test.
Existing pairs stay the same.
Before this, the concern was skipped and added no score.
The skip line wasno test found by convention.
A concern that was skipped can now run.
That can change a score or a gate.
--baselinetreats new survivors as new.
Refresh the baseline.
Seedocs/ci.md, "Refresh the baseline".
A failed namespace load can move a--thresholdrun from exit 0 to exit 1.
Those mutants move fromno_coveragetouncapturable.
Anilscore with a broken mutant then fails the gate.
The share of mutants with no verdict also rises.
Withuse-bundler: false, an Action with an emptyversioninstalls the latest gem.
Pinversion, or refresh the baseline, before the next run.
Withuse-bundler: true, the Gemfile picks the version.
A failed namespace test does not mark a concern that has a test at itsconcerns/path.
A namespace split name pairs under #408. - A concern pairs with a split test at its namespace path (#408).
app/models/concerns/storage/totaled.rbcan pair withtest/models/storage/totaled_bytes_test.rb.
Theconcerns/path still wins when it has a test that pairs, exact or split.
A split owned by a longer source does not count.
A longer source keeps its file when that source pairs with it.
That includes a longer concern with no test of its own, and a model orlib/file of the longer name.
A model keeps a split file it already pairs with. The concern may use that same file. The model does not lose it.
A shorter concern keeps a namespace split when the longer concern already has a test at theconcerns/path.
The longer concern does not add that file.
A namespace spec stays the only file. A failed split beside it does not mark the concern.
Before this, a namespace split name left the concern skipped, with no score.
A concern that was skipped can now run. That can change a score or a gate.
--baselinetreats new survivors as new. Refresh the baseline.
Seedocs/ci.md, "Refresh the baseline".
A failed namespace split moves those mutants fromno_coveragetouncapturable.
On the Fizzy tree measured for #379, no unpaired concern had a namespace split name.
One of 352app/sources paired with a split file. - Mutineer now reports the methods that got no mutants (#386). A
method whose body is oneifwith arescuegets no mutant from the
default operators, and the run did not say so. The dry run summary now
ends withno mutants in N of M methods, and--dry-run --verbose
prints oneno mutants:line per method. The human report has a
No mutants:row. The JSON report addssummary.subjects,
summary.unmutated_subjectsand a top-levelunmutated_subjects[]
list, andschema_versionis now1.10. A program that calls
Reporter#reportwithout the method counts getsnullfor all three,
not0; the CLI always fills them. An empty run says how many
methods it found and points to--list-operators. Scores, exit codes,
the mutants that run and the default operators do not change. The
README namescondition_trueandcondition_falsefor these methods.
For a one-statementrescuebody, use the new opt-in `rescue_...
v1.8.2
Fixed
- Redaction no longer hides a password inside other words (#341). A
form of the database password (raw, decoded or escaped) longer than 3
characters, made only of ASCII letters, digits and underscores, is now
hidden only as a whole word, like a short one. With
passwordtest, the backtrace pathuser_test.rbstays readable. Such a
password inside a longer word is no longer hidden. The wordteston its
own is still hidden, also right after an escape such as%40or\n, a
colour code, or a-pflag. That escape rule now covers short forms
too. A longer form with any
other character is still hidden wherever it appears. - Redaction keeps the host after a password with an
@(#341). In
postgres://u:p@ss@h/db and p@ss, onlyp@ssis hidden now. Before, the
host and the rest of the line were hidden too. One known limit: in a URL
other than DATABASE_URL, a password that holds an@, a host, a/,:,
?or#, and a space, shows the part after that@when the URL's own
host has nothing after it, or has an@in its path or query. #341 tracks
it. - Daemon shutdown no longer waits a second (#341). A process the app
started could keep the daemon's error output open. Each quit or respawn of
--daemonthen waited one second. Mutineer now stops waiting once the
daemon has exited and its output is read. - One line that Prism 2 rejects no longer skips every mutant in the file
(#392). Prism 2.0 and later report an operator write to a bare constant
inside a method (T += 1,T ||= 1,T &&= 1) as a dynamic constant
assignment. Ruby'sparse.ydoes the same, but Ruby 3.4 and 4.0 load such
a file with their own Prism 1. Mutineer kept a mutant only when the mutated
file parsed with no errors, so on Prism 2 every mutant of that file was
counted asskipped (invalid). When Prism reports an error, Mutineer now
asks the running Ruby to compile the mutant, and skips it only when Ruby
cannot. This holds in the dry run, in process, with--test-command, on
--daemon, and for the redefine strategy's method snippet. A project gets
Prism 2 when its bundle loads it, for example
through minitest 6.1. On Prism 2, the mutants of such a file that were
skipped now get a verdict, so the score, the counts, and the JSON report
can change for those projects. Prism 1 runs do not change. --test-commandwarns about a random test order (#338). Mutineer
sets no seed for your command, so the clean run and each mutant run can
run the tests in different orders. An order-dependent suite can then fail
a mutant run for a reason the mutant did not cause, and that mutant
scores a falsekilled. When the clean run prints a random seed
(MinitestRun options: --seed N, RSpecRandomized with seed N) and
Mutineer finds no fixed seed, it now prints one warning with the fix for
that runner: putSEED=1before themutineercommand for Minitest, or
add--order definedto an RSpec command and keep--order randout of
SPEC_OPTS, which RSpec reads after the command. The docs for
--test-commandsay the same. Mutineer does not set the seed itself, so scores do not
change. It cannot see a seed set in a wrapper script or.rspec, or a
Minitesttest_order, so the warning can print when the order is
already fixed. #368 tracks these limits.- Reload places a hook-block mutant before the rest of its file runs
(#348). Under--strategy reload, a placeable mutated method from a
concern'sincludedorprependedblock reached the classes that include
or prepend the concern only after the whole mutant file loaded. Methods
that reload cannot place still scoreunplaceable. A later line in the same
file that called the method while the file loaded, such as
TOTAL = Host.new.doubled(2), still used the original method. Mutineer
now places the mutated method when the load reaches the hook call, before
the rest of the file runs, in-process and on--daemon. Such a mutant can
now be killed where it used to survive or scoreran_at_load, so the
score of an affected file can change. A mutant that raises in such a
load-time call now scoreserror. The alias and kept-method checks
still scoreerror. The kept-method check runs after the load, so a
method handle the file holds only while it loads does not count. - The Action creates the cache folder when
outputpoints into it
(#340). On a fresh checkout,output: .mutineer/pr.jsonexited 2,
because.mutineer/did not exist yet. The CLI already created that
folder (#270), but the Action checks the path itself before it runs the
CLI. The Action now creates the cache folder when it is the missing parent
ofoutput:.mutineerby default, thecache_dirfrom.mutineer.yml,
or a--cache-dirinextra-args(or a prefix of it down to--cac, such as
--cache). Any other missing folder still exits 2. - Reload runs a Rails class body a second time without errors (#378,
#384). Reload loads the mutant file over the class that is already
defined, so its body runs again. A model that declares anenum, and a
class that callsskip_callback(a controller'sskip_before_actionor
allow_unauthenticated_accesscalls it), raised on that second run, and
every mutant in the file scorederror. While the mutant file loads, an
enum the class had before the load now skips Rails' conflict check, and
skip_callbackdoes not raise for a callback the first run already
removed: one named by a method the class has, such as a callback from the
parent class or an included concern. A skip that names no such method or
parent callback still raises. Both checks run as usual while the tests
run. This holds on--daemonand with--strategy reload. - Reload no longer runs validators and block callbacks twice (#390).
The second run of a class body added a second copy of eachvalidates,
eachvalidateblock, and each callback written as a block or a lambda,
such asbefore_save do ... end. Rails removes a repeated callback only
when it names a method. A test that counts errors or side effects then
failed for every mutant in the file, so those mutants scoredkilled
falsely. After the mutant file loads, Mutineer now keeps one copy of each:
the new one, in the place of the old one, so the order of callbacks does
not change. Entries from an included concern stay, because the reload does
not run the concern's block again. When Mutineer cannot tell which old
entry a new one repeats, it keeps both, as before (#413). This holds on
--daemonand with--strategy reload. With eager loading on, a mutant
of a parent class file still adds a callback back to each subclass that
skipped it (#414).
Score change: a Rails model with such a test can now show survivors
that were false kills before, so its score can go down. - Concern mutants load when the test app loads lazily (#383). With
eager_loadoff (the Rails test default), the mutant's load was the first
use of the concern, so Rails loaded the original file inside it. The
originalincludedblock then came back, and the mutant's block raised
MultipleIncludedBlocks. Mutineer now loads the original file first, through
the app's autoloader, so the mutant's block replaces it. - The pending-migration check no longer trips
shared_databases(#385).
rails/test_helpchecks every database for pending migrations when the
tests load, shared ones too. A parallel--daemonrun then reported the
shared database as used and scored every mutanterror. Mutant forks of a
run withshared_databasesnow turn that check off. The clean check, which
runs before any mutant, still makes it. - No Rails deprecation warning from concern mutants (#387). Placing a
mutant inside anincludedblock looked at every module. Rails 8.2 keeps a
deprecated constant as a module that warns on every call, so each such
mutant printedLoadInterlockAwareMonitor is deprecated. Mutineer now skips
that module.
v1.8.1
Fixed
- No "already initialized constant" warnings under reload (#364). Each
mutant loads its whole file again, so every constant in the file was
assigned twice. Ruby then printed two warning lines per constant, per
mutant, on stderr. Mutineer now turns warnings off while it loads the
mutant file, in-process and on--daemon, and turns them back on before
the tests run. Warnings that the tests raise still print. Other warnings
raised while the mutant file loads, such as an unused variable under-w,
no longer print either. Scores and the JSON report do not change. - The "not green" message says it once (#360). When the clean run of a
file's tests failed, the message said "the unmutated suite is not green"
twice. It now says it once, then names the test files afterFailed:.
A failure that only the check of all test files together finds is listed
asFailed: combined suite.
Under Minitest, the seed hint follows them.
The message itself still does not name the failing tests or the first
error; #360 tracks that. On the default in-process path, the failing
test's own report still prints on stderr before the message. - An unexpected error no longer prints an unredacted message (#341).
An error that no other handler expected went to Ruby's default printer,
which did not hide a database password in its message. The CLI now prints
that error with the password hidden, with its backtrace, and exits 1, as
before. No known path raised such an error with the password. - An ambiguous or malformed option is a usage error (exit 2). An option
that matches more than one flag, such as--ver(--verboseor
--version), or a value given to a flag that takes none, such as
--verbose=1, crashed with a backtrace and exit 1. It now prints one line
and exits 2, like every other bad option. This holds forinitand
triagetoo.
v1.8.0
Highlights
Read these before you upgrade. Details are in the sections below.
- Scores can change after the upgrade. Under
--strategy reload, most methods
in a concern'sincludedorprependedblock are now scored (#297). Runs
that listcall_to_receiverget more mutants (#274). Runs that list
rangeget new mutants for endless ranges (#299). A mutant that two
operators make now counts once (#275). These can change counts, scores
and survivors. If you gate on--baseline, write a new full baseline. - Delete
ignore:entries forrangemutants (#299). A newrange
mutant can take an old id, so an old entry can hide a real survivor. - Fewer false kills. Under
--test-command, a non-zero exit other than 1
scoreserror, and so does an exit 1 with no failing test in the summary
line (#292). An Active Record connection or lock error inside a Minitest
test scoreserrortoo (#296). Before, these scoredkilled. - Exit codes match STABILITY.md (#288). A boot, source or
require:
file that raises while it loads exits 1. A missing boot orrequire:
file exits 2. --thresholdcompares the exact score (#280). A run with 1999 of
2000 mutants killed shows 100.0% but no longer passes--threshold 100.- New report data. The JSON report adds
summary.declinedand
summary.declined_by_operator(schema1.9), and stderr warns when many
mutants could not run (#291). - Parallel
--daemonwith unused test databases (#316). List them under
shared_databasesto run more than one worker. Needs Rails 7.1 or later. - The gem ships the agent skill (#315).
- Releases wait for a maintainer. Nothing publishes on a schedule or on
a tag push.
Added
shared_databasesconfig key (#316). An app with a second test
database ran one worker under--daemon. List the databases your tests
never use (shared_databases: [cable]), and the workers share them and run
in parallel. After each mutant, a parallel worker checks those databases.
The mutant scoreserrorwhen a pool for one is still connected after the
tests, or ran SQL during them. A replica in the sameconnects_tocounts
as its database. Listing the primary database stops the run with exit 2.
A name that matches no test database warns and is not checked. The check
needs Rails 7.1 or later. Three safety changes come with it for
--rails --daemon: a test database withdatabase_tasks: falsenow
counts toward the one-worker limit, and the run keeps one worker when the
app's databases cannot be listed or the first daemon does not boot
(before, each case could start several workers). See
More than one database
for the transactional-tests caveat.- Counts of mutants that were not scored (#291). Operators now count the
candidates they decline: a heredoc they cannot mutate safely, or an
equivalent skip such as an endless range or a skipped chain-link name. The
dry run prints the count. The human report prints it when the run has
mutants; when every candidate was declined, the empty-run warning names the
count instead (that warning is off under--allow-empty). The JSON report
addssummary.declinedandsummary.declined_by_operator(schema1.9).
A new stderr warning fires when unplaceable and skipped-invalid mutants are
more than one and more than 10% of the mutants a run tried to score. The
warning does not change the exit code. Measured on commit caa9e1a (this
change merged with main) with every operator: the dry run onlib/
declines 195 candidates beside 17,374 mutations (call_to_receiver 122,
chain_link 45, range 12, statement_removal 10, return_nil 6); the dry run
on the Rails fixture app declines 0 beside 69 mutations. A
--strategy redefine --railsrun on the fixture app models (order,
price_list, billing/fee) has 0 unplaceable and 0 skipped-invalid of 12
attempted, so no warning. The fixture has no concerns, so it cannot tune
the 10% limit. - Included and prepended block methods under reload. After you upgrade,
follow the full baseline refresh recipe
with--no-since --format json --output .mutineer/refreshed-baseline.jsonand your
existing source and test options. Review the report and replace the stored
baseline. This applies only to runs that use--strategy reload: under
redefine these methods stayunplaceable. GitHub Action users on reload,
the Action's default (a Rails app without a daemon uses redefine), must
replace the file passed to thebaselineinput too. A check against an older
report lists the new survivors as regressions.
Under--strategy reload, in-process and on the daemon, a method in a
concern'sincludedorprependedblock is now mutated and scored (#297).
Mutineer loads the mutated method into each class that already has it. A
prepended wrapper that callssuperreaches that mutated method too. A
concern included into an object's singleton class is supported. A class
that includes the concern later gets it from the reloaded block. The
rest of the block does not run again, so validators and callbacks are not
added twice. These mutants used to beunplaceable, so a reload score can go
down and a--thresholdgate can start to fail. Some methods stay
unplaceable: a method in a class, module orStruct.newblock inside the
block, a method inside another method or block, a method with an explicit
receiver other thanself(such asdef base.taginincluded do |base|),
a hook-block method whose source line conflicts with anotherdefor an
explicit Ruby method creator (the full rule is indocs/json-schema.md),
every method in a block that usesaliasoralias_method, a method the
block itself calls, and every method in a block whoseincluded docall is
not a plain line of the module
body (for example insideif,unless, orx = included do). The full list
is indocs/json-schema.md. If a class aliases a method that Mutineer does
run (one not in that list) outside the block, the mutant scoreserror,
because the alias keeps the original method. This also applies when a
wrapper replaces the original name and calls the saved alias. A retained
MethodorUnboundMethodfor the original also makes the mutant score
error: a wrapper can call that handle without testing the mutated method.
--strategy redefinestill
reports all of these methodsunplaceable. - The gem ships the agent skill.
skills/mutineer/SKILL.mdis now
packaged in the gem, and the gemspec opts in as a
Rails Hyperdrive
companion (hyperdrive_targets: "*",hyperdrive_artifacts: "skill"). In a
Rails app that bundles mutineer,bin/rails hyperdrive:initinstalls the
skill to.claude/skills/mutineer/SKILL.md. A test in the default suite
fails if the gemspec stops shipping the skill or drops the opt-in (#315).
Fixed
--daemonstops on MySQL older than 8.0 (#298). The support matrix said
MySQL 5.7 is not supported, but--daemonran on it. Mutineer now reads the
server version before it copies any worker database. MySQL older than 8.0
stops the run withcould not provision <base>: MySQL <version> is not supported. Pass--no-daemon(or setdaemon: false) to stay serial.
MariaDB, MySQL 9 or later, and a server version Mutineer cannot read print
one warning and run.- An in-test database error scores
error, notkilled(#296). When
the first failing test of a Minitest mutant run failed with an Active
Record connection or lock error (ConnectionNotEstablished, including pool
timeouts,ConnectionFailed,Deadlocked,LockWaitTimeout), the mutant
now scoreserroron--daemonand in-process. This includes an error in
setup. Errors are out of the score, so a lost database connection during
a run no longer raises the score. A stderr line names the error. A mutant
that really causes a deadlock, or that breaks connection-handling code, now
also scoreserror. Moreerrorverdicts can fail a--thresholdrun
through the existing no-verdict gate. Other database errors, such as a bad
SQL statement, still scorekilled. So doesConnectionNotDefined, which
Rails 8 raises for an unknown role or shard. Rails 7.2 raises a plain
ConnectionNotEstablishedthere, so such a mutant scoreserroron 7.2.
With--matrix, a database-failed test is never a killer and leaves its
row incomplete, even if an earlier assertion already killed the mutant. - An old temp file of the other mode no longer stops the boot (#287).
Before the app boots, each mode sweeps the source directories. In 1.7.0
an in-process run removed only oldmutineer_mutant*.rbfiles and the
daemon removed only oldmutineer_daemon*.rbfiles, so a file the other
mode left made Zeitwerk raise. Now both boot sweeps remove an orphan of
either mode, and the daemon tells Zeitwerk to skip both names. You no
longer delete such a file by hand. A file of your own whose name only
starts withmutineer_mutantormutineer_daemon, such as
mutineer_mutant_helper.rb, is no longer removed: a sweep now matches
only the temp file shape. An unlocked old file is removed even when its
creator's pid has been reused; a held file lock still protects it.
Do not run Mutineer older than 1.7.0
at the same time in the same checkout: it still writes temp files
without the dot, and a newer run's mutants error when Zeitwerk finds
one. - A second in-process run no longer deletes a live mutant file (#295).
Each in-process run sweeps the source directories before its boot and
after its run. That sweep treated every.mutineer_mutant*.rbfile a...
v1.7.0
Added
- Call-to-receiver operator (Tier-2, opt-in via
--operators):
call_to_receiverreplaces the last call of a chain with its receiver, with
the arguments and block of that call (user.full_name->user). The mutant
survives when no test checks what that last call does. It does not overlap
withchain_link:chain_linkdrops a call in the middle of a chain, and
call_to_receiverdrops only the last call. The two give the same source
only when the last call repeats the call before it (node.name.name). It
skips the same names aschain_link, and it skips operator calls and
setters. - Reasons on ignored mutants. Text after
--on
# mutineer:disable-line, and a.mutineer.ymlignore:mapping
(idplusreason), is stored on the ignored mutant. The JSON report
(schema_version1.8) and the HTML report show it. A mapping with no
id, or with an unknown key, warns. Mutineer 1.7 is the first version
that reads a mapping. An older version matches no mutant to a mapping
and does not warn, so the mutant it names is not ignored. Upgrade
every gate that reads the file before you add a mapping.
mutineer migraterewrites the old id inside a mapping and keeps its
reason. mutineer triage. It printsignore:mapping entries for survivors
in a JSON report.--reasonis required. Pass--allor--id. It
does not edit files.- Equivalent-skip page. The docs list every built-in chain-link skip
and the endless-range skip, and why each one is equivalent. - Postgres worker databases for
--rails --daemon. Each worker gets
<base>-mutineer-<N>, copied from the test database before mutants run.
A second run cannot drop a database another live run holds. An app with
more than one database config runs one worker and says so once.
The worker databases stay after the run. - MySQL 8 worker databases for Minitest on
--rails --daemon. Each worker gets
<base>-mutineer-<N>onmysql2ortrilogy, with the base shortened to fit
MySQL's 64-byte name limit. Mutineer creates the
database, loadsschema.rborstructure.sql, and copies rows once at
the start. Triggers from the schema do not fire on the copied rows.
Mutineer creates them again after the copy.
A second run cannot drop a database another live run holds.
That work uses the TLS and timeout settings from the database config.
MariaDB is not tested. MySQL 5.7 is not supported. The worker databases
stay after the run. mutineer migrate. It rewrites old-format ids in.mutineer.yml
ignore:to the current ids. It keeps comments and every other line.
One old id that matches several mutants becomes every new id. An id that
matches nothing stays, and the command exits 1. A block scalar under
ignore:is not supported: the command exits 2 and does not change the
file.--dry-runprints each change and writes nothing. It does not
rewrite a baseline. Generate a new baseline with--format json. An
old-id warning namesmutineer migrateand says that this becomes an
error in Mutineer 2.0. The baseline warning says to generate a new
baseline with--format json.mutineer init. It writes a commented.mutineer.ymlwith the default
operators.sincestays commented, so the first run is a full scan.
init --railsalso setsrails: trueand prints the first run command.
An existing file is kept unless you pass--force.- Rails rollout guide. The page walks from one model, to a report-only CI
job, to a--sinceand--baselinepull request gate, to--threshold
on chosen folders. It has a GitHub Actions recipe and a GitLab CI recipe. - A mutation-testing primer in the README. "What is mutation testing?"
walks through one small example, from a surviving mutant to the test that
kills it, and says what each report row means and what to do with a
survivor. The usage block lists all four commands. - A production-ready section and a stability contract. The README opens
with a support matrix for Ruby, database, framework, and parallelism.
STABILITY.mdlists what a release keeps and what 2.0 will change.
SECURITY.md,CONTRIBUTING.md, and GitHub issue forms are new.
The docs home page takes the same section from the README at site build. --allow-emptyandallow_empty:. A run with no mutants now warns
that Mutineer 2.0 will fail it, unless--allow-emptymarks it as expected.
A--sincerun whose changes hold no mutants does not warn: it says the
changes hold nothing to test. A--sincerun with no mutants even before
scoping still warns. The exit code does not change in 1.x.--no-daemon. It beats a.mutineer.ymldaemon: truefor one run,
like--no-fail-fast. A Minitest--railsrun without adaemonvalue now
prints a notice that 2.0 uses--daemonby default for--rails;
--no-daemon(ordaemon: false) keeps the run serial and stops the
notice.- "Did you mean" hints. Unknown names in
--operators, in
.mutineer.ymloperators:and keys, and in# mutineer:disable-line
markers get the closest valid name. The.mutineer.ymlwarnings also say
that 2.0 makes them errors. --coverage-from FILElets parallel shards share one coverage capture.
A--rails/--bootrun reads thecoverage.jsonof an earlier run with
the same tests over these sources or more. It skips the capture. Each test
that covers one of its sources still runs alone, as on a cache hit, because
a mutant runs only those tests. A test that covers none of them does not,
so a shard runs far fewer tests alone than a full suite. All tests then run
once together, and the run saves its own cache.
Boot-mode caches now also record each source's fingerprint, the ownership
list and a digest of the other inputs. A capture that does not fit is an
error (exit 2), never a silent recapture. So is one where loading its other
sources ran these sources' code, which happens under lazy loading with
--strategy redefine. Without boot mode, and with--daemon,
--test-commandor--dry-run, the flag warns that it has no effect.
Changed
- Rails test workers stay off during a mutant run. Mutineer sets
PARALLEL_WORKERSto1for in-process,--daemon, and--test-command
runs. A value you already set is replaced, and Mutineer prints one line.
The coverage cache stores thePARALLEL_WORKERSvalue, so a cache from an
older version rebuilds once.--test-commandalso setsDISABLE_SPRING=1.
The README explainsparallelize, Bootsnap, and Redis after a fork. --test-commandexit codes 126 and 127 are scorederror, not
killed. A wrapper that could not start the suite ran no test. Under
--threshold, such a run can now score lower or fail the no-verdict limit,
where before it passed on false kills.- A run with errored mutants and no
--thresholdsays so. Errors are not
in the score, so they do not fail the run; the message says what
--thresholdwould enforce. --sincealso narrows suppressed mutants. A scoped report lists only
ignoredmutants on changed lines, as--dry-run --sincealready did.- The empty-run message moved from the human report to one stderr warning
for every format. "No mutations generated" is gone. An empty--since
run prints "nothing to test in the changes since REF" instead.
Fixed
- Daemon errors no longer show the database password. Rails parses
DATABASE_URLitself, and a malformed URL makes it raise an error that
quotes the whole URL. The daemon sent that error to the terminal when
the app failed to boot, and the same could happen when it set up worker
databases or the coverage map, and in a--verbosecoverage warning.
Mutineer now replaces the password in any URL, and theDATABASE_URL
password anywhere else (escaped or cut short included), with
[REDACTED]before it reports the error. --strategy reloadruns concern files with anincludedorprepended
block (#256). Every mutant in such a file waserrored: ActiveSupport
raisedMultipleIncludedBlocks(orMultiplePrependBlocks) because the
mutant loads from a temp file with another path. Before each reload mutant
loads, in-process and on the daemon, the forked child now forgets the blocks
that the source file stored, so methods in the concern body and in
class_methodsare killed or survive as usual. A block the mutant's load
does not store again (a concern under anunless defined?guard) is put
back. A method inside an
includedorprependedblock isunplaceableunder reload too: the block
does not run again in a class that already includes or prepends the
concern, so the mutant would falsely survive. That includes a method of a
class defined inside the block. Use--test-commandto run those mutants.- No heredoc opener without its body. A mutation that kept a heredoc
opener but replaced the lines of its body gave source that does not parse,
such asoperand_removalonfoo(<<~EOS) ||with the body andbaron
the next lines. The mutant showed only asskipped (invalid). Now the
operator does not emit it. - An RSpec run that fails only on an error outside examples scores the
mutanterrored, notkilled. A spec file that raised while loading, or
a failingbefore(:suite)hook, made RSpec exit 1, and every mutant was
scoredkilled. On a Rails app,rails_helpercalledSimpleCov.start
after--railshad suspended Coverage (#228), so each spec file raised as
it loaded, and 18 of 18 mutants were scoredkilled; 3 of them survive.
Now the runner returns 2 when RSpec recorded an error outside examples and
no example failed, so the mutant iserrored: out of the score and in
no_verdict, so that run now fails--thresholdinstead of passing at
100%. A failed example still kil...
v1.6.0
Added
- An
unplaceablemutant status. Under--strategy redefine, a mutant in
a method whose class or module has no constant name is not run. It is left
out of the score and, unlikeuncapturable, out ofno_verdict, so it does
not fail--threshold. The human report gives it anUnplaceable:row, the
HTML report a count, and the JSON reportsummary.unplaceableand an
unplaceable[]list;schema_versionis1.7. - The agent skill installs with
gh skillandnpx skills(#125). It
moved fromdocs/skill.mdtoskills/mutineer/SKILL.md, the layout of the
Agent Skills specification, sogh skill install davidteren/mutineer mutineerfinds it andnpx skills add davidteren/mutineer --skill mutineer
installs oneSKILL.mdinstead of the wholedocs/site. The site build
copies that file toskill.md, so its URL still works. The skill's
description now says when to use it, and the README shows how to install it. - A
ran_at_loadmutant status (#187). A mutant on a method-body line
that ran while the app booted or its class loaded gets this status when it
survives, or when only the load ran the line. A kill stays a kill. It is
left out of the score and out ofno_verdict, so it does not fail
--threshold. The human report gives it aRan at load:row that points
to--test-commandto verify these mutants, the HTML report a count, and
the JSON reportsummary.ran_at_loadand aran_at_load[]list, within
schema1.7.
Changed
- Mutants no longer run under Coverage with
--bootor--rails(#228). Coverage
starts before the app boots so the coverage map can be built, and it used to
keep running in every mutant fork after that, though nothing read it. It is
now suspended once the map is built, from a fresh capture or from the cache.
Coverage that a host process started before the run is left running.
On a small app whose test calls tiny methods in a loop, the CPU time of a
whole in-process--bootrun fell from 0.76s to 0.57s (median of five). On
the Rails fixture app the run is mostly boot, and the change is within noise
(1.61s before, 1.60s after).--daemonruns its mutants in worker daemons
that never start Coverage, so its time does not change (3.48s before, 3.44s
after); the map-building daemon now suspends Coverage after the map too. - A mutant runs its paired test files first, then the cheapest (#203). The
covering test files used to run in the order the coverage map stored them,
so a mutant that one fast file kills could still scoretimeoutwhen slow
files ran first. Now the files that pairing finds for the source run first,
then the rest from fastest to slowest by the time each file took during
coverage capture, less the fastest file's time (the startup cost each
capture pays). The order is fixed for a given cache. Costs compare in
doubling buckets, so files of close cost keep their path order when a
rebuilt cache measures them again; a file near a bucket edge can still move,
so keep.mutineer/between CI runs for the most stable--baseline. The
order applies to Minitest (after the seeded class shuffle, with
parallelize_me!classes still last) and to RSpec.--daemonruns every
covering test with no stop, so the order does not change its verdicts. The
coverage cache now saves these timings, so a cache from an earlier version
rebuilds once. A complete run gives the same verdicts and--matrixrows,
but the new order can change which mutants time out, so a--baselinegate
can see a one-time change on upgrade. - Job selection moved from
Mutineer::Runnerto a newMutineer::JobPlan
module (#75). The in-process runner, the--daemonbackend and
--dry-runshare it, so each file now requires what it uses and no
require_relativecycle exists inlib/mutineer. The moved methods are
collect_jobs,filter_since,coverage_selection,ran_at_load?,
load_verdict,abort_if_unclean!,test_load_roots,source_dirs,
sweep_orphans,suppress_map,suppressed?andresult_keys. These were
internal methods, soRunnerkeeps no copies. Runs, reports and caches do
not change.
Fixed
-
An endless method that only the tests call is no longer
no_coverage
under--rails,--bootor--daemon(#209). Ruby counts no line when
an endless method such asdef half(x) = x / 2runs, only itsdefline
when it is defined. The boot defines it before any test runs, so the forked
coverage capture never saw that line run. Coverage now also
counts methods (Coverage.start(lines: true, methods: true)), and a test
that calls such a method covers itsdefline. Coverage caches from earlier
versions rebuild once. -
module_function :namein a reopened module insideclass << self
promotes the method from the earlier opening (#216). Ruby treats both
openings as one module, but Mutineer matched them only within one body, so
it named the method#<Class:App>::M#cinstead of#<Class:App>::M.c.
The two openings now match by the module's full name. A module whose name
Mutineer cannot be sure of still matches only within its own body: a path
such asself::XorFoo::Xwritten directly insideclass << self, a
name under an anonymous class, or a module in aclass << selfinside a
method or aclass_evalblock. Amodule self::Xin a block, or a
self::X = ...in aclass_evalblock, matches only openings in the same
block. Aself::Xlike these now also stays apart from a
module of the same name outsideclass << self, where amodule_function
in that module used to promote its methods. -
A
module self::Xorclass self::Xin a block isunplaceableunder
--strategy redefine(#229). In a block,selfis not always the
enclosing class: inOther.class_eval doit isOther, and in
Foo = Class.new doit is the new class. Mutineer gave these modules a
known owner on the enclosing namespace, so redefine loaded the mutant onto
a module the tests never use, and the mutant could falsely survive or
scoreerror. Such a module, everything nested in it, and a
self::X = ...in a block that does not build a class now have
their owner unknown: redefine reports their mutants asunplaceable, and
--strategy reloadstill runs them. In aData.define,Struct.new,
Class.neworModule.newblock, the module is now named under the
builder's constant (Outer::Foo::X#c, wasOuter::X#c), so the subject
names and mutant ids of those methods change.self::Xdirectly in a class
or module body is unchanged. -
--daemonworker databases start as a copy of the test database
(#222). Each worker slot's SQLite database used to start from
db/schema.rbalone. Rows the daemon wrote while it booted, in
initializers or--requirefiles, were only in the base test database, so
a test that read them passed in-process but failed under--daemon. On its
first use, a worker database is now a copy of the base test database (schema
and rows, made with the SQLite online backup API). The worker still loads
db/schema.rbwhen the copy's schema differs from it (another schema
version, or another or noschema_sha1checksum inar_internal_metadata),
for example when the test database is empty, out of date, or set up with a
plainload "db/schema.rb". That load drops the
copied rows of the tables it defines. -
Standalone coverage capture loads the
--requirefiles (#217). A run
without--bootrequires the sources and then each--require(or
require:) file before it forks the mutants, but the coverage capture
loaded only the sources. A source method that a--requirefile called
while it loaded was then not seen as run at load, so its mutant was a false
survivedinstead ofran_at_load. The capture and the clean-suite check
now load the--requirefiles after the sources. With or without
--boot, the coverage cache rebuilds when one of them changes, is added,
is removed, or moves in the load order. -
--daemonloads the--requirefiles (#220). The daemon booted the app
but never loaded the--require(orrequire:) files, so a test that
needs one failed under--daemonand passed in-process. A source method
that such a file called while it loaded was also not seen as run at load,
so its mutant could be a falsesurvivedinstead ofran_at_load. The
daemon now loads the files after the app boots, as an in-process run does,
and the daemon's coverage cache rebuilds when one of them changes. -
The release workflow publishes only stable tags (#161). Its trigger,
v*.*.*, also matches a prerelease tag such asv1.3.1-rc1, and the only
guard checked that the tag matchedMutineer::VERSION. A prerelease tag
could then publish the gem and move the floating major tag (v1) that
Action users pin. The workflow now stops with a clear error before any
publish step unless the tag is a stablevMAJOR.MINOR.PATCH. -
Coverage capture keeps to
capture_timeoutfrom start to finish. Under
--bootand--daemon, a hung test file ignored the timeout and blocked
the run; it is now skipped as "timed out", like a standalone capture. A
process that a test started and left running could keep the result pipe
open and make a boot-mode capture wait for it to exit. Each capture now runs
in its own process group, and a timeout stops the whole group, not only the
test process. A capture whose test finished just before the deadline could
also lose its valid result and report "invalid coverage output"; it now
gets a short grace period to read the result. (#101, #129) -
--daemonno longer waits forever on a daemon that stops answering
(#101). A mutant whose verdict has not arrived 30 seconds after its own
timeout is scorederror, and the daemon is killed an...
v1.5.0
Added
-
--timeout SECONDSand--capture-timeout SECONDS, alsotimeout:and
capture_timeout:in.mutineer.yml. They set the in-process per-mutant
time limit (default 10s) and the coverage-capture time limit (default 120s),
which were fixed. On a large Rails suite the clean run of the unmutated
tests took longer than 120s, so every run that offered the whole suite
stopped as not green, and a mutant on a line many tests cover took longer
than 10s and was scoredtimeout.--daemonkeeps its own per-mutant
limit but uses--capture-timeoutfor its coverage capture;
--test-commanduses neither. Either backend warns about a limit it was
given and does not use. -
--cache-dir DIR, alsocache_dir:in.mutineer.yml, sets where the
coverage cache is written (default.mutineer, as before). Two runs from the
same project root shared.mutineer/coverage.jsonand could overwrite each
other's map; each can now have its own directory.--daemonuses it too.
A blank value (an unset CI variable) exits 2 rather than writing the cache
into the project root. -
The README states how timeouts affect the score (#62): a timed-out
mutant is left out of the score, and counts as a mutant with no verdict
under--threshold. -
--matrixreports which tests kill each mutant (matrix: truein
.mutineer.yml). Each mutant runs every test in its covering files instead
of stopping at the first failure, and the child sends each test's outcome to
the parent as it is recorded. The report names blind tests, which ran in a
complete row and killed no mutant, and redundant tests, whose every kill
another test also makes. Redundancy is judged one test at a time, so delete
redundant tests one at a time. The human report lists up to 20 of each, the
HTML report all of them, and the JSON report moves to schema1.5with a
matrixblock that appears only with the flag. A test is its file and its
id (the example id under RSpec), so examples that share a description stay
apart and an example whose generated description changes with the mutant
stays one test. Verdicts, the score and the exit code do not change: each
mutant gets the verdict a run without the flag gives. After a serial test
fails, an exit, a crash or the timeout in a later test, class or group
leaves itkilled; an end in the code that run still reaches (the failing
test's class wrapper or group hooks, the suite's own cleanup) and a failure
in aparallelize_me!class keep the exit status. A row is complete only
when the child's stream arrived in order and every test reported (under
Minitest the recorder saw every test, under RSpec every planned example),
so a caughtInterruptleaves it incomplete. The human report names up
to 20 incomplete rows and the HTML and JSON reports all of them; the human
and HTML reports warn not to delete a blind test until its rows are
complete.
Minitest and RSpec 3.3+ are supported, on the in-process backend only;
--daemon,--test-command,--fail-fastand--dry-runexit 2 with it.
--no-matrixand--no-fail-fastbeat the matching.mutineer.ymlkey,
so a file that sets one can still run with a flag that conflicts with it. -
no_coverage[]anduncapturable[]name each mutant. Every entry in
the JSON report now hasoperator,tokenandid, asignored[]has, so
mutants on one line can be told apart and anidcan go inignore:. The
entries are sorted by(file, line, operator, id).schema_versionis1.6. -
The site and README explain "clean-room" and compare Mutineer with
Mutant, and the landing page hero has a one-minute explainer video. It
plays at 55% volume while in view (not with reduced motion; muted until
the first click or key where the browser requires it), pauses when it scrolls
away or the tab is hidden, and has an Enlarge button.
Changed
- The human report has a
Timeout:row, and the score line lists timeouts
apart from errored mutants. Before, theErrored:row and the score line
added the two together, while the JSON report kept them apart. The HTML
summary now shows atimeoutcount apart fromerroredtoo. - A test file given as a source after
--testexits 2.--testtakes one
file, so inmutineer run app/x.rb --test spec/a_spec.rb spec/b_spec.rbthe
second spec became a source to mutate, and the run tested with one spec file
without a word. Now a source named*_test.rbor*_spec.rbinside a
test/orspec/directory, given with--test, is a usage error that says
to repeat--test. The usage line now reads--test <test> [--test <test>...].
Fixed
- The RSpec runner keeps what gems configured before the run. It called
RSpec.reset, which drops the whole configuration. Underrails: true, a
--bootfile orrequire:, the app's gems load first, so a setting such as
rspec-retry'sverbose_retrywas gone, and a support file that set it
raisedNoMethodError. The run stopped with "the unmutated suite is not
green". The runner now callsRSpec.clear_examples, which keeps the
configuration. Hooks that gems added now apply too: an example with
rspec-retry'sretry: 3runs up to 3 times per mutant, as underrspec. --sinceno longer scores a file that Git ignores (#168). A file
outside the index is scored in full, so a new file is not missed. That
check also caught files that Git ignores on purpose (.gitignore,
.git/info/excludeor the global excludes file), such as generated code
inside a directory run. Those files now get no mutants under--since,
and a warning names each one. Other untracked files are still scored in
full, and so is a file when Git cannot tell, such as one beyond a symlink or
inside a submodule.- The human report escapes control characters from the source (#163).
A surviving line that held a terminal control byte (for example ESC) was
printed as that byte, so it could change what the terminal showed. The
token, the replacement, the diff lines, method names, file paths and the
kill matrix's test names now print each control character except tab as
its Ruby escape (\e), and a byte that is not valid UTF-8 (a Latin-1
source) as\xNN. The JSON and HTML
reports already escaped text and do not change. - One edit on a line is scored once (#159). Some opt-in operators emitted
two mutants that give the same source:literal_mutationon0(the
"change to 1" and "add 1" rules),negation_removalon!!x(either!
removed), andchain_linkoroperand_removalon a repeated part such as
a.b.b.corx || x. Each copy had its own id, so one surviving edit
counted twice. Now only the first copy on a line runs. Copies that start
on different lines (a multi-line chain) both stay, so--sincenever
loses the edit. Ids are assigned before the copy is dropped, so every kept
mutant keeps its id, and the dropped copy's id simply stops appearing.
Default runs do not change. If you use these operators with--baseline,
regenerate the baseline after upgrading: the score can move with no code
change, and in a full (unscoped) comparison a dropped copy that survived
is listed as fixed. An ignore entry for one copy's id still ignores only
that copy; the other copy still runs. # mutineer:disable-lineinside a string no longer silences a line
(#158). The marker was found with a text search, so a string such as
"# mutineer:disable-line"ignored every mutant on its line and left
them out of the score. Only a real#comment counts now. A heredoc or
regex that holds the text is ignored too../lib/x.rband an absolute path pair with a test (#104). Auto-pairing
found the test forlib/calc.rbbut not for./lib/calc.rbor the
absolute path to the same file. It reported "no test found by convention"
and exited 2. A file argument inside the project is now made relative to
the project root first, so equivalent spellings (./,.., absolute)
pair with the same test and run once. A symlink keeps its own name, so it
stays a separate source from its target. Reports show that root-relative
path. A path outside the project stays as typed.- A mutant on another line of a multi-line statement runs its tests
instead of being reported asno_coverage. Ruby counts one line of a
statement only. When the mutant's own line has no count, the runner now uses
the tests that ran the statement that holds the mutation. A heredoc body
belongs to its statement. This fallback skips a first or last line that also
holds other code, such as thedefline, and the body ofx while cand
x until c. It also gives no tests to a mutant in code of the statement
that runs only sometimes: awhenorincondition, arescueclass
list, a parameter default, the rescue side ofx rescue y, the value of
||=or&&=, the operand ofdefined?, the right side ofa ||or
a &&, a branch ofx if c,x unless c, a ternary orelse, the
pattern ofv in porv => p, an
argument or block ofx&.m(...), and the value ofx&.m += v. Those
mutants stayno_coverage, as before. - A survivor diff for a multi-line mutant applies with
git apply(#106).
The JSONdiffheader always read@@ -N +N @@, even when the mutant
removed two lines and added one, sogit applyrejected the patch as
corrupt. The header now carries each side's line count (@@ -3,2 +3 @@).
A one-line diff keeps its old header. A mutant that empties a line now
shows that empty line as a+line. A side whose last line has no final
newline gets\ No newline at end of file. Lines of a CRLF file now keep
their\r, so the patch applies to that file. - Split test files pair with their source (#87). A source such as
app/foo/bar.rbnow also uses `test/foo/bar_upsert_test....
v1.4.0
Added
- Condition-forcing operators (Tier-2, opt-in via
--operators):
condition_trueandcondition_falsereplace anif/elsif/unless,
ternary, modifier orcase/inguard condition with(true)or(false), so
its branch always runs or never runs. A surviving mutant means no selected
test detected the forced condition. A literal condition, also in
parentheses, makes no mutant: forcing it changes nothing or repeats the
boolean_literalflip. A condition that holds a heredoc makes no mutant.
A condition that assigns a variable keeps its code, and only its value
is forced ((m = x; true)), so later code still sees the variable.
The never-runs side of an else-less conditional can
be the same program as thenilthatstatement_removalorreturn_nil
puts in its place; it is still made, so these mutants and their ids do not
depend on which other operators run. - Operator-assignment operator (Tier-2, opt-in via
--operators):
operator_assignmentswaps the operator of a compound assignment:
+=<->-=,*=<->/=, and%=and**=->*=. It covers local,
instance, class and global variables, constants, calls (a.b += 1) and
index calls (a[i] += 1). Thearithmeticoperator never sees these
forms, because Prism does not parse them as calls. The operator does not
change||=,&&=, or the bitwise and shift forms (|=,<<=).
Fixed
- A typed
--rails,--verboseor--debugbeatsrails: falseand
verbose: falsein.mutineer.yml(#103). The flag was dropped, so the
run went on without Rails boot or verbose output. The config layers now keep
only the keys the user wrote, and "did the user write this" comes from those
keys, not from a hand-kept list. framework: rspecin.mutineer.ymlsurvives test pairing (#103).
Pairing re-detected the framework from the test file names, so an RSpec
suite intest/calc_test.rbran under Minitest.- A bad number is an error, not a rounded value (#105).
--jobs 1.9and
--baseline-epsilon abcno longer become1and0.0. One option schema
parses each value once, where it enters, for both the command line and
.mutineer.yml. --sincekeeps a source file that git does not track yet (#156). Before,
such a file counted as unchanged. The run scored no mutants, and a positive
--thresholdstill exited 0. The file is now new in full, so all its
mutants run.
Changed
- These now exit 2 with a message that names the option:
jobs: 1.9or
jobs: truein.mutineer.yml(before:1, or a crash); a bad, negative
or non-finite--baseline-epsilon(before:0.0); a boolean key in
.mutineer.ymlthat is nottrueorfalse, such as the string"yes"
(before:false). --jobstakes plain digits only.0x2,+2," 2"and1_0exit 2.
Before,Integer()read them as 2, 2, 2 and 10.--thresholdand
--baseline-epsilontake a plain decimal, such as2or2.5, from a
string.0x10,+2,1_0and1e2exit 2. Before,Float()read them
as 16.0, 2.0, 10.0 and 100.0.- A string option that gets
true,falseor no value exits 2, such as
only: falseorbaseline:in.mutineer.yml. A blanksincealso exits 2,
in the file and on the command line, so an unset shell variable does not
turn scoping off.since: falsestill means no scoping. - Config file errors start with the file and key, for example
.mutineer.yml: threshold must be a number between 0 and 100, where they
started with--threshold. - The docs site is built in CI — a Pages workflow runs
rake site:build
and deploys the result, so the YARD HTML under/api/,llms-full.txt,
json-schema.htmlandsitemap.xmlare no longer committed. CI checks
that the site builds, in place ofrake yard:pages:check. The site keeps
its URLs (#153).
v1.3.0
Changed
- Mutant ids now include the project-relative file path (#126). Before, two
mutants with the same method name, operator and token in different files got
the same id. That happened with a top-leveldefor block in two files, and
with a class reopened in another file. Oneignore:entry then suppressed
both mutants, and--baselinecould hide a new survivor behind an old one.- Every survivor id changes once in this release. This affects any external
tool that tracks survivors by id. - Moving or renaming a file now changes its ids.
- Ids follow the project root: the directory mutineer runs from, or the
Action'sworking-directory. A run from a different root gives different ids.
mutineer finds.mutineer.ymlby walking up, so a run from a subdirectory
prints one[mutineer]warning that the loaded ignore ids will not match. - A source outside the project root uses its absolute path, so its ids differ
between machines. - Two methods with the same qualified name in one file (for example two
top-leveldef indexin two DSL blocks) now get different ids. The second
and later ones hash their position among those methods; the first keeps its
id.
- Every survivor id changes once in this release. This affects any external
- The JSON report marks its id format (schema
1.4, additive).
summary.id_formatis2for ids that include the file path.
summary.legacy_id_matchescounts old-formatignore:entries (ignore) and
survivors matched only through an old-format baseline id (baseline). The
GitHub Action shows one warning annotation when either count is not zero.
Deprecated
- Old-format ids in
ignore:and in baselines. Matching on them is removed
in 2.0. Replace each oldignore:entry with the new ids from the warning
(only the intended ones when it over-matched several mutants).
Regenerate a baseline (--format json) only after every gate that reads it
runs this version or later (the Action'sversion:pin, your CI
Gemfile.lock). An older version treats every new-format survivor as new.
Fixed
module_function :nameandmodule_function def namepromote only
their own module's methods — a class or module in the same file with a
method of the same name kept an instance method in Ruby, but mutineer
named it as a class method.--strategy redefinethen mutated a method the
tests never call, so a killable mutant falsely survived, and
--only Class#nameselected nothing (#98). The affected methods now get
their correct names, so their mutant ids change: regenerate any ignore
entries or baseline survivors that pointed at them.- A root-anchored reopening names the top-level constant — a
module ::Rootorclass ::Solowritten inside another module now gives
RootandSolo, notOuter::RootandOuter::Solo. This applies to every
method in such a body, with or withoutmodule_function, so their mutant
ids change too.--strategy redefinerebuilds such a body's scope as
written (module Outerthenclass ::Solo), so a constant fromOuter
still resolves in the mutated method, as it does underreload. Before,
the method raised NameError in the test, which counted as a false kill
(#145). - Old-format ids keep working, with a warning (#126). An old-format
ignore:entry still suppresses the mutants it matched before. The run prints
one[mutineer]warning per entry with each new id, its file and its method.
When the entry matched several mutants (in different files, or same-named
methods in one file), the warning says it
over-matched and to keep only the ids for the mutant you meant to ignore. A
--baselinefile withoutsummary.id_formatmatches on new ids, or on old
ids from the same file, so no survivor reads as new or fixed only because its
id changed. A stored file outside the project root (a baseline written on
another machine) matches on the old id alone. The run prints one
[mutineer]warning to regenerate the baseline.
v1.2.0
Added
- Operand-removal operator (Tier-2, opt-in via
--operators):
operand_removalreplacesa && bwith(a)and with(b), and does the
same for||,andandor. The mutant survives when no test needs the
operand that the mutant removes. The operator never keeps a jump operand
(return,break,next,redo,retry) alone, because a jump does not
parse in a value context. It never removes an operand that holds a heredoc,
because the heredoc body stays behind as code. It skips nested method
definitions, because mutineer mutates each one as its own method. - Array-literal operator (Tier-2, opt-in via
--operators):
array_literalreplaces a non-empty array literal, such as[a, b]or
%i[a b], with[]. The mutant survives when no test checks the contents
of the array. The operator skips an implicit array (x = 1, 2), an array
that holds a heredoc, and nested method definitions. - Sources also pair with Minitest's
test/**/test_*.rbfiles — after the
_test.rbforms, so existing projects pair as before.lib/helper.rbdoes
not pair with thetest/test_helper.rbsupport file. A failed capture of a
test_<name>.rbfile now marks<name>.rbuncapturable, as<name>_test.rb
does (#120). Withoutframework:set, a source withspec/<name>_spec.rb
andtest/test_<name>.rbbut no<name>_test.rbnow pairs with the
Minitest file, as the "Minitest first" order says.
Fixed
reloadloads the mutant by an absolute path — a relative source path
gave the mutant relative backtrace paths, so code that checks its own frames
by absolute path failed for every mutant, a false kill (#123).require "test_helper"works withoutRUBYOPT— a standalone run puts
lib, then each test file'stest_helper.rbdirectory, on the load path,
as boot mode andrake testdo. A run where no test records coverage because
captures failed now exits 1 instead of reporting N/A (#119).- A disable-line marker warns about an operator it does not know — a
reason written without--became part of the operator name, so the marker
suppressed nothing and said nothing (#124). A marker followed only by spaces
or commas, such asdisable-line -- why, now disables the whole line. - Coverage capture and the clean check run each source once — they read
sources withload, so a test's ownrequireran them again: aStruct
superclass raisedsuperclass mismatch, and load-time code ran twice (#122).
A mutant of such a class still errors under--strategy reload, which loads
the mutated file again;--strategy redefineruns it.
Code that guards itself to run once (unless defined?(X)) can now show as
covered, so its mutants run where they wereno_coveragebefore. - A red unmutated suite now shows why it failed — in a standalone run, the
Minitest summary or RSpec output of the failing test, with its failure
message, goes to stderr before the "not green" error. A passing run prints
nothing extra. Boot mode (--rails,--boot) is unchanged (#121).