Repository navigation
Fixed
-
The clean baseline reads current local source, tests and helpers.
Timestamp bytecode caches could reuse stale code after a same-size edit
with an unchanged mtime, accepting a failing baseline and running mutants.
Leela now removes only the project's source and assertion-rewrite caches
before coverage and the unmutated baseline, leaving installed packages alone. -
Installed packages in a virtualenv inside the project are no longer
evicted between mutants. Leela evicted every module whose__file__
sat under the cwd (and, before a run, everything under the temp dir).
With uv's default.venvinside the project that included third-party
packages, so C extensions such as numpy failed to re-import with
ImportError: cannot load module more than once per process. That
crash was scored as a kill, so every mutant reported killed,
including mutants no test caught. Eviction now skips anything under
sys.prefix,sys.base_prefix,sys.exec_prefix,
site.getsitepackages(),site.getusersitepackages()or a
site-packages/dist-packagesdirectory, while still evicting the
project's own modules so mutated code reloads.Scores reported by earlier versions may be inflated for projects
whose virtualenv lives inside the project directory. Re-run to get an
honest number. -
Database tests under pytest-django can run inside leela's inner
sessions. pytest-django blocks DB access at configure time and only
restores it at unconfigure, which is after leela runs. Each inner
session'sDjangoDbBlockerrecorded that outer blocking wrapper as the
"real"ensure_connection, sodjango_db_blocker.unblock()could
never reach the database: every DB test errored in setup, and each
setup error was scored as a kill, including for mutants no test
could detect. Leela now lifts the outer block (via pytest-django's own
unblock()) while it runs. Scores from earlier versions on
pytest-django projects with DB tests may be inflated. -
Django model and admin modules are no longer reloaded between
mutants. Re-executing them re-registered models ("Reloading models is
not advised") and admin classes (AlreadyRegistered), and re-entered
import cycles that only resolve in Django's app-loading order. Any
project module they reference is kept too, so a kept model never ends
up subclassing a stale copy of a reloaded mixin. -
The summary no longer reports a negative "pruned by type analysis"
count. A typed rule can add operators the untyped rule lacks (int
+gains//), andcount_prunedsubtracted list lengths, so such
points counted as negative pruning ("-1 pruned"). It now counts only
untyped mutations the typed rule dropped. -
Under
--diff, the summary's candidate, pruned and type-enrichment
counts cover only the changed lines. They counted every mutation
point in the target files while only changed-line mutants ran ("159
candidates" for 88 tested). The diff filter now applies before type
enrichment, counting and generation.
Added
-
Clean baseline before any mutant. The selected tests run once with
no mutation applied, through the same in-process path as every mutant.
If anything fails or errors, the run aborts with a message naming the
failing tests and a non-zero exit, instead of scoring every mutant as
killed. -
A third mutant status,
error. A mutant whose inner run crashed,
exited abnormally (usage error, nothing collected, interrupted) with no
test failure or collection failure, or ran zero tests, was previously
reported as killed (crash) or SURVIVED (tests_run: 0). It is now
an error: reported with its reason in the terminal, HTML and JSON
reports, excluded from the mutation score, and failing the session by
default.MutantResultgainserrorand astatusof"killed",
"survived"or"error";RunResultgainserrorsand
mutants_scored. Timeouts remain kills. -
A test module that fails to collect under the mutant is a kill.
The clean baseline shows it collects unmutated, so the failure is the
mutant's doing: the target raising at import, or a test module's own
module-level check failing. Those modules are the killing tests.
MutatingLoaderrecords exceptions raised by the mutated source, and a
conftest that fails to import because of one is a kill too. A
conftest's own failing check, or a conftest or test module that calls
pytest.exit()at import, is anerror: it is not attributed to the
mutant. So is a conftest collection hook that callspytest.exit();
the error names the node being collected. Error text renders each
collection failure aspath: Type: message. A run that
stays green or empty is not a kill: an import caught by the test is
SURVIVED, and a module-level skip, a skip marker or a conftest that
skips is anerror, because no test ran. A conftest raisingSkipped
at import no longer crashes the whole leela run:Skippedis a
BaseException, outside what inner runs caught.
0.8.0's verdict for a mutant that broke the target's import depended
on the layout. With the virtualenv and the temp dir outside the
project, it was SURVIVED withtests_run: 0. With the virtualenv
inside the project, re-importing installed packages could crash the
inner run, and the mutant was KILLED as<crashed>, a verdict
indistinguishable from leela failing to test it. -
--leela-benchmarkno longer raisesTypeError. It called the
target-discovery helpers without thepython_filespatterns they
require. -
The
djangoextra requirespytest-django>=4.10. 4.10 is the
first release whosepytest_unconfigurerestores the session's
database block, which leela's nested sessions depend on. -
fail_on_erroroption in[tool.pytest-leela](defaulttrue).
Set it tofalseto keep errored mutants from failing the session.