diffcone 0.4.0

Upgrading from 0.3.0: make your execution-evidence recordings again. The
store format is now 6 (0.3.0 wrote 1): --evidence auto skips an older
recording, naming one is an error, and the run action plans from the
code until record makes a new one. Expect plans to select more tests
than 0.3.0 did: most of the changes below close cases where diffcone
missed an affected test. The JSON report keeps schema version 3 and gains
fields: always_run and analysis.counts.always_run;
environment_checked and preloaded_modules in analysis.evidence;
rootdir, configfile, junit_prefix and run_arguments in pytest's
discovery[].config; and evidence_not_used in the plan run -o writes.
check --format json gains mapping.rootdir and mapping.junit_prefix.
New discovery notes: unparsable_config (discovery incomplete),
unreadable_plugins and plugin_outside_roots.

Planning

• diffcone.toml takes [[always_run]] entries: the tests (or
  benchmarks) whose id matches a pattern are selected in every plan, with
  or without a recording, and the report gives the entry as the reason and
  counts the targets each entry matched; the text report flags an entry
  that matches nothing. A malformed entry, or one naming an unknown
  runner, is an analysis error, so the plan selects everything rather than
  quietly leaving those tests out. diffcone report computes a job's
  selected share without them.
• A test is selected when the fixtures, hooks or plugins around it change
  though its own code did not (rule lifecycle_changed), for example when
  another test module adds a plugin to pytest_plugins; with a recording,
  compared with what ran around it when it was recorded. pytest plugin
  modules are dependencies of every test, since their import-time code
  runs for the whole session.
• diffcone now assumes a test does not depend on what an earlier test left
  behind. A change to module state made by code that runs only inside one
  test counts for that test only, not for the other tests and benchmarks
  reading the same state: the test itself, a function-scoped fixture,
  unittest's setUp/tearDown, and a helper in a test module or
  conftest that only such code calls. A patch undone when the test ends
  counts the same way, also for lookups on a library module it patches:
  monkeypatch and mocker methods on pytest's fixtures, a with mock.patch(...) block, a @mock.patch(...) decorator,
  MonkeyPatch.context(), and a test or fixture that saves a module
  attribute and puts it back. Still counted for every reader: a test
  module's top-level code, wider-scoped fixtures, hooks, all project code
  (a cache a test fills through it), a doctest's function, a manifest
  entry on project code, a test or setUp that module-level code, a
  wider-scoped fixture or a hook calls, and a store through a
  monkeypatch that a wider-scoped fixture requests or the test module
  overrides (one overridden in a parent conftest is still taken as
  undone). If a test relies on an earlier test's state,
  declare the dependency in diffcone.toml.
• A fixture or function that only empties or fills a module-level dict,
  list or set (REG.clear()), or resets another module's variable
  (mod.X = None), does not count as reading it: an autouse reset fixture
  no longer selects every test when the code filling the registry
  changes.
• An attribute read on an object of unknown type is matched to
  module-level functions, classes and variables only in modules such an
  object can be: modules passed around or bound to another name, obtained
  by name or named in a string, and test modules and conftests, which
  pytest hands out. A name bound only to modules (m = ops, then
  m.value()) is followed like the import itself. Methods still match
  whatever the receiver.
• A text a test writes or runs (pytester.makepyfile, exec, -c code)
  whose template field the test's own parameter fills, when
  @pytest.mark.parametrize on the test gives that parameter literal
  values, is read once per value instead of counting as text built at run
  time, which any change reached. A value that is not a literal, an
  indirect parameter, another decorator on the test, or a parametrize
  on its class or in pytestmark keeps the old rule.
• Indexing a very long function is no longer quadratic in its calls.
• Editing the docstring of a method under @classmethod, @staticmethod
  or @property no longer counts as code run at import (which selected
  every test importing the module): these builtins only copy the
  docstring. Code that reads the docstring is still selected, and so is a
  project decorator with one of those names.

Fixed

Static analysis

• A change to a dependency, build or CI file outside the source roots
  selected nothing when the roots don't hold the repository root. These
  now select every target: lock files (uv.lock, poetry.lock, any
  *.lock, pylock.toml, conda-lock.yml), requirements* files (also
  dev-requirements.txt, Requirements.txt, requirements.pip),
  environment* files, a .txt/.yml/.yaml file in a directory named
  deps or dependencies, a workspace member's pyproject.toml,
  uv.toml, hatch.toml, poetry.toml, pdm.toml, pytest.toml,
  .env and .pth files, sitecustomize.py, noxfile.py,
  .github/workflows/, checker configurations that pytest plugins run as
  tests (mypy.ini, ruff.toml), compiled sources, anything under a
  *.dist-info, *.egg-info or *.egg directory, a build script
  anywhere under a root (a nested setup.py), and a lock-file change
  between INDEX and WORKTREE. Evidence mode treats them the same way.
• A name taken from a dict, list or set written in the code
  (getattr(handlers, NAMES[key]), a lazy-export table) is bounded by it
  only while every use of the table is a read: a table changed through an
  alias, a helper it is passed to, its module (core.TABLE[k] = v),
  globals(), vars(), sys.modules, monkeypatch or mock.patch (an
  object or a dotted string) is no longer read as its literal. A
  parameter or class-body name that shares its name with such a table is
  no longer bounded by it.
• A lookup on a standard-library or third-party module
  (getattr(logging, name)) counts writes that reach the module
  indirectly: an alias, a helper it is passed to, a loop, __setattr__,
  vars(), a module found at run time, a submodule handed on
  (register(os.path)). Handing a library object such as
  datetime.timezone.utc to a call does not make every lookup on that
  library depend on the caller.
• Modules whose name is known only at run time: once code can reach one,
  no literal table is bounded. A name that starts with fixed text
  (import_module(f"plugins.{name}")) stands for the modules with that
  prefix; a new types.ModuleType is no project module unless it is
  installed in sys.modules under one's name; exec(code, namespace)
  does not reach the calling module; m.__dict__["X"] = v changes only
  X. runpy.run_module(name), runpy.run_path(path) and python -m <name> with a name nothing bounds count as an import of any module, as
  importlib.import_module(name) does.
• An object put in sys.modules (assignment, setdefault, update,
  monkeypatch.setitem, mock.patch.dict) was ignored. It is now the
  module of that name: lookups on that module see it, and while it stays
  installed, the module's importers depend on the code that installed it.
  Without a recording, code that installs an object under a name computed
  at run time makes every test importing a project module depend on it.
• A change to a module-level variable made through a module object the
  analysis did not resolve to the variable's module is a dependency of
  the variable's readers: importlib.import_module("pkg.reg").TABLE["x"] = f, sys.modules[...], a function-local import, a star import, or a
  module passed to a helper that writes into it. A function's
  __globals__, a frame's f_globals, pickle.loads,
  pkgutil.resolve_name, pydoc.locate, gc.get_objects() and
  importlib.util.module_from_spec copies are recognised as ways to
  change a module's tables, and a lookup by a computed name on a module
  sees objects other code puts on that module. Star-imported names
  (from sys import *, then modules[...]) are seen too.
• State changed through an argument or a receiver (register(REGISTRY),
  registry.add(x)) did not reach its readers, in either mode; nor did a
  change to code that may call such a writer differently, or a write
  through a module attribute and an item (store._CACHE["k"].append(x),
  del store._CACHE[k]). A method called on an instance attribute
  (self._client.get()) counts as that class's method only when every
  assignment of the attribute creates an instance of one class (or of a
  third-party class), and a module-level object made by a third-party
  call is changed only by methods of classes deriving from a third-party
  class; calls on such attributes are still matched by name. A function
  a module-level variable's initialiser calls runs at import, so a change
  to it reaches the module's importers.
• Changing the arguments of a call made at import time (X = set_mode("slow"), a class attribute, a decorator's or default's
  argument) did not select the tests that read the state the callee
  changes. An import added to or removed from a module that starts or
  stops running another module's import-time code reaches the readers of
  what that code writes (a registry filled at import), in both modes;
  added imports stay non-structural otherwise.
• Moving a module's or a class's statements around (a constant moved
  below the decorator that reads it, two @register functions swapped, a
  def moved below a try/except import of the same name) was no
  change at all: the order of statements both sides have is now part of
  the definition, so a reorder reaches importers. Only swapping
  neighbouring plain defs (no decorator that does anything, no
  annotation evaluated at import) still selects nothing.
• A decorator diffcone can't show leaves the function alone (one from an
  installed package, or a project decorator that calls or registers it)
  makes the decorated function count as code that runs on import, and its
  parameters are no longer bounded by its call sites. A name imported in
  a try/except or if/else, rebound by a star import, or redefined
  in the module is no longer taken for typing.final, a quiet decorator
  or a structural base; attribute uses of a name bound several ways
  depend on every binding.
• A local name, parameter or function-local import was taken for a
  module-level import or builtin of the same name, and a call of an
  unresolved function named rebinds, registers or mutated_by for one
  of diffcone's own kinds of dependency.
• A dynamic import, script path or getattr receiver bounded by its call
  sites is unbounded when a caller may be unseen (a call through an object
  of unknown type, an unseen construction).
• Reading members by name or wholesale is a dependency: an attribute a
  class does not define (C.__doc__, C.__type_params__, one a decorator
  sets), C.__dict__[name] and obj.__dict__[name], mod.__dict__[name]
  and inspect.getmembers(mod) (a test parametrized over a module's
  functions), match class patterns, operator.attrgetter/
  methodcaller and str.format fields ('{0.attr}'); a non-literal
  name is a dynamic reference. C.__subclasses__(), C.__mro__ and
  C.__bases__ reach the classes they return.
• Code that passes a module to other code (read(ops, name)) depends on
  that module's functions and on what its imports bind (api.core.TABLE);
  a module from a literal import_module("pkg.sub") or __import__ that
  is handed on counts as handed on. An instance handed to code outside the
  project (json.dump(x, w), print(file=w), dict(obj),
  csv.writer(w)) depends on every method of its class, not only its
  dunder methods.
• A class obtained by a name nothing resolves (m.Alt() on an object of
  unknown type, a lazy __getattr__ export) reaches its __init__ and
  __new__. A class with an enum.Enum-family base or an in-scope
  metaclass depends on its __init__/__new__, which run when the class
  is created (Color.RED.code reaches Color.__init__); dir(mod)
  depends on mod.__dir__, and dir, hasattr and vars of a library
  module your code writes to are lookup sites.
• A docstring-only change selects the tests that read that docstring, in
  both modes, also through a parameter (inspect.getdoc(fn) in a
  helper), getattr(obj, "__doc__") and a module's __doc__.
• A change to PEP 695/696 type parameters (bounds, defaults, their number)
  was no change at all; a bound or default naming project code is now a
  dependency. A subscript in an evaluated annotation (X: Registry["k"] = 1) counts as code that runs on import.
• Code a test runs in a new process: a test running a script path, -m module or -c code named in the code did not depend on it. A module's
  file is an import of it; a .py file no module name maps to
  (scripts/gen-data.py) is read as -c code is (what it imports counts,
  and a change to the file reaches the test; if it imports by a name it
  computes, any change does); a Python command whose program is
  built at run time is affected by any change. A child whose interpreter
  is not the first word (env X=1 python -m ..., uv run python ...,
  timeout 30 python ..., asyncio.create_subprocess_exec(sys.executable, ...), os.execl) is followed. python -c code that changes
  sys.path, sys.meta_path or sys.modules by any assignment, deletion
  or augmented assignment, or calls os.chdir, can import anything.
• Text a test writes out and runs (exec, eval, compile, timeit,
  pytester.makepyfile) is read as the code it is: its imports and calls
  count. Text built at run time from pieces that are not written out can
  import anything.
• A test file given to a test runner (python -m pytest path,
  pytest.main([...]), python -m unittest module, unittest.main (module=...), loadTestsFromName) counted as an import; it runs whole,
  conftests and fixtures included. unittest discovery,
  doctest.testfile, DocFileSuite and DocFileTest, a directory or a
  non-literal argument can run anything, so any change selects the test.
• Functions obtained from pickle.load/loads (and cloudpickle, dill,
  joblib) or from a module loaded by a computed file path were invisible
  without a recording; such code is now affected by any change. A round
  trip (loads(dumps(x)), a temporary file), pickle bytes written in the
  code and a path in the module's own directory are bounded by what they
  load.
• Patches: one that may outlast its test (patch(...).start(), a patcher
  kept for later, a MonkeyPatch() of your own, a with mock.patch(...)
  around a fixture's yield, setattr on any other receiver) is a store
  on the module it patches, so lookups on that module follow it, and set
  in sys.modules that way it counts as installed for good. A with mock.patch.dict(os.environ) or warnings.catch_warnings() block whose
  body yields does not put the process state back. mock.patch.multiple ("pkg.mod", NAME=...) depends on those names and the module; any dotted
  patch target counts as an import of its module, also through a patcher
  called on a value (pytest.MonkeyPatch().setattr("pkg.mod.NAME", ...)).
• A write to process-global state (os.environ, warning filters,
  sys.path) is recognised whichever import binds the name: from os import *, an import in try with another in except, or a
  function-local one. A with statement counts as putting state back only
  when every binding of its name is the restoring one.
• Snapshots: WORKTREE did not see edits to files flagged
  assume-unchanged or skip-worktree. It now reads git-ignored .py files
  in directories git does not ignore as a whole (a generated
  _version.py, local_settings.py); ignored directories are still
  skipped. File mode changes (chmod +x) under the source roots are
  changes, in commits and the working tree. A submodule under the roots
  no longer makes every WORKTREE plan select everything (one checked out
  at another commit, or with uncommitted edits, still counts as a
  change); a path containing a line break, or a tracked file replaced by
  a directory, no longer fails the plan.
• A file nested too deeply to analyse is an analysis error that selects
  everything (a test, conftest or benchmark module: a file that did not
  parse, which makes discovery incomplete), instead of a crash. A
  diffcone.toml that is present but can't be read (left unmerged by a
  conflicted merge) is an analysis error too; it was read as no file. A
  manifest that is not valid JSON says so.

Discovery

• A fixture defined with a decorator whose name ends in fixture
  (pytest-twisted's async_fixture and async_yield_fixture,
  pytest-trio's trio_fixture) was not seen; when its name was also a
  well-known plugin's (pytest-django's client), it was taken for that
  plugin's fixture and what it requested reached nothing. Such decorators
  now make fixtures, and a plugin's fixture name is no longer assumed when
  the test's class, module, conftests or plugins define it in a way
  diffcone can't read: the name is an unknown fixture, selected on every
  change.
• An autouse fixture imported into a conftest, a test module or a plugin
  (by name, through star imports at any depth, or as a module attribute)
  was not a dependency of the tests it applies to; nor was a fixture with
  name= imported under its function name.
• Conftests pytest loads at startup (in the test* directories of a path
  it starts from) count even when ignored or under norecursedirs;
  --ignore, --ignore-glob and norecursedirs apply only below the
  paths pytest starts from, and not under a conftest whose
  pytest_ignore_collect can return False.
• pytest_plugins is read however it is built (+, +=, .append,
  .extend, a comma-separated string, a variable) or imported from
  another module; one that can't be read keeps every test selected
  (unreadable_plugins), as does a plugin module in the repository
  outside the source roots (plugin_outside_roots). The pytest_*
  methods of an object registered as a plugin
  (config.pluginmanager.register(...)), and plugins registered with
  import_plugin("name"), apply to every test.
• pytestmark lists built with +, +=, .append or .extend, or
  imported, apply their usefixtures; indirect= given as a variable, as
  the third positional argument or through **kwargs no longer drops the
  fixture from the test's dependencies.
• Tests defined, aliased or marked __test__ = True inside if/try
  blocks, and tests imported from such modules, are found; an imported
  test discovery can't follow is reported, not dropped.
• Code pytest runs while collecting that changes the process (a conftest,
  a test module or a module they import that sets an environment variable
  or extends sys.path at import, a pytest_generate_tests that does,
  also through a patch nothing undoes: mock.patch.dict(os.environ, ...).start(), pytest.MonkeyPatch().setenv(...), syspath_prepend)
  is a dependency of every test.
• A collection hook imported into a conftest (by a star import, or from
  outside the source roots), or a star import from outside the roots in a
  conftest or plugin, makes discovery possibly incomplete.
• A pytester/testdir test depends on the project's registered plugins
  (pytest11 entry points, pytest_plugins, -p) and on what its inline
  files import.
• ASV: setUp and TearDown in any case, and the benchmark class's
  __init__, are dependencies of its benchmarks; a timeraw_ benchmark
  bound under another name, or renamed with benchmark_name, depends on
  what its code imports; benchmark_name applies wherever the function is
  bound, inherited methods included; benchmark_dir: "." finds
  benchmarks.

Execution evidence

• A module stored into pytest's doctest_namespace (pandas' autouse
  add_doctest_imports does doctest_namespace["pd"] = pd) no longer
  counts as handed to unknown code. When that module could reach gc,
  every test that ran the fixture counted as able to look up anything, so
  a docstring edit selected every test.
• A module variable filled once by one function and then only read (a
  lazily computed constant such as pandas' _pkg_dir in
  find_stack_level) no longer counts as state that function writes for
  later tests: the value is the same whoever computes it first, so a
  change that only makes the function run earlier or later reaches nothing
  through it. A change to the function itself, or a reader outside it,
  still counts.
• Code that ran while pytest collected a test file (building the ids of
  parametrized tests, an ids= function, pytest_generate_tests and what
  it calls) now selects that file's tests when it changes, and the tests
  reading what it stored, instead of every test; so does a change to a
  file read then. It still selects every test when what ran it can set
  something for the whole process (an environment variable) or reaches
  pytest's configuration or session (.config, .session, .stash,
  .pluginmanager, another hook). Code run by session hooks
  (pytest_configure, pytest_collection_modifyitems) selects every test
  as before.
• A test that failed or errored while it was recorded is always selected:
  its recording stops where it failed, so where it passes (a flaky failure,
  a different environment) it can run code the recording never saw.
  VIRTUAL_ENV is now part of the environment a recording is checked
  against: a tool run from its own environment passes its VIRTUAL_ENV to
  the suite, and IPython warns on one that is not the interpreter's.
• A test module listing the machine's locales while pytest collected it
  (locale -a, as pandas' locale tests do) made every change select every
  test. Listing locales runs no Python, and no longer counts as a process
  the recording can't follow. Nor does ninja, which an editable
  meson-python install (pandas) runs on import to rebuild compiled code.
• Before Python 3.14, a test that had a library evaluate string
  annotations (from __future__ import annotations, forward references)
  through typing.get_type_hints, as pydantic and FastAPI do, recorded
  nothing of them, so a change to those annotations, or to a class they
  name, did not select it. The recorder now credits the test with the
  annotations' owner and what their names resolve to, through typing,
  annotationlib and inspect alike; inspect's evaluation no longer
  counts as running text code, which selected the test on every change.
• A recording no longer disables pytest's cache plugin: a configured
  --failed-first made recording a usage error. The plugin starts from
  an empty cache of its own, so --lf/--ff neither narrow nor reorder
  the recorded run, and the repository's .pytest_cache is left alone.
• Editing one test selected every test of its file: pytest looks up each
  test's location as the test starts, which read the file inside the
  test's record. That lookup is no longer recorded; a test that reads its
  own file still is.
• On Python 3.14, editing the body of a function whose annotations were
  read outside the tests selected every test, or every test importing its
  module. Python 3.14 evaluates annotations in code of their own, which
  pytest runs while collecting (reading a signature) and frameworks run at
  import (FastAPI, dataclasses, pydantic). That code no longer counts as
  the function running there, nor as reading a change its annotations
  don't name; a change to the annotations still counts.
• Editing a test file selected every test when the suite was recorded
  with a virtual environment inside the checkout (uv run pytest runs
  .venv/bin/pytest): pytest reading files during collection looked like
  the project reading them outside any test. An environment in the
  checkout, or inside a source directory (where it hid that directory's
  code or gave a false "installed copy" error), is now installed code.
• A session hook that lists installed distributions
  (importlib.metadata.distributions()) made adding any file anywhere
  select every test; those scans are no longer recorded.
• The recorder starts before pytest loads any -p plugin (addopts,
  PYTEST_ADDOPTS, the command) and before a project sitecustomize.py,
  so what they import is recorded. Modules imported before it could start
  are listed (preloaded_modules), collect warns, and any change in
  them is planned from the code.
• Files the recording missed are recorded: files a library reads while it
  is imported (matplotlib's matplotlibrc), files reached through a
  directory descriptor (dir_fd=, os.listdir(fd), os.fstat), entries
  of an os.scandir listing, os.readlink, sqlite3 file: URIs,
  symlinked paths, and on macOS and Windows a file opened under another
  capitalisation. A descriptor the recorder cannot trace flags the test.
• A process started while a conftest.py is imported or in a hook was not
  credited to anything; what it runs now counts as part of that import or
  hook, and one the recording cannot follow makes every change plan from
  the code (pytest-xdist workers are not affected). A test using a
  multiprocessing pool, forkserver or forked process that an earlier
  test started is always selected while that process runs.
• Tests that run code compiled from text no file holds (doctests,
  --doctest-modules, timeit, a notebook kernel) are always selected
  (rule text_code). A string skipif condition naming nothing of the
  project and autospec=True mocks do not count as such text.
• A change to code that runs at import, or that ran during an import,
  reaches the tests reading what it writes into another module, Cython
  code included. Code that can obtain a test module without importing it
  by name sees changes to it: library code importing one by a computed
  name, an unpickler, a loader of a computed path. A lazy-export
  __getattr__ whose table only test-time code can change does not count
  as such code, nor do gc.get_stats(), gc.collect() and
  gc.callbacks, and hasattr(obj, "name") notices only name.
  pytest_plugins in a test module selects every test when it changes.
• A function added, deleted or redefined with an annotation that calls
  code (def f(x: set_mode("fast"))), or a module-level variable whose
  annotation does, reaches what that code writes: before Python 3.14
  annotations run with the definition. A definition whose annotations call
  nothing is not escalated.
• Caches: a module-level cache filled by an earlier test hid the function
  that filled it from later tests' records, in every order; a change to
  that function now reaches the tests reading the cache. So does a cache
  kept on an object, a class, a closure, a function attribute or in a
  memoising decorator diffcone knows (functools.cached_property,
  cachetools, Django's and Werkzeug's cached_property); a decorator
  from an installed package it doesn't know is taken to keep nothing. A
  store made by a test's own code counts for that test only.
• A non-literal getattr on a standard or third-party module sees
  changes anywhere in the project, not only in its own imports, and a
  test module's import-time change reaches lookups in library code.
• Code a hook ran that changed, or that reads a change, selects every
  test.
• Cython: edits to blank lines or # lines inside a string in a function
  body were ignored.
• A recording stored under a path containing #, ? or % was skipped
  as if there were none.
• When run couldn't use a recording because only -O or PYTHONPATH
  differed, it listed no differences; every difference is now named.

Running, check and validate

• validate and corpus could not run the suites of several projects.
  A checkout of a commit lacks a git-ignored, build-generated module such
  as the _version.py setuptools-scm or hatch-vcs writes, so the package
  could not be imported; the repository's copy of each git-ignored module
  inside a package is now copied into every checkout that lacks it
  (not a conftest or test file; --setup-command runs after it). pytest's cache plugin is no longer
  disabled, which made a configured --failed-first or --lf a usage
  error; it starts from an empty cache of its own instead, so those options
  neither narrow nor reorder the run, and the repository's .pytest_cache
  is left alone (a run whose cache could not be moved is refused). With --coverage, a suite that did not run is reported
  as such instead of as a missing coverage database; pytest-cov is loaded
  with -p when plugin autoloading is disabled; the database's place is
  set in diffcone's coverage config instead of COVERAGE_FILE, which the
  tests' own processes inherited; and parallel parts beside it are read
  too. Errors name the snapshot (base <commit> or head) whose suite
  failed.
• validate and corpus reported tests outside the target list as
  misses even when the plan said discovery was incomplete (it exits 3,
  and run refuses it). They are now listed as "outside the target list"
  and don't fail the check, and coverage recall is measured over the
  targets. When discovery claims to be complete, such a test is still a
  miss, labelled MISSED (not a target). The JSON gains
  discovery_incomplete, outside_targets and a target flag on each
  record; counts.targets now counts targets only.
• diffcone check decided "already failing at the baseline" per test
  function, so a parametrized case that failed at the baseline excused a
  new failure of another case: a miss reported as clean. Cases are now
  compared one by one.
• check and validate work when pytest's rootdir lies below the
  repository (an ini file in a subdirectory): the plan records pytest's
  rootdir, configuration file and --junit-prefix, check --rootdir and
  --junit-prefix override them, and a directory that fails to collect
  counts as a collection failure of every test under it. Before, every
  failure was an unknown miss, which the record action confirmed.
• validate, corpus and collect run the whole suite whatever -x or
  --maxfail is configured, and validate and corpus stop with exit
  code 2 when the suite reported fewer outcomes than it collected: a miss
  can no longer hide behind an early stop. validate runs pytest without
  -v and reports the command line it ran.
• run refuses a working tree that differs from the planned snapshot in
  any file the plan would have selected on (data and configuration files
  under the roots, diffcone.toml, runner configuration, build inputs,
  conftests, ignored .py files, file modes, dirty submodules, files
  flagged assume-unchanged or skip-worktree, .py files with non-ASCII
  names) and names them.
• run's "did not collect" message names --lf, --ff, --sw, -k,
  -m or --deselect among the effective pytest arguments and where
  they are set.
• collect and run --collect no longer refuse a clean checkout because
  untracked bytecode or ASV output has a non-ASCII name; a renamed file
  counts as both names.
• run --dry-run --evidence prints a command you can run, without
  diffcone's internal recorder plugin.
• discover honours --no-cache and --cache-dir: it uses the same
  cache as plan, and its output is unchanged.
• On Windows, a relative --command resolved for validate and corpus
  was quoted for a POSIX shell, and one written with forward slashes
  (.venv/Scripts/python.exe) was not resolved.
• An unexpected error in any command exits 2; check, report,
  collect and prune exited 1, which check uses for a miss.

CI actions

• The record action with check: true could pass a push that had a
  miss: a re-run of the job restored the newest recording instead of its
  own commit's (so the kept verdict was lost and the push was re-checked
  against a later baseline), and a check that crashed, or a re-run whose
  JUnit could not be read, was recorded as checked with no miss. A
  recording that is not from before the pushed commit is no longer
  checked against. A re-run's other failures (when the job's pytest
  arguments name a path after an option) no longer count as misses, and
  the re-run of a missed test runs a command with quotes or spaces as
  written.
• diffcone report ignores recording artifacts uploaded by pull-request
  runs (a pull request from a fork runs its own code), counts an artifact
  it cannot read as a problem instead of failing, reports pull-request
  runs whose selected tests pytest did not collect, keeps a test id with
  backticks inside its code span, and escapes | in "not checked"
  reasons.
• The report action failed whenever a job of a reported run had been
  re-run (two artifacts of the same name); it reads the latest attempt's.
  Its window starts at the last report that posted, so a run with only a
  job summary (a manual run with posting off) no longer hides the runs
  before it.
• The record action's re-run of a new unselected failure stopped at the
  first failure when pytest-args or addopts had -x or --maxfail,
  and the misses it never reached were reported as flaky. It now re-runs
  with --maxfail=0, and a missed test the re-run did not run still
  counts as a miss.

Discovery

• pytest options in PYTEST_ADDOPTS, plugins in PYTEST_PLUGINS and
  pytest arguments written into the run's command are read: plan and
  discover take --command, the CI actions pass it, and the report
  lists what was read (run_arguments). A command whose pytest arguments
  cannot be found is reported.
• Understood now: . as an argument, a .txt or .rst file named as an
  argument (always a doctest), --doctest-glob patterns with a directory,
  option values written as separate arguments (--ignore tests/slow),
  absolute paths inside the repository, -o name=value overrides and
  paths in addopts.
• A pytest configuration file below the repository root that pytest
  would use (one nearer the paths it starts from than the root's) is not
  read by discovery, so it makes discovery incomplete
  (unmodelled_runner_option, exit code 3; run refuses without
  --allow-incomplete-discovery).
• A pytest configuration file (pytest.toml, pytest.ini,
  pyproject.toml, tox.ini, setup.cfg) or an asv.conf.json that
  can't be parsed made discovery quietly use the defaults; it now makes
  discovery incomplete (unparsable_config, exit code 3), as does a
  benchmark_dir outside the repository.
• asv.conf.jsonc is read like asv.conf.json, and trailing commas in
  either are accepted, as asv allows.

Execution evidence

• A Python process a test starts with subprocess (directly, through
  asyncio, or through uv run) records itself on Linux and macOS, and
  the test is credited with what it ran instead of being selected for
  every change; so is every test that runs while the process is still
  running (a server or a warm worker another test started). A python -c
  probe that runs project code is caught by its own recording, and a
  project's own sitecustomize.py does not stop subprocesses from
  recording themselves. A process started another way (os.system, a
  shell, multiprocessing, anything on Windows), one that cannot record
  itself (a replaced environment, python -S, Python before 3.12), one
  whose recording was cut short (a full disk), and one running project
  code from a -c snippet or a script outside the source roots still
  select their test for every change.
• A test run that changes its own environment (a test that installs or
  upgrades a package with the test interpreter) is named: collect warns
  with the packages, the recording keeps the environment it started in,
  and run says when it meets that environment. The recording is still
  keyed by the environment at the end, so such a run plans from the code.
• plan --evidence says the environment was not checked
  (environment_checked: false): only run can check it, and its -o
  report says true when the environment matched.
• run --collect on a recording made with --reverse-check runs the
  selected tests again in reverse order, so the advanced records are
  order-checked too.
• A method or attribute added to (or changed or deleted on) a fake class
  in a test module selects only the tests that ran library code calling a
  method of that name (x.read()) and also ran code able to hand that
  code a fake: code naming the class or a subclass, or finding it without
  naming it (a base's __subclasses__(), gc, a frame's globals). A
  fake built at import, in a decorator or during collection, or one whose
  class or bases sit outside test code, still selects every such test.
• A lookup on a standard or third-party module that tests patch
  (dir(builtins) or hasattr(os, name) in library code, with
  monkeypatch.setattr(builtins, ...) in a test) no longer makes every
  change anywhere select every test that ran the lookup: the tests that
  patch the module are selected through their own code. A module written
  at import or in a hook is treated as before, and a change to code that
  runs at import still reaches the lookup when it can run a writer.
• Editing an ordinary module named build.py (not at the repository root
  or beside pyproject.toml/setup.cfg) selects the tests that ran it,
  as static planning does, instead of every test.
• Recordings no longer select every test for every change when pytest
  plugins or test helpers ask the machine or the repository something
  outside the tests: git reporting the commit, branch, remote or
  configuration (pytest-benchmark does as it starts) and ps
  (shellingham's shell detection) join uname and locale. A git
  command that prints a file or writes still counts.
• Running the suite under coverage (--cov in addopts) no longer makes
  every source file look read by the module importing it: coverage
  resolving a file it is about to measure is nobody's read.
• Tests that build URLs with werkzeug (every Flask app) no longer count as
  running code compiled from text, which selected them for every change.
• A change to a module no test imports (a Sphinx docs/conf.py) no
  longer reaches code that looks names up on whatever it is handed (a
  frame's globals, a function's __code__, sys.modules), which selected
  every test importing that code. A module imported by any test, even
  inside one, or by an import added in the same change, still does.

CI

• record describes every recording for diffcone report, also without
  check (a nightly recording) and when the recording failed: its
  results output always holds context.json.
• report takes repository (report on another repository's runs; a
  public one needs no extra access), issues-repository and
  issues-token (post the report and miss issues elsewhere, each source
  repository with its own report issue), and comment: on-problem
  (comment only on a miss, a failed check or an incomplete report). Miss
  issues outside diffcone's tracker link to a pre-filled diffcone issue.
• run -o adds evidence_not_used (why the recording was not used, and
  what differed) to the plan it writes, and diffcone report shows those
  differences per job.