fix(test): strip inline tests from local-extended children of remote presets#339
Conversation
intent(test): prevent preset-authored tests from being evaluated under downstream user overrides when a remote preset reaches its child files via local-path extends (the layout used by runok-presets/base, which extends ./readonly-unix.yml etc.). Previously only the outermost remote file was stripped; nested children leaked their tests and failed under user overrides such as deny: 'find *'. decision(test): track an under_remote flag through resolve_extends_recursive so any preset reached via a remote ancestor is stripped, regardless of whether each child reference itself is remote or local. The user's own config and presets they extend directly via local paths keep their tests. rejected(test): stripping based purely on PresetReference::Local at load time — the load site cannot see whether the parent in the chain was remote, so it cannot decide on its own.
intent(test): close a gap raised in pre-push review — verify that crossing the remote boundary partway through an extends chain (user → local → remote → local-under-remote) only strips presets at and after the boundary. The previously added tests covered remote-at-the-root and local-only chains but not this mixed shape, which is what real downstream configs look like. decision(test): also pin the invariant — `load_preset_with` and `resolve_extends_recursive` apply the same strip in two places (the former for direct callers like `load_preset`, the latter for nested children reached via local paths). The double pass is idempotent but easy to retrofit incorrectly, so leave a comment requiring future strip targets to be added in `strip_preset_tests` rather than only at the load site.
intent(docs): satisfy the project rule that release notes entries must carry their PR number; the placeholder existed because the PR was not yet open at draft time.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #339 +/- ##
==========================================
+ Coverage 88.78% 88.89% +0.11%
==========================================
Files 53 53
Lines 12322 12369 +47
==========================================
+ Hits 10940 10996 +56
+ Misses 1382 1373 -9
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Code Review
This pull request fixes a bug where runok test would evaluate inline tests from presets reached via a remote ancestor, even if those presets were referenced through local paths. The fix involves tracking the remote status through the recursive extension resolution and stripping tests from any preset loaded under a remote boundary. A review comment correctly identifies that the new tests repeat setup logic for PresetCache and CacheMetadata, which should be extracted into a #[fixture] according to the repository's style guide to improve maintainability.
intent(preset): the four preset-strip tests duplicated the same PresetCache and CacheMetadata setup, which the project test guide says should live in a fixture or helper. Moving the construction into `make_cache` and `seed_remote_preset` makes the failure-relevant differences (which preset files each scenario seeds) the only thing each test spells out.
Purpose
runok testevaluates a large number of preset-authored inline tests regardless of any user override, and a user override turns those intoexpected allow, got denyfailuresgithub:fohte/runok-presets/baseand denyingfind *in the user config produces a flood of failures such asfind . -name "*.txt" => expected allow, got denyReproduction steps
/tmp/test-strip.ymlrunok test -c /tmp/test-strip.ymlevaluates the preset's own inline tests in bulk, e.g.592 passed, 0 failed, 592 totalApproach
extendschain crosses into any remote preset, treat every descendant as preset-authored and strip its inlinetestsand top-leveltests:blockrunok-presets/basewhoseextends: [./readonly-unix.yml, ...]reaches its siblings via local paths left those siblings outside the stripDesign decisions
Where to apply the strip
load_preset_withand additionally strip inresolve_extends_recursivewhile propagating anunder_remoteflag down the chainload_presetthat skip extends resolution (such as the version check inupdate-presets) keep stripping the remote root unchangedstrip_preset_testsso both call sites stay in syncload_preset_withand centralise it inresolve_extends_recursiveload_presetthat skip extends resolution would stop stripping the remote root, changing existing behaviour