Skip to content

feat(cli): --order-by=defects to run the last run's failures first #1011

Description

@Chemaclass

Problem

--rerun-failed replays the last failing tests and drops everything else (src/config/rerun.sh:113 filter_functions). That is the right tool while fixing a specific failure, but it is the wrong tool for CI and for the final check before pushing, where you want the full suite — just with the known-bad tests first, so --stop-on-failure trips in seconds instead of minutes.

The failure cache that makes this possible already exists (.bashunit/last-failed, src/config/rerun.sh); nothing consumes it for ordering.

Proposal

--order-by <mode>     defects | defined (default) | random
  • defects — tests that failed on the last recorded run execute first, in the order recorded; everything else follows in its normal order. The full suite still runs.
  • defined — current behaviour, named so it can be set explicitly in .bashunitrc.
  • random — equivalent to the existing --random-order; keep --random-order and --seed working unchanged as aliases.

With no cache file, defects degrades silently to defined.

Where to change

  • src/config/rerun.sh:68 load, :83 files, :96 allows — read the cache without filtering it.
  • src/helper/discovery.sh:208 load_test_files and :62 get_functions_to_run — ordering hook, same place --random-order applies.
  • src/main/test.sh, src/config/env.sh (BASHUNIT_ORDER_BY), .env.example.

Acceptance criteria

  • --order-by defects runs the full suite with previously failing tests first
  • --order-by defects --stop-on-failure stops on the first known-bad test on a repeat run
  • Missing or empty .bashunit/last-failed degrades to defined order without a warning storm
  • --order-by random behaves identically to --random-order, and --seed still applies
  • An unknown mode exits non-zero with a usage error listing the valid modes
  • --order-by and --rerun-failed together: --rerun-failed still narrows the set, --order-by only orders it
  • Ordering is stable under --parallel in the sense that the cached failures are dispatched first
  • Unit tests in tests/unit/config/rerun_test.sh plus an acceptance test

Repo checklist (agent)

  • TDD: RED → GREEN → REFACTOR. Write the failing test first.
  • Bash 3.0+ only: no printf -v, no += append, no declare -A, no [[ ]], no ${var,,}, no &>>, no ${arr[-1]}. Expanding a possibly-empty array under set -u needs ${arr[@]+"${arr[@]}"}.
  • A new CLI flag must be wired in all of these or a parity test fails:
  • Gates: make sa, make lint, ./bashunit tests/, ./bashunit --parallel tests/. Never run shfmt -w.
  • Docs: update docs/command-line.md. Editing docs/assertions.md invalidates the bashunit doc acceptance snapshot — regenerate it.
  • CHANGELOG.md: add one line under ## Unreleased.
  • Fixtures under tests/acceptance/fixtures/ must not end in *test.sh.
  • One issue = one PR.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions