Skip to content

v0.3.2

Choose a tag to compare

@m-mcgowan m-mcgowan released this 02 Sep 04:15
· 11 commits to main since this release

Fixed

  • RESUME_AFTER silently dropped tests (high severity — sessions could
    skip whole suites and still exit PASSED). apply_resume_after() computed
    the resume point as an index over all registered tests, then passed it to
    doctest's first option, which is an index over tests that pass filters
    (doctest compares it against numTestCasesPassingFilters). Any
    doctest::skip()-attributed or filtered-out test before the resume point
    made the two spaces diverge, so first was set too high and silently
    excluded that many tests after the resume point. The error compounded
    across sleep/wake cycles; when the inflated index passed the end of the
    list, a cycle ran nothing at all. Observed in the field: 148 of 186 tests
    ran, 8 of 35 suites never started, result reported PASSED.

    Resume is now implemented by force-setting m_skip on the already-completed
    prefix instead of using first. This removes the index arithmetic entirely,
    lets doctest's own filter chain do the counting, and composes correctly with
    filters and env requirements applied after the resume is processed.

  • Under-run is now a failure, not a pass. The runner compares tests
    actually executed against the runnable count the device reported on its
    first cycle, and reports an ERRORED case (runner/under_run) when tests
    went missing. The summary line also shows the shortfall
    (N ran of M expected | K NEVER RAN). A run that cannot account for its
    own tests no longer looks green.

  • ETST:COUNTS run/skip were wrong during resume cycles. The count
    applied a manual skip_count adjustment that double-counted once resume
    skips became visible to count_passing_filters(), and silently did nothing
    in the run == skip_count case. The adjustment is removed; the reported
    counts now reflect what will actually run.

  • etst::require_env broke on include order. env.h referred to
    doctest::detail::TestCase unqualified from inside namespace etst, which
    resolves to etst::doctest — the runner's own namespace — as soon as any
    header declaring it is included first, failing with
    'etst::doctest::detail' has not been declared. Now spelled ::doctest::.
    Latent until resume.h was introduced; it would have bitten any consumer
    that included a runner header before env.h.

Changed

  • scripts/release.sh releases one repo at a time. It previously released
    embedded-bridge and pio-test-runner in lockstep under a single version, which
    the two repos had already outgrown (bridge at v0.2.0, runner at v0.3.1) and
    which hard-failed whenever one repo had an empty [Unreleased] section. Each
    repo now carries its own version and is released on its own; --repo <dir>
    picks the target. The dependency is still enforced, but as a check rather
    than a chain: releasing pio-test-runner stops if embedded-bridge has commits
    past its own last release tag, and says nothing when it does not. Also adds
    --no-push, which does the local bump/commit/tag and prints the publish
    commands instead of pushing.

Added

  • etst/doctest/resume.h — the resume-point selection (sorted_registry(),
    select_resume_after()) extracted out of runner.h into a header carrying no
    Arduino or ESP dependency, so the native C++ harness can test the real code
    instead of a hand-mirrored copy. runner.h keeps apply_resume_after()
    unchanged as the logging wrapper, so consumers need no changes.

  • Native regression coverage for resume selection
    (tests/test_doctest_internals.cpp, six cases, run in CI). Fixture cases
    place doctest::skip()-attributed tests deliberately before the resume
    point — the exact condition under which the two index spaces diverged — and
    assert that tests after the resume point still run, that the completed prefix
    is skipped exactly, that already-skipped tests after the point are untouched,
    that an unknown resume point reports -1 and mutates nothing, and that
    sorted_registry() holds its (file, line) wire ordering. A seventh case
    guards the fixture itself, failing loudly if a future edit stops reproducing
    the divergence.

    Verified discriminating: rebuilding the harness against a shadow resume.h
    carrying the old first-based selection fails exactly one case
    (tests after the resume point still run despite skipped tests before it)
    by exactly two tests — one per skip-attributed fixture case.