Skip to content

A @data_provider that yields no data makes the test vanish #1145

Description

@Chemaclass

Problem

A @data_provider that names a function which is not defined — a typo, or a
provider that was renamed — produces no data sets. The runner's per-data-set
loop body then never runs, so the test leaves the run without a trace:

# @data_provider provider_typo
function test_x() { assert_same "a" "$1"; }
Tests:      0 total
 No tests found

Two things are wrong. The summary blames a missing test when what is missing
is the provider, and the name of the annotation is never printed — so the one
piece of information that identifies the typo is withheld.

A provider that is defined but returns nothing drops the test the same way, and
is harder to spot: the name resolves, so nothing looks wrong.

Worse under --parallel

The per-file loop runs in a background subshell there, so a counter incremented
in it dies with the subshell — only .result files reach the aggregate. A file
with one good test and one broken provider prints the error and exits 0:

✗ Error: Broken
Tests:      1 passed, 1 total     # exit 0

Sequential exits 1 for the same file. CI on the parallel path goes green with
the reason on screen.

Fix

Report the test as an error naming the provider, distinguishing "is not
defined" from "produced no data", and write a .result payload so the failure
survives parallel aggregation.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions