Skip to content

twister: test-only failed to work #80332

@golowanow

Description

@golowanow

Describe the bug
Twister --test-only execution mode does not work after #77250 dfc7860

To Reproduce
Run --build-only and then --test-only

$ git checkout dfc7860ab1f9
$ ./scripts/twister -p qemu_x86 -T tests/benchmarks/latency_measure -b
INFO    - Zephyr version: v3.7.0-4982-gdfc7860ab1f9
....
INFO    - Total complete:    3/   3  100%  built (not run):    2, skipped:    1, failed:    0, error:    0
INFO    - 3 test scenarios (3 test instances) selected, 1 configurations skipped (1 by static filter, 0 at runtime).
INFO    - 0 of 3 test configurations passed (0.00%), 2 built (not run), 0 failed, 0 errored, 1 skipped with 0 warnings in 23.09 seconds
INFO    - 0 test configurations executed on platforms, 2 test configurations were only built.

$ ./scripts/twister -p qemu_x86 -T tests/benchmarks/latency_measure --test-only
Keeping artifacts untouched
INFO    - Zephyr version: v3.7.0-4982-gdfc7860ab1f9
....
INFO    - 3 test scenarios (0 test instances) selected, 0 configurations skipped (0 by static filter, 0 at runtime).
INFO    - 0 of 0 test configurations passed (0.00%), 0 built (not run), 0 failed, 0 errored, 0 skipped with 0 warnings in 2.13 seconds
INFO    - In total 0 test cases were executed, 0 skipped on 0 out of total 849 platforms (0.00%)
INFO    - 0 test configurations executed on platforms, 0 test configurations were only built.

Expected behavior
Tests executed:

$ ./scripts/twister -p qemu_x86 -T tests/benchmarks/latency_measure -b
INFO    - Zephyr version: v3.7.0-4981-gf1bf24e4dde2
...
INFO    - Total complete:    3/   3  100%  built (not run):    2, skipped:    1, failed:    0, error:    0
INFO    - 3 test scenarios (3 test instances) selected, 1 configurations skipped (1 by static filter, 0 at runtime).
INFO    - 0 of 3 test configurations passed (0.00%), 2 built (not run), 0 failed, 0 errored, 1 skipped with 0 warnings in 28.97 seconds
INFO    - 0 test configurations executed on platforms, 2 test configurations were only built.

$ ./scripts/twister -p qemu_x86 -T tests/benchmarks/latency_measure --test-only
Keeping artifacts untouched
INFO    - Zephyr version: v3.7.0-4981-gf1bf24e4dde2
....
INFO    - Total complete:    2/   2  100%  built (not run):    0, skipped:    0, failed:    0, error:    0
INFO    - 3 test scenarios (2 test instances) selected, 0 configurations skipped (0 by static filter, 0 at runtime).
INFO    - 2 of 2 test configurations passed (100.00%), 0 built (not run), 0 failed, 0 errored, 0 skipped with 0 warnings in 17.18 seconds
INFO    - In total 2 test cases were executed, 0 skipped on 0 out of total 10 platforms (0.00%)
INFO    - 2 test configurations executed on platforms, 0 test configurations were only built.

Impact
--test-only mode is not working

Environment (please complete the following information):

  • OS: linux
  • Toolchain Zephyr SDK, 0.16.8
  • Commit SHA dfc7860

Metadata

Metadata

Assignees

Labels

RegressionSomething, which was working, does not anymorearea: TwisterTwisterbugThe issue is a bug, or the PR is fixing a bug

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions