Add support for filtering tests based on "features" in runtests.pl #16533
aquilamacedo
started this conversation in
Ideas
Replies: 1 comment
-
I think I like option 2 the most. If it turns out to match too many tests too easily, maybe we could have prefix like |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi, this is comming for a discussion on Debian, from
https://salsa.debian.org/debian/curl/-/merge_requests/39
We’ve been planning to run tests that rely on the "Debug" feature
because some tests developed to validate CVE fixes (such as
tests/data/test780
,tests/data/test781
, etc.) require a Debug build in order to run.However, when running the tests, we need to execute only those that have the
Debug feature set. Initially, we considered using keyword-based filtering
(
runtests.pl [keyword]
), but I noticed that it doesn’t work because theparser only checks values inside the
<keywords>
field, ignoring<features>
.We want to scope the test run to Debug-tests-only because we will do an extra
build of curl (Debug build) just for them, whereas our regular build will run
the non-Debug tests. We want to save time by not runing the non-debug tests
twice.
So, I'm starting this thread to understand the best approach to handle this
scenario.
We see three possible alternatives:
Add a new parameter (
--feature=
) toruntests.pl
to allow selecting testsbased on
<features>
. Example:--feature=debug
.Modify the current keyword filtering behavior (
[keyword]
) to also consider<features>
.Set "Debug" as a test keyword so it works with the existing option.
However, this approach would require modifying all tests that have it as a
feature and duplicating it to the field.
Any other ideas?
I'm willing to implement the solution and open a PR, depending on which
alternative is considered the best
Beta Was this translation helpful? Give feedback.
All reactions