-
-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
zig test
needs a way to run tests for multiple files
#10018
Comments
Every project has different constraints on logical dependency of how tests are build. Think of tests requiring external dependencies, fetching up-to-date data from the system etc. This point is more of a question, how this should be communicated.
This should be (de)motivated by a use case. I dont understand the performance implications for different project sizes. Maybe you can also argue, how this can not be fixed in the caching system?
That is a point against using
Many projects invoke automatic tests after building or making sure dependencies are build etc. Take arocc. Sidenode: Tests are also build, but then "run". One could distinguish here within the build system, but in practice users expect them to be run immediately on default. |
As someone who is new to Zig having two ways to run tests has been confusing, but I understand why both exist. It does seem like It occurs to me that if we made it easier to replicate all the |
I'm very much in favor of the entire proposal! I think the fact that there are entire blog posts and threads on Reddit about |
It would be nice if This would likely reduce the confusion when it looks as if only three tests were picked up when you have 30 or so tests in other files. |
Currently, if you pass multiple files to
zig test
, it fails with this error:Say you have many files with tests in them, and you want to run all the tests for those files.
For test runners in other languages, the command for running tests in one file is the same for running tests in several files.
Rust:
Go:
JavaScript:
Why not
zig test main.zig
?You can run
zig test
on the entry point, but it's a footgun.zig test
silently skips tests in files not referenced by runtime code. That means it reports all tests passed without necessarily running all the tests. This is probably an unintentional side effect of zig's fantastic dead code elimination.Why not
zig build test
?zig build
cache invalidates frequently, making it several seconds slower than manually invokingzig test
on the target filezig build
sounds like it's for building, not for testing.Proposal
Part 1
Support multiple file paths as input in
zig test
:zig test src/file1.zig src/file2.zig
This would run the tests in those files.
Note: I am not suggesting a glob matcher. It would be nice, but the shell can do that.
Part 1A
Change the default behavior of
zig test
to only runtest { }
andtest ".*" { }
for the input file(s), instead of all files referenced recursively.zig test src/file1.zig
This would only run the
test
blocks defined insrc/file1.zig
. Ifsrc/file1.zig
imports another file withtest
blocks in them, those tests would not be run.This change would:
zig test
src/file1.zig
, I don't want to see test failures or build failures from tests insrc/strings.zig
.The current way to do this is by adding a prefix to every test name and then passing that prefix to
zig test
. Humans shouldn't have to add a prefix to every test's name.Part 1B
Being able to run all tests for an entry point is very useful.
Rather than removing that, this proposal suggests changing it from the default behavior to a flag.
Here are two ideas for flag names that come to mind:
Part 2
Add support for
build.zig
to specify default options forzig test
, without wrappingzig test
aszig build test
Why? Currently, projects with dependencies can't run
zig test
without adding a lot of flags.My current incantation for running a test looks something like this:
This proposal does not go into the details of what the API would look like for
build.zig
to specify default options forzig test
.Summary
If this entire proposal were to be implemented:
Other
This is the script I currently use to generate a file that imports all files with tests, regardless of whether the test files are referenced by runtime code in the real entry point.
The text was updated successfully, but these errors were encountered: