New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Omitting src/tests.rs from the package causes crater to skip this crate #425
Comments
First of all, thanks for what you're doing! Having crater cover more things is always a plus. Given what you're doing, I presume you know some details about crater; apologies if this is not the case. I have a couple questions:
|
I know a bit about crater.
It is not. As far as I can tell, the GitHub repositories that are the source of published crates are omitted.
Phew, good thing you asked. I think this exact problem affects other crates as well, and it turns out I do not currently have a solution on hand. Crater does not build or run tests with I think I'll toss something together into a crater PR, and keep this issue in sync when I do. |
More than me, which is basically what I was asking 🙂
That certainly makes sense.
Well, that's certainly not ideal. I'll still likely end up making the change here, but having some metadata like there is for docs.rs would be wonderful. |
While having an option to request crater run with certain flags would be ideal, this is still not the case. As such, I implemented a neat workaround. If all features are not enabled, just spawn a child process with all the features enabled, failing the single test if the process as a whole fails. This is actually surprisingly easy to accomplish. I then updated doctests to work with the default feature set, so running I've removed the file exclusion from |
I've been manually inspecting the crater logs from the most-downloaded crates, and I think a lot of people don't realize how many published crates are skipped.
This crate is skipped because its tests do not build based on the package that is uploaded to crates.io. Crater attempts to run
cargo test --no-run
and it gets:Full log here: https://crater-reports.s3.amazonaws.com/beta-1.58-1/1.57.0/reg/time-0.3.5/log.txt
This causes crate to classify this crate as build-fail (build compiler error), and thus it is not inspected for regressions between Rust releases. Though this crate's code may be well-tested by its widespread use, so whether you care about this is really up to you.
I suggest not excluding this file from the package. It would be nice to increase the precision of crater.
The text was updated successfully, but these errors were encountered: