Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testssuite #5022

Closed
wants to merge 2 commits into from

Conversation

Projects
None yet
4 participants
@rochamatcomp
Copy link
Contributor

commented Feb 8, 2018

Reorganize integration tests as one crate with many modules #4867.

@rust-highfive

This comment has been minimized.

Copy link

commented Feb 8, 2018

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @matklad (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

@matklad

This comment has been minimized.

Copy link
Member

commented Feb 8, 2018

Excellent @rochamatcomp!

Could you do some measurements to test that it actually improves things instead of making them worse?

I think the following metrics are interesting (before and after this change):

  1. the size of /target directory after cargo test --no-run
  2. the time to compile the tests (first, cargo build --lib && cargo build --bin to build cargo itself, then actually measure cargo test --no-run)
  3. the time to recompile a single test after Cargo library is edited
  4. the time to recompile a single test after the test itself is edited
@matklad

This comment has been minimized.

Copy link
Member

commented Feb 8, 2018

Oh, because we'd want to take advantage of incremental compilation, I think it makes sense to do the measurements using the current beta.

@rochamatcomp

This comment has been minimized.

Copy link
Contributor Author

commented Feb 10, 2018

rustup show
beta-x86_64-unknown-linux-gnu (default)
rustc 1.24.0-beta.11 (03f456d3c 2018-02-03)

  1. the time to compile the tests (before cargo build);
  2. the time to compile the tests (after cargo build);
  3. the time to recompile a single test after Cargo library is edited;
  4. the time to recompile a single test after the test itself is edited.

The time in seconds.

Test Command Original Testsuite Absolute diff Relative diff (%)
1 cargo test --no-run 125.64 73.40 -52.24 -41.58
2 cargo build --lib 56.15 55.83 -0.32 -0.57
cargo build --bins 9.41 9.42 +0.01 +0.11
cargo test --no-run (after cargo build) 68.87 19.95 -48.92 -71.03
TOTAL 134.43 85.20 -49.23 -36.62
3 cargo test build --no-run (after Cargo library edit) 56.33 11.62 -44.71 -79.37
4 cargo test build --no-run (after test build.rs edit) 3.38 4.96 +1.58 +46,75
  1. The size of /target directory in gigabytes.
Test Command Original Testsuite Absolute diff Relative diff (%)
1 du -hs target/ 5.8 1.1 -4.7 -81.04
@matklad

This comment has been minimized.

Copy link
Member

commented Feb 10, 2018

Awww, awesome, @rochamatcomp !

@alexcrichton seems like a massive win overall to me, what would you think?

@alexcrichton

This comment has been minimized.

Copy link
Member

commented Feb 10, 2018

Thanks for getting those numbers! I'm game for merging.

@matklad

This comment has been minimized.

Copy link
Member

commented Feb 11, 2018

@rochamatcomp could you rebase the PR so that it merges cleanly?

bors added a commit that referenced this pull request Feb 21, 2018

Auto merge of #5063 - Eh2406:TestssuiteV3, r=matklad
Reorganize integration tests as one crate with many modules. Issue #4867. Rebased V3

This is an automatic rebase of @rochamatcomp's work in #5022, as requested in [#5038](#5038 (comment)).

I take no credit for the changes; I just wanted faster test for my work. :-)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.