Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] split libtest types and formatting into a libtest-utils crate #57068

Closed
wants to merge 1 commit into from

Conversation

gnzlbg
Copy link
Contributor

@gnzlbg gnzlbg commented Dec 22, 2018

Do not merge: I just want to get initial feedback about the general direction. I'll close this afterwards and submit a more finished PR.

cc @djrenren @alexcrichton we'll better chat about this, but this is basically a "small" step towards allowing custom test frameworks to re-use some of the libtest facilities.

@rust-highfive
Copy link
Collaborator

r? @Kimundi

(rust_highfive has picked a reviewer for you, use r? to override)

@rust-highfive
Copy link
Collaborator

⚠️ Warning ⚠️

  • These commits modify submodules.

@rust-highfive rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Dec 22, 2018
@rust-highfive
Copy link
Collaborator

The job x86_64-gnu-llvm-6.0 of your PR failed on Travis (raw log). Through arcane magic we have determined that the following fragments from the build log may contain information about the problem.

Click to expand the log.
travis_time:end:0ce0eb40:start=1545511078985212849,finish=1545511133064664832,duration=54079451983
$ git checkout -qf FETCH_HEAD
travis_fold:end:git.checkout

Encrypted environment variables have been removed for security reasons.
See https://docs.travis-ci.com/user/pull-requests/#pull-requests-and-security-restrictions
$ export SCCACHE_BUCKET=rust-lang-ci-sccache2
$ export SCCACHE_REGION=us-west-1
Setting environment variables from .travis.yml
$ export IMAGE=x86_64-gnu-llvm-6.0
---
#########################                                                 35.7%
######################################################################## 100.0%
[00:01:16] extracting /checkout/obj/build/cache/2018-12-09/cargo-beta-x86_64-unknown-linux-gnu.tar.gz
[00:01:16]     Updating crates.io index
[00:01:24] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:01:24] Build completed unsuccessfully in 0:00:19
[00:01:24] Makefile:81: recipe for target 'prepare' failed
[00:01:24] make: *** [prepare] Error 1
[00:01:25] Command failed. Attempt 2/5:
[00:01:25] Command failed. Attempt 2/5:
[00:01:25] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:01:25] Build completed unsuccessfully in 0:00:00
[00:01:25] make: *** [prepare] Error 1
[00:01:25] Makefile:81: recipe for target 'prepare' failed
[00:01:27] Command failed. Attempt 3/5:
[00:01:27] Command failed. Attempt 3/5:
[00:01:28] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:01:28] Build completed unsuccessfully in 0:00:00
[00:01:28] make: *** [prepare] Error 1
[00:01:28] Makefile:81: recipe for target 'prepare' failed
[00:01:31] Command failed. Attempt 4/5:
[00:01:31] Command failed. Attempt 4/5:
[00:01:31] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:01:31] Build completed unsuccessfully in 0:00:00
[00:01:31] make: *** [prepare] Error 1
[00:01:31] Makefile:81: recipe for target 'prepare' failed
[00:01:35] Command failed. Attempt 5/5:
[00:01:35] Command failed. Attempt 5/5:
[00:01:35] error: the lock file /checkout/Cargo.lock needs to be updated but --locked was passed to prevent this
[00:01:35] Build completed unsuccessfully in 0:00:00
[00:01:35] make: *** [prepare] Error 1
[00:01:35] Makefile:81: recipe for target 'prepare' failed
[00:01:35] The command has failed after 5 attempts.
---
travis_time:end:063952c8:start=1545511237902351042,finish=1545511237907228779,duration=4877737
travis_fold:end:after_failure.3
travis_fold:start:after_failure.4
travis_time:start:1d97b95a
$ ln -s . checkout && for CORE in obj/cores/core.*; do EXE=$(echo $CORE | sed 's|obj/cores/core\.[0-9]*\.!checkout!\(.*\)|\1|;y|!|/|'); if [ -f "$EXE" ]; then printf travis_fold":start:crashlog\n\033[31;1m%s\033[0m\n" "$CORE"; gdb --batch -q -c "$CORE" "$EXE" -iex 'set auto-load off' -iex 'dir src/' -iex 'set sysroot .' -ex bt -ex q; echo travis_fold":"end:crashlog; fi; done || true
travis_fold:end:after_failure.4
travis_fold:start:after_failure.5
travis_time:start:290f2740
travis_time:start:290f2740
$ cat ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers || true
cat: ./obj/build/x86_64-unknown-linux-gnu/native/asan/build/lib/asan/clang_rt.asan-dynamic-i386.vers: No such file or directory
travis_fold:end:after_failure.5
travis_fold:start:after_failure.6
travis_time:start:223099d2
$ dmesg | grep -i kill

I'm a bot! I can only do what humans tell me to, so if this was not helpful or you have suggestions for improvements, please ping or otherwise contact @TimNN. (Feature Requests)

@djrenren
Copy link
Contributor

Hmmm the *Desc structs still seem to be tied to the libtest types. Currently a Desc contains a functional test but it seems like this effort would benefit from restructuring the Desc to be orthogonal to the underlying test. Otherwise it'll be hard to conveniently reuse the formatters with other test formats.


/// Test description.
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
pub struct Desc {
Copy link
Contributor Author

@gnzlbg gnzlbg Dec 22, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@djrenren

Hmmm the *Desc structs still seem to be tied to the libtest types. Currently a Desc contains a functional test

So not really. Like you can see here, the Desc struct only contains the test name, whether it is ignored, whether it should panic, or is it allowed to fail.

FWIW I think we might want a better API for this than a struct, e.g., some kind of trait, but I think this is not too bad of a start, and if we put this on a crate on crates.io, we can iterate on this using normal crate.io releases, instead of tying people to particular versions of rustc.

@djrenren
Copy link
Contributor

djrenren commented Dec 23, 2018 via email

@bors
Copy link
Contributor

bors commented Dec 24, 2018

☔ The latest upstream changes (presumably #57079) made this pull request unmergeable. Please resolve the merge conflicts.

@alexcrichton
Copy link
Member

Thanks for this @gnzlbg! I'm not entirely certain what to do about this though. I feel like this is a good direction to go in, and libtest is one of those pieces that seems like it would hugely benefit from living outside this repository and being developed externally. So, for example, let me paint a picture of where we might take this PR to the limit.

  • Let's assume we create a repository like rust-lang/libtest
  • This repository would all be 100% stable Rust and publishable on crates.io, organized however it would like
  • The rust-lang/rust repository would continue to have the "one and only" libtest, and libtest would just depend on these crates.io crates published from rust-lang/libtest
  • The libtest in rust-lang/rust would be a very thin shim which defines the actual interface that rustc expects. This way all the unstable code would live in this repository, and then the rust-lang/rust libtest would delegate to rust-lang/libtest when it's called.

That way, long term, we'd have a dedicated project to libtest and all the rust-lang/rust repository would contain is thin shims that translate between the stable crates.io interface and the unstable rustc interface.

How's that sound?

@gnzlbg
Copy link
Contributor Author

gnzlbg commented Jan 3, 2019

That's pretty much what I had in mind.

@alexcrichton
Copy link
Member

@gnzlbg and I discussed a bit on IRC and the conclusion was that @gnzlbg would open up an internal discussion to see if others have thoughts on this, and assuming others agree I think we can just delete most of the crate in this repository, move it externally, publish it, reintegrate here, and we should be good to go!

@gnzlbg
Copy link
Contributor Author

gnzlbg commented Jan 3, 2019

I've opened the discussion here: https://internals.rust-lang.org/t/a-path-forward-towards-re-usable-libtest-functionality-custom-test-frameworks-and-a-stable-bench-macro/9139

I think we can just delete most of the crate in this repository, move it externally, publish it, reintegrate here, and we should be good to go!

I'm closing this because I prefer to open a PR to do just that first.

@gnzlbg gnzlbg closed this Jan 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants