New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow crate to attach uniform initialization behavior to every `#[test]` #1664

Open
pnkfelix opened this Issue Jul 2, 2016 · 9 comments

Comments

Projects
None yet
5 participants
@pnkfelix
Member

pnkfelix commented Jul 2, 2016

Back when support for logging macros like debug! and what not was more heavily baked into the language and stdlib, I think we were able to see output from such logging macros from the #[test] unit tests.

But now, one cannot, at least not without adding explicit code to initialize the logging subsystem to every unit tests.

See also this StackOverflow Q here: http://stackoverflow.com/questions/30177845/how-to-initialize-the-logger-for-integration-tests

It is a shame that we offer both of these features (integrated unit tests and a logging subsystem), and yet they do not compose as seamlessly as one might hope.

@pnkfelix pnkfelix added the T-libs label Jul 2, 2016

@pnkfelix

This comment has been minimized.

Member

pnkfelix commented Jul 2, 2016

(as a quick counterpoint: this is relatively low priority, since when one is debugging the behavior on a particular test, it is usually a very small cost to add the single line of env_logger initialization code to that test and recompile. That is, usually tests are cheap to recompile. the exception is when the tests are embedded as unit tests inline within the source code file, because then that usually implies recompiling the whole original source crate, at least for the time being, though presumably incremental compilation will eventually make that "free" too.)

@durka

This comment has been minimized.

Contributor

durka commented Jul 5, 2016

People ask all the time for setup and teardown facilities in the test harness. You can hack it in with a macro but it's not the best and doesn't work for doctests as you said.

@LeoTindall

This comment has been minimized.

LeoTindall commented Apr 6, 2018

Just coming to chime in on the utility of this. I'm working on two projects, one of which uses a database I'd like to be able to set up and tear down without manually running commands, and the other I just committed about 25 times to solve an issue that occurred only in my CI. If I could have initialized logging that would have been solved.

@SimonSapin

This comment has been minimized.

Contributor

SimonSapin commented Oct 7, 2018

In servo/servo#21884 I’ve exploited a loophole in how rustc --test interacts with Cargo in order to override main and run some initialization code before running the test harness. It’d be nice to have a supported way to do this.

I’m aware of rust-lang/rust#50297, but besides it not being implemented I’d prefer to have to write an entire test harness. I like #[test].

CC @rust-lang/libs

@SimonSapin

This comment has been minimized.

Contributor

SimonSapin commented Oct 7, 2018

This issue’s title says “to every #[test]”, but the issue description suggests something that only runs once per process in order to e.g. initialize static items. The latter is what I’ve implemented for Servo.

@LeoTindall

This comment has been minimized.

LeoTindall commented Oct 7, 2018

This issue’s title says “to every #[test]”, but the issue description suggests something that only runs once per process in order to e.g. initialize static items. The latter is what I’ve implemented for Servo.

Really, the two are related but not identical problems. For my use case, I actually need both.

  • Code running before all tests (that is, prior to any test running, once), to ensure the environment is correctly set up. No tests should run if the database is unavailable.
  • Code running after each test, each time, to clean up the database in case of failure.
  • Code running after all tests are finished to nuke the test database completely.
@SimonSapin

This comment has been minimized.

Contributor

SimonSapin commented Oct 7, 2018

after each test

(You’re probably aware of this, but in that case you might need to force tests to run on a single thread as that’s not the default. Otherwise a test test might still be running at the time another finishes.)

@LeoTindall

This comment has been minimized.

LeoTindall commented Oct 7, 2018

after each test

(You’re probably aware of this, but in that case you might need to force tests to run on a single thread as that’s not the default. Otherwise a test test might still be running at the time another finishes.)

Yes, that's necessary but unfortunately insufficient.

My current solution is build the test environment, get the data I want to check, perform cleanup, and then assert! my conditions. This is ugly and verbose, however.

@alexcrichton

This comment has been minimized.

Member

alexcrichton commented Oct 8, 2018

@SimonSapin nowadays I think it's also a viable option to have something like:

#[my_test_macro]
fn my_test(with: &MyArgs, and: &GlobalResourceInitializedOnce) {
    // ...
}

Would that work for Servo?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment