Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow crate to attach uniform initialization behavior to every #[test] #1664

Open
pnkfelix opened this issue Jul 2, 2016 · 12 comments
Open

Allow crate to attach uniform initialization behavior to every #[test] #1664

pnkfelix opened this issue Jul 2, 2016 · 12 comments
Labels

Comments

@pnkfelix
Copy link
Member

@pnkfelix pnkfelix commented Jul 2, 2016

Back when support for logging macros like debug! and what not was more heavily baked into the language and stdlib, I think we were able to see output from such logging macros from the #[test] unit tests.

But now, one cannot, at least not without adding explicit code to initialize the logging subsystem to every unit tests.

See also this StackOverflow Q here: http://stackoverflow.com/questions/30177845/how-to-initialize-the-logger-for-integration-tests

It is a shame that we offer both of these features (integrated unit tests and a logging subsystem), and yet they do not compose as seamlessly as one might hope.

@pnkfelix pnkfelix added the T-libs label Jul 2, 2016
@pnkfelix
Copy link
Member Author

@pnkfelix pnkfelix commented Jul 2, 2016

(as a quick counterpoint: this is relatively low priority, since when one is debugging the behavior on a particular test, it is usually a very small cost to add the single line of env_logger initialization code to that test and recompile. That is, usually tests are cheap to recompile. the exception is when the tests are embedded as unit tests inline within the source code file, because then that usually implies recompiling the whole original source crate, at least for the time being, though presumably incremental compilation will eventually make that "free" too.)

@durka
Copy link
Contributor

@durka durka commented Jul 5, 2016

People ask all the time for setup and teardown facilities in the test harness. You can hack it in with a macro but it's not the best and doesn't work for doctests as you said.

@NoraCodes
Copy link

@NoraCodes NoraCodes commented Apr 6, 2018

Just coming to chime in on the utility of this. I'm working on two projects, one of which uses a database I'd like to be able to set up and tear down without manually running commands, and the other I just committed about 25 times to solve an issue that occurred only in my CI. If I could have initialized logging that would have been solved.

@SimonSapin
Copy link
Contributor

@SimonSapin SimonSapin commented Oct 7, 2018

In servo/servo#21884 I’ve exploited a loophole in how rustc --test interacts with Cargo in order to override main and run some initialization code before running the test harness. It’d be nice to have a supported way to do this.

I’m aware of rust-lang/rust#50297, but besides it not being implemented I’d prefer to have to write an entire test harness. I like #[test].

CC @rust-lang/libs

@SimonSapin
Copy link
Contributor

@SimonSapin SimonSapin commented Oct 7, 2018

This issue’s title says “to every #[test]”, but the issue description suggests something that only runs once per process in order to e.g. initialize static items. The latter is what I’ve implemented for Servo.

@NoraCodes
Copy link

@NoraCodes NoraCodes commented Oct 7, 2018

This issue’s title says “to every #[test]”, but the issue description suggests something that only runs once per process in order to e.g. initialize static items. The latter is what I’ve implemented for Servo.

Really, the two are related but not identical problems. For my use case, I actually need both.

  • Code running before all tests (that is, prior to any test running, once), to ensure the environment is correctly set up. No tests should run if the database is unavailable.
  • Code running after each test, each time, to clean up the database in case of failure.
  • Code running after all tests are finished to nuke the test database completely.
@SimonSapin
Copy link
Contributor

@SimonSapin SimonSapin commented Oct 7, 2018

after each test

(You’re probably aware of this, but in that case you might need to force tests to run on a single thread as that’s not the default. Otherwise a test test might still be running at the time another finishes.)

@NoraCodes
Copy link

@NoraCodes NoraCodes commented Oct 7, 2018

after each test

(You’re probably aware of this, but in that case you might need to force tests to run on a single thread as that’s not the default. Otherwise a test test might still be running at the time another finishes.)

Yes, that's necessary but unfortunately insufficient.

My current solution is build the test environment, get the data I want to check, perform cleanup, and then assert! my conditions. This is ugly and verbose, however.

@alexcrichton
Copy link
Member

@alexcrichton alexcrichton commented Oct 8, 2018

@SimonSapin nowadays I think it's also a viable option to have something like:

#[my_test_macro]
fn my_test(with: &MyArgs, and: &GlobalResourceInitializedOnce) {
    // ...
}

Would that work for Servo?

@vertexclique
Copy link
Member

@vertexclique vertexclique commented May 21, 2019

This is still an issue. I agree with what @alexcrichton suggested but slightly different use might be better.

#[my_test_macro]
fn my_test(with: &MyArgs, and: &Box<dyn BeforeAndAfterAll>) {
    // ...
}

Where multiple traits make sense like BeforeAndAfterEach etc. so a trait method might be used later for scheduling the callbacks.

@rjloura
Copy link

@rjloura rjloura commented Oct 23, 2019

This issue’s title says “to every #[test]”, but the issue description suggests something that only runs once per process in order to e.g. initialize static items. The latter is what I’ve implemented for Servo.

This would be ideal for my use case.

setup()
test1()
test2()
test3()
teardown()

This would allow all tests to be run in parallel instead of having to run with --test-threads=1

Is this still being considered under this RFC?

@shepmaster
Copy link
Member

@shepmaster shepmaster commented Oct 23, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants