Pass stdout when running cargo test --verbose

[kornelski]

I wrote a new test and expected it to fail, but it unexpectedly passed. I've added println! to the code to see what happened, but the text was hidden by cargo.

I've tried cargo test --verbose, but it still had suppressed output I wanted to see. I had to make the test panic to see the output.

I prefer to always see all output, even if it means megabytes large dump to the terminal (my terminal handles that well). Could you make --verbose be really verbose or add another option (in ~/.cargo/config) to always print everything?

[alexcrichton]

This is somewhat of an issue for the test harness rather than Cargo itself, as Cargo just runs the test binary and it then figures out what to do with the output. It seems reasonable to me, though, that --verbose passed to the test harness would print this out by default.

[kornelski]

I'm not sure which project is responsible for the test harness. Should I report the bug in rust-lang/rust repository?

[matklad]

I had to make the test panic to see the output.

@pornel, I think you want cargo test -- --nocapture

It seems reasonable to me, though, that --verbose passed to the test harness would print this out by default.

Hm, the harness currently does not accepts --verbose option

[matklad]

@alexcrichton what do you thing about making cargo --verbose test to imply -- --nocapture?

EDIT: hm, looks like this will bind Cargo to the particular test harness, which is unfortunate.

[kornelski]

Uh-oh. I've found there's a catch - the tests are run in parallel, so order of the output is shuffled (unless I run it with RUST_TEST_THREADS=1).

nocapture is helpful, but not ideal. Maybe -vv could enable it? The optimal solution would be if test output was captured and then printed in order.

[durka]

The response to requests like this is always "we don't want to bind tightly to a specific test runner", which is a little silly because there's only one test runner currently. As it is, cargo knows that the user wants more output when they run cargo test --verbose, but it doesn't tell the test runner about that. The test runner is not able to respond to the user's wishes because it doesn't know about them. The -- --nocapture incantation is very undiscoverable.

What if cargo test --verbose passed on --verbose to the test runner, or set RUST_TEST_VERBOSE=1 in its environment, but took no official position on whether or how any specific test runner interprets that? Then, we could randomly pick one out of the one extant test runners and implement support for the argument/variable.

[alexcrichton]

@pornel yeah the harness currently lives in src/libtest in the rust-lang/rust repo, so having -v print more info (or imply --nocapture) would be more appropriate over there most likely.

@matklad I'd personally want to avoid --verbose implying --nocapture as that's pretty libtest-specific, as you've mentioned.

@durka this is basically what it looks like to err on the side of being conservative. This keeps the door open to alternate test harnesses in the future instead of forcing everyone to adhere to one set of wacky conventions that the first test harness just happened to set forth.

With custom test harnesses, however, it's likely that Cargo will have a contract with all implementations of some form, however. For example flags like --verbose and --color need to be communicated. We just need to figure out a standard method to do this, and try to write all this down as guidelines for custom test harnesses.

[durka]

Yeah, what I'm advocating for is defining this contract, instead of using
the prospect of alternate test harnesses to avoid defining one. I guess
that would need an RFC?

On Mon, Jun 20, 2016 at 12:32 PM, Alex Crichton notifications@github.com
wrote:

@pornel https://github.com/pornel yeah the harness currently lives in
src/libtest in the rust-lang/rust repo, so having -v print more info (or
imply --nocapture) would be more appropriate over there most likely.

@matklad https://github.com/matklad I'd personally want to avoid
--verbose implying --nocapture as that's pretty libtest-specific, as
you've mentioned.

@durka https://github.com/durka this is basically what it looks like to
err on the side of being conservative. This keeps the door open to
alternate test harnesses in the future instead of forcing everyone to
adhere to one set of wacky conventions that the first test harness just
happened to set forth.

With custom test harnesses, however, it's likely that Cargo will have a
contract with all implementations of some form, however. For example flags
like --verbose and --color need to be communicated. We just need to
figure out a standard method to do this, and try to write all this down as
guidelines for custom test harnesses.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#2792 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AAC3n3HzWR90vN6IoI6IKL-9MDn-JXAlks5qNsCggaJpZM4I5QFK
.

[alexcrichton]

It would probably be best to design much of custom test harnesses all at once in one RFC rather than piecemeal, and then this would interact with issues like #2790

[carols10cents]

Closing since this sounds like this issue should either be:

• made in rust-lang/rust

  yeah the harness currently lives in src/libtest in the rust-lang/rust repo, so having -v print more info (or imply --nocapture) would be more appropriate over there most likely.

• not done at all

  I'd personally want to avoid --verbose implying --nocapture as that's pretty libtest-specific, as you've mentioned.

• require an RFC as part of designing extensible test harness support

  With custom test harnesses, however, it's likely that Cargo will have a contract with all implementations of some form, however. For example flags like --verbose and --color need to be communicated. We just need to figure out a standard method to do this, and try to write all this down as guidelines for custom test harnesses.