Provide low verbose to li3 test for skipped tests #481

Open
wants to merge 2 commits into
from

3 participants

@hans-d

li3 test now only shows that tests have been skipped, and only with verbose you can identify them. But verbose is very verbose. With this flag, test shows the normal test output followed by the skipped tests.

@davidpersson
Union of RAD member

Not sure about this, also if than we want the option to be named --show-skipped and mapped to $showSkipped. However currently such a mapping isn't possible due to missing functionality in the command construction.

As an alternative, we could provide values to verbose, eg high for the current verbose and skipped for my variation.
With no value it would default to high...

@davidpersson davidpersson was assigned May 24, 2012
@nateabele
Union of RAD member

Yeah, I like --verbose=skipped.

@hans-d hans-d --showskipped renamed to --verbose=skipped
--verbose alone wil still function as previously (highly detailed).
70623cb
@davidpersson
Union of RAD member

Name this method verbosity()?
Also: why do we need an extra method?

@davidpersson
Union of RAD member

Having a string as the option to verbose is a little uncommon. If you look at other *nix commands they either take an integer or something like this -v [ --verbose ] be more verbose (include multiple times for more verbosity e.g. -vvvvv) (mongod).

@davidpersson
Union of RAD member

Leaving out the skipped test in normal mode is intended.

This makes sense because outputting a bunch of skipped tests as the last thing will cause you to focus the skipped tests (as they are the last thing output) instead of keeping your focus on the result (PASS/FAIL). So this is *good and the behavior should be preserved.

While the fact that there have been skipped tests (number of assertions would normally be bigger) is important what tests have been skipped and why is detailed information. Detailed information is what you want in verbose mode.

One should strive to have as little output modes as possible.

This really complicates things. And when you think further adding another output mode is possibly just a symptom for one of the existing modes not being sufficient. So when do you need to see which tests have been skipped? Does this really warrant a new output mode if there's already one giving you the skipped tests plus a bit more?

@davidpersson
Union of RAD member

...another name which comes to my mind is --skipped.

@nateabele
Union of RAD member

I know it's somewhat non-standard, but I think it makes sense in this case, and I like that you can explicitly specify which things you want verbosity about, rather than arbitrary numeric levels, which I've never liked, and really don't make sense in this case.

For maximum flexibility, we could even allow multiple comma-separated values like we do for --filters. Example: if stack trace output was disabled by default, then we could allow something like --verbose=skipped,traces.

@hans-d
@hans-d

So, how should I proceed? Keep it as a value for verbose?

@davidpersson
Union of RAD member

Give me some time I'll cook something up.

@nateabele
Union of RAD member

@davidpersson Do you know when you'll have an update on this?

@nateabele
Union of RAD member

@hans-d Okay, let's do this: we'll call the flag --verbose, and allow it to accept a comma-separated value, which can be any one of passed,failed,skipped,trace, which will optionally show passed, failed, or skipped tests, as well stack traces (we can disable stack traces by default).

Please be sure to update the documentation and the tests accordingly. Thanks!

@hans-d

I will put this on my todo list

@nateabele
Union of RAD member

Hey @hans-d, sorry this got sidelined forever. :-/ If you want to re-submit it to the dev branch, I'll merge it, and we can worry about enhancing it later.

@davidpersson davidpersson removed their assignment May 23, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment