Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Provide low verbose to li3 test for skipped tests #481

Open
wants to merge 2 commits into from

3 participants

@hans-d

li3 test now only shows that tests have been skipped, and only with verbose you can identify them. But verbose is very verbose. With this flag, test shows the normal test output followed by the skipped tests.

@davidpersson

Not sure about this, also if than we want the option to be named --show-skipped and mapped to $showSkipped. However currently such a mapping isn't possible due to missing functionality in the command construction.

As an alternative, we could provide values to verbose, eg high for the current verbose and skipped for my variation.
With no value it would default to high...

@davidpersson davidpersson was assigned
@nateabele
Owner

Yeah, I like --verbose=skipped.

@hans-d hans-d --showskipped renamed to --verbose=skipped
--verbose alone wil still function as previously (highly detailed).
70623cb
@davidpersson

Name this method verbosity()?
Also: why do we need an extra method?

@davidpersson

Having a string as the option to verbose is a little uncommon. If you look at other *nix commands they either take an integer or something like this -v [ --verbose ] be more verbose (include multiple times for more verbosity e.g. -vvvvv) (mongod).

@davidpersson

Leaving out the skipped test in normal mode is intended.

This makes sense because outputting a bunch of skipped tests as the last thing will cause you to focus the skipped tests (as they are the last thing output) instead of keeping your focus on the result (PASS/FAIL). So this is *good and the behavior should be preserved.

While the fact that there have been skipped tests (number of assertions would normally be bigger) is important what tests have been skipped and why is detailed information. Detailed information is what you want in verbose mode.

One should strive to have as little output modes as possible.

This really complicates things. And when you think further adding another output mode is possibly just a symptom for one of the existing modes not being sufficient. So when do you need to see which tests have been skipped? Does this really warrant a new output mode if there's already one giving you the skipped tests plus a bit more?

@davidpersson

...another name which comes to my mind is --skipped.

@nateabele
Owner

I know it's somewhat non-standard, but I think it makes sense in this case, and I like that you can explicitly specify which things you want verbosity about, rather than arbitrary numeric levels, which I've never liked, and really don't make sense in this case.

For maximum flexibility, we could even allow multiple comma-separated values like we do for --filters. Example: if stack trace output was disabled by default, then we could allow something like --verbose=skipped,traces.

@hans-d
@hans-d

So, how should I proceed? Keep it as a value for verbose?

@davidpersson

Give me some time I'll cook something up.

@nateabele
Owner

@davidpersson Do you know when you'll have an update on this?

@nateabele
Owner

@hans-d Okay, let's do this: we'll call the flag --verbose, and allow it to accept a comma-separated value, which can be any one of passed,failed,skipped,trace, which will optionally show passed, failed, or skipped tests, as well stack traces (we can disable stack traces by default).

Please be sure to update the documentation and the tests accordingly. Thanks!

@hans-d

I will put this on my todo list

@nateabele
Owner

Hey @hans-d, sorry this got sidelined forever. :-/ If you want to re-submit it to the dev branch, I'll merge it, and we can worry about enhancing it later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on May 24, 2012
  1. @hans-d

    Add --showskipped to li3 test

    hans-d authored
  2. @hans-d

    --showskipped renamed to --verbose=skipped

    hans-d authored
    --verbose alone wil still function as previously (highly detailed).
This page is out of date. Refresh to see the latest.
View
22 console/command/Test.php
@@ -52,12 +52,14 @@ class Test extends \lithium\console\Command {
public $format = 'txt';
/**
- * Enable verbose output especially for the `txt` format.
+ * Enable verbose output especially for the `txt` format. Options are `high`
+ * (default if none given) for full details and `skipped` for only information
+ * on the skipped tests.
*
- * @var boolean
+ * @var string
*/
public $verbose = false;
-
+
/**
* Enable plain mode to prevent any headers or similar decoration being output.
* Good for command calls embedded into other scripts.
@@ -73,6 +75,16 @@ class Test extends \lithium\console\Command {
* @var array
*/
protected $_handlers = array();
+
+ public function verboseLevel() {
+ if ($this->verbose === false) {
+ return false;
+ }
+ if ($this->verbose === 'skipped') {
+ return 'skipped';
+ }
+ return 'high';
+ }
/**
* Initializes the output handlers.
@@ -110,7 +122,7 @@ protected function _init() {
}
};
- if ($command->verbose) {
+ if ($command->verboseLevel() == 'high') {
$reporter = function($result) use ($command, $colorize) {
$command->out(sprintf(
'[%s] on line %4s in %s::%s()',
@@ -151,7 +163,7 @@ protected function _init() {
$command->out($report->render('result', $stats));
$command->out($report->render('errors', $stats));
- if ($command->verbose) {
+ if ($command->verboseLevel()) {
$command->out($report->render('skips', $stats));
}
View
2  tests/cases/console/command/HelpTest.php
@@ -54,7 +54,7 @@ public function testRunWithName() {
$this->assertTrue($result);
$expected = 'li3 test [--filters=<string>]';
- $expected .= ' [--format=<string>] [--verbose] [--plain] [<path>]';
+ $expected .= ' [--format=<string>] [--verbose=<string>] [--plain] [<path>]';
$expected = preg_quote($expected);
$result = $command->response->output;
$this->assertPattern("/{$expected}/", $result);
Something went wrong with that request. Please try again.