Skip to content
This repository

Provide low verbose to li3 test for skipped tests #481

Open
wants to merge 2 commits into from

3 participants

Hans Donner Nate Abele David Persson
Hans Donner
hans-d commented

li3 test now only shows that tests have been skipped, and only with verbose you can identify them. But verbose is very verbose. With this flag, test shows the normal test output followed by the skipped tests.

David Persson
Owner

Not sure about this, also if than we want the option to be named --show-skipped and mapped to $showSkipped. However currently such a mapping isn't possible due to missing functionality in the command construction.

As an alternative, we could provide values to verbose, eg high for the current verbose and skipped for my variation.
With no value it would default to high...

Nate Abele
Owner

Yeah, I like --verbose=skipped.

Hans Donner hans-d --showskipped renamed to --verbose=skipped
--verbose alone wil still function as previously (highly detailed).
70623cb
David Persson

Name this method verbosity()?
Also: why do we need an extra method?

David Persson
Owner

Having a string as the option to verbose is a little uncommon. If you look at other *nix commands they either take an integer or something like this -v [ --verbose ] be more verbose (include multiple times for more verbosity e.g. -vvvvv) (mongod).

David Persson
Owner

Leaving out the skipped test in normal mode is intended.

This makes sense because outputting a bunch of skipped tests as the last thing will cause you to focus the skipped tests (as they are the last thing output) instead of keeping your focus on the result (PASS/FAIL). So this is **good* and the behavior should be preserved.

While the fact that there have been skipped tests (number of assertions would normally be bigger) is important what tests have been skipped and why is detailed information. Detailed information is what you want in verbose mode.

One should strive to have as little output modes as possible.

This really complicates things. And when you think further adding another output mode is possibly just a symptom for one of the existing modes not being sufficient. So when do you need to see which tests have been skipped? Does this really warrant a new output mode if there's already one giving you the skipped tests plus a bit more?

David Persson
Owner

...another name which comes to my mind is --skipped.

Nate Abele
Owner

I know it's somewhat non-standard, but I think it makes sense in this case, and I like that you can explicitly specify which things you want verbosity about, rather than arbitrary numeric levels, which I've never liked, and really don't make sense in this case.

For maximum flexibility, we could even allow multiple comma-separated values like we do for --filters. Example: if stack trace output was disabled by default, then we could allow something like --verbose=skipped,traces.

Hans Donner
hans-d commented
Hans Donner
hans-d commented

So, how should I proceed? Keep it as a value for verbose?

David Persson
Owner

Give me some time I'll cook something up.

Nate Abele
Owner

@davidpersson Do you know when you'll have an update on this?

Nate Abele
Owner

@hans-d Okay, let's do this: we'll call the flag --verbose, and allow it to accept a comma-separated value, which can be any one of passed,failed,skipped,trace, which will optionally show passed, failed, or skipped tests, as well stack traces (we can disable stack traces by default).

Please be sure to update the documentation and the tests accordingly. Thanks!

Hans Donner
hans-d commented

I will put this on my todo list

Nate Abele
Owner

Hey @hans-d, sorry this got sidelined forever. :-/ If you want to re-submit it to the dev branch, I'll merge it, and we can worry about enhancing it later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Showing 2 unique commits by 1 author.

May 24, 2012
Hans Donner hans-d Add --showskipped to li3 test 8cdf551
Hans Donner hans-d --showskipped renamed to --verbose=skipped
--verbose alone wil still function as previously (highly detailed).
70623cb
This page is out of date. Refresh to see the latest.
22 console/command/Test.php
@@ -52,12 +52,14 @@ class Test extends \lithium\console\Command {
52 52 public $format = 'txt';
53 53
54 54 /**
55   - * Enable verbose output especially for the `txt` format.
  55 + * Enable verbose output especially for the `txt` format. Options are `high`
  56 + * (default if none given) for full details and `skipped` for only information
  57 + * on the skipped tests.
56 58 *
57   - * @var boolean
  59 + * @var string
58 60 */
59 61 public $verbose = false;
60   -
  62 +
61 63 /**
62 64 * Enable plain mode to prevent any headers or similar decoration being output.
63 65 * Good for command calls embedded into other scripts.
@@ -73,6 +75,16 @@ class Test extends \lithium\console\Command {
73 75 * @var array
74 76 */
75 77 protected $_handlers = array();
  78 +
  79 + public function verboseLevel() {
  80 + if ($this->verbose === false) {
  81 + return false;
  82 + }
  83 + if ($this->verbose === 'skipped') {
  84 + return 'skipped';
  85 + }
  86 + return 'high';
  87 + }
76 88
77 89 /**
78 90 * Initializes the output handlers.
@@ -110,7 +122,7 @@ protected function _init() {
110 122 }
111 123 };
112 124
113   - if ($command->verbose) {
  125 + if ($command->verboseLevel() == 'high') {
114 126 $reporter = function($result) use ($command, $colorize) {
115 127 $command->out(sprintf(
116 128 '[%s] on line %4s in %s::%s()',
@@ -151,7 +163,7 @@ protected function _init() {
151 163 $command->out($report->render('result', $stats));
152 164 $command->out($report->render('errors', $stats));
153 165
154   - if ($command->verbose) {
  166 + if ($command->verboseLevel()) {
155 167 $command->out($report->render('skips', $stats));
156 168 }
157 169
2  tests/cases/console/command/HelpTest.php
@@ -54,7 +54,7 @@ public function testRunWithName() {
54 54 $this->assertTrue($result);
55 55
56 56 $expected = 'li3 test [--filters=<string>]';
57   - $expected .= ' [--format=<string>] [--verbose] [--plain] [<path>]';
  57 + $expected .= ' [--format=<string>] [--verbose=<string>] [--plain] [<path>]';
58 58 $expected = preg_quote($expected);
59 59 $result = $command->response->output;
60 60 $this->assertPattern("/{$expected}/", $result);

Tip: You can add notes to lines in a file. Hover to the left of a line to make a note

Something went wrong with that request. Please try again.