Tests that accompany code run and produce output at default #17

Closed
martinmoene opened this Issue May 31, 2016 · 2 comments

Projects

None yet

2 participants

@martinmoene
Contributor
martinmoene commented May 31, 2016 edited

In the case that unit tests accompany code, user may not expect, or like that at default

  • the tests are run,
  • the test run produces output (even) if all tests pass.

If tests take a negligible amount of time, running the tests at default may be fine, provided they produce no output on success.

If tests take a non-negligible amount of time, I can imagine people like to skip the tests at default.

Provided running the test at default is retained, it would be nice if the default non-running case could be achieved through a user-supplied main() in combination with a new command line option --run, or --unittest (the flag used by D).

int main(int argc, char** argv) {
    doctest::Context context(argc, argv); // initialize

    // overrides
    context.addFilter("no-run", !context.flag("run")); // control --no-run from --run
    ...
}

Note that currently configuring the context can only occur after processing the commandline options in Context's constructor, hence the comment mentioning overrides. The lack of being able to set defaults before commandline processing leads to the convoluted example above.

Supporting configuring defaults, e.g.:

int main(int argc, char** argv) {
    doctest::Context context;
    context.addFilter("no-run", true);  // default

    // --run (--unittest) on the commandline now overrides the --no-run default
    context(argc, argv);  // or context.apply(argc, argv);, or ...
    ...
}
@onqtam
Owner
onqtam commented Jun 1, 2016 edited

Indeed there should be an option to omit all output in the case of only success - this will be added.

Also it really is important for the user to be able to set defaults and not just overrides.

perhaps I should eliminate the --no-overrides option altogether? and allow the user to construct a Context without passing argc/argv, then to set some options and then to apply argc/argv by calling context.applyCommandLine(argc, argv);?

And in this way the user will still be able to set real and final overrides after the command line has been applied - again with setOption - but there will be no way to override them.

This seems like the cleanest solution - keeping the --no-overrides option will force the addition of things like setDefaultOption() which I don't want - and I'm almost sure removing the --no-overrides option is the right way to go.

What do you think about this?

@martinmoene
Contributor

Conforming to Catch's Amending the config as you describe above seems like a good idea.

@onqtam onqtam added a commit that referenced this issue Jun 2, 2016
@onqtam fixed #17 b946f27
@onqtam onqtam added a commit that closed this issue Sep 21, 2016
@onqtam fixed #17 7d5c0d5
@onqtam onqtam closed this in 7d5c0d5 Sep 21, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment