Make the "XML Configuration File" win against command line options

[daniela-waranie]

Some IDEs have some command line switches hardcoded to support phpunit (e.g. netbeans). The only way to configure phpunit behavior is to provide a "XML Configuration File" - one big issue is that command line switches wins against xml options.

E.g.: netbeans runs phpunit by using the command line switch "--log-junit". At the moment there is no way in netbeans to add the "--verbose" switch. In the xml configuration file there is no separate xml element to make phpunit to be verbose. The only way is to use this:
<logging> <log type="junit" target="/tmp/logfile.xml" logIncompleteSkipped="true"/> </logging>
But this xml element is igored by phpunit when "--log-junit" command line switch is given.

Solution:
a) add an independent xml element (verbose) to supplement the "--log-junit" command line switch
b) add a "force" attribute to each xml element to determ if a xml option should override a command line switch (to invert default override behavior)

To me solution a would be better, because i don´t know at what path netbeans expects the result xml. Maybe you implement both to make phpunit more flexible.

Yes indeed a change to the code of netbeans can solve this issue, too. But we all know how long to wait to have a feature request implemented in closed souce software... And i think this is not an issue specific to netbeans, there are probably other programs out there calling phpunit by hardcoded command line switches.

[whatthejeff]

Netbeans is actually open source and very active. You can submit issues here: http://netbeans.org/community/issues.html.

With that said, I really think this should be submitted as a netbeans feature request. Currently netbeans does not support the 'logIncompleteSkipped' option. Even if you could accomplish what you're trying to do, the netbeans test parsers are not configured for information about incomplete or skipped tests so you still wouldn't have the results you're looking for.

[daniela-waranie]

The issue is known by the netbeans team since 2009-11-09, see http://netbeans.org/bugzilla/show_bug.cgi?id=176157 for details. Last post is from 2010-07-26, nothing happens.

To have different outputs (command line output / xml) is not consistent. The command line shows the skipped and incompleted tests but corresponding xml elements are missing, if you use "--log-junit".

If you think the missing "--verbose" should lead into consealing the I and S in the result xml, then it should be missing in the command line output, too. If you think I and S should be in the command line output even on missing "--verbose", then it should be in the result xml, too.

Maybe the "logIncompleteSkipped" is not 100% the same as "--verbose" switch. If so, a option to turn on the "verbose-mode" via configuration xml could help, too.

[dinomite]

All command line programs I can think of do just the opposite—options passed on the command line always win. Having different behavior sounds dangerously confusing.

[daniela-waranie]

@dinomite: i agree, the default behavior should not be changed. Command line switches should win against xml configuration file at default.

Due to the fact, that incomplete and skipped tests can not be printed to a "junit test result xml", because the junit xml only knows errors and failures it makes no sense to have a 'logIncompleteSkipped="true"' in the xml when having a 'type="junit"'. The command line should print a warning that 'logIncompleteSkipped="true"' makes no sense for junit xml output.

I am not sure if "logIncompleteSkipped" makes sense for type=json or tap. If it makes no sense we should disallow it as a attribute to the <log> element anymore.

In case of type=junit the logIncompleteSkipped only have effect on the command line output, so i want to request this "command line output setting" out of the <logging> element to a more general element as e.g.
<commandline> <printIncomplete>true</printIncomplete> <printSkipped>true</printSkipped> <verbose>true</verbose> </commandline>.

[daniela-waranie]

Would be nice if you introduce a warning message (see last comment)