Test.php suffix convention

[adam-lynch]

All tests filenames must end with Test.php (as is enforced in the SuiteLoader).

I could be wrong, but should this always be done? Should it be done only if a .xml.dist file isn't given?

[dbaltas]

Filenames ending with Test.php is a strong PHPUnit convention.
Even running PHPUnit suite standalone would ignore php files not ending with this suffix when specifying a folder.

You could write a patch by setting $relaxTestPattern to true in the call of tryLoadTests
$this->tryLoadTests($path . DIRECTORY_SEPARATOR . $file, true);
https://github.com/brianium/paratest/blob/master/src/ParaTest/Runners/PHPUnit/SuiteLoader.php#L61

Alternatively, this option, relaxTestPattern could be configurable in the command line options, but I am not sure if Brian would like to go down this path.

Is there a good reason to not have your tests filenames ending with ...Test.php?

[brianium]

I am of the same opinion as Dimitris. Unless there is overwhelming demand for this, I think we should leave it as is. The configuration via command line would be an acceptable solution if it were necessary. Let's close this for now and revisit it if we see some demand for it.

[adam-lynch]

Ok, no problem. But just let me comment on a couple of things.

Is there a good reason to not have your tests filenames ending with ...Test.php?

It's not a massive problem for us since we're just beginning to write the (Selenium) test files we intend to parallelize, but it just breaks our convention of having our test directories mirror our source directories in structure and filenames (which we have for our set of existing plain PHPUnit tests).

Filenames ending with Test.php is a strong PHPUnit convention.
Even running PHPUnit suite standalone would ignore php files not ending with this suffix when specifying a folder.

I just noticed this because in our .xml.dist we have specified:

<testsuites>
    <testsuite name="Blah project">
        <directory suffix='.php'>blah/blah</directory>
        <directory suffix='.php'>blah2/blah</directory>
        <exclude>blah/blah/blah</exclude>
    </testsuite>
</testsuites>

Which is fine when ran regularly with PHPUnit but via Paratest, the suffix we have specified is ignored. So, I guess it's blocking a PHPUnit configuration feature. It's not a big problem for us but might be for someone with a large set of existing tests they want to parallelize.

[brianium]

The configuration aspect of PHPUnit is more difficult to cover with Paratest. I think it would be awesome if we could support more of it. I started the work that supports the limited configuration here https://github.com/brianium/paratest/blob/master/src/ParaTest/Runners/PHPUnit/Configuration.php. Maybe this context is something we could begin to support?

[adam-lynch]

Yeah I guessed that, since you'll basically be doing work PHPUnit will do anyway when it reads the dist itself.

Unless there's anyway to leverage the internal classes of PHPUnit itself?

[adam-lynch]

This is really a frustrating issue. I got bitten by it just now again.

[brianium]

I am thinking on the best way to support this. The easy way is to allow suffix patterns to be specified via the cli. It might be worth looking into a better way to handle configuration concerns in general. It doesn't seem like it would be too difficult to make the suite loader aware of directories and their provided suffixes. The SuiteLoader already has access to the configuration that is parsing those suites. Maybe we can parse those suite nodes into more complete objects (instead of a dictionary of name => path pairs).

[brianium]

https://github.com/brianium/paratest/blob/master/src/ParaTest/Runners/PHPUnit/SuiteLoader.php#L36
https://github.com/brianium/paratest/blob/master/src/ParaTest/Runners/PHPUnit/Configuration.php#L34

These two points are our opportunity. I will look into creating a more complete Configuration\Suite object.

[giorgiosironi]

I agree with @adam-lynch in expecting a phpunit.xml[.dist] suffix attributes to work. If one of the goals of the project is transparency, it shouldn't be necessary to add new syntax (command line options) for features already supported by PHPUnit.

[dbaltas]

True that.
Parsing PHPUnit configuration would cause less confusion.
I am sure other people like Adam would expect a configuration set there simply to work.

When we get to the point where paratest supports other testing frameworks,
we can add this option in paratest. It would default to PHPUnit config when set.
Similar to what we did with the bootstrap file #34.

[rusitschka]

I'd also highly appreciate the PHPUnit configuration approach. Here's an example from one of our test suites:

    <testsuites>
        <testsuite name="all">
            <directory suffix="Test.php">test/application</directory>
            <directory suffix="SharedTest.php">../mzlib/test/application</directory>
        </testsuite>
    </testsuites>

The paths are interpreted correctly but the suffixes are currently ignored by paratest.

[julianseeger]

Thanks to @rusitschka this is finally implemented.