test results of two methods in one test case may get mixed up

[paranoiq]

class SomeTest extends TestCase
{
    public function testAaa()
    {
        Assert::same(2, 2);
    }

    public function testBbb()
    {
        Assert::same(3, []);
    }
}

both fail with error array() should be 3 in testBbb()

only when runned with test runner, not directly

the reason is, that both test methods produce expected and actual files with the same name

[dg]

How do you run TestCase?

[fprochazka]

only when runned with test runner

He probably wasn't accepting the test method cli argument so the test case had been ran twice (because there are two methods) and in both cases, the test case ran both methods, so every time it ended with failure.

(new SomeTest)->run();

vs

(new SomeTest)->run(isset($_SERVER['argv'][1]) ? $_SERVER['argv'][1] : NULL);

[paranoiq]

@dg this way:

$testCase->run(isset($_SERVER['argv'][1]) ? $_SERVER['argv'][1] : NULL);

but there could be something on the "test case ran both methods" thing, because i saw some testing data leaking from one method to another. and they should be isolated. i'll check it again

[milo]

@paranoiq I guess this happend when test was annoted by @testcase and you called $testCase->run() (or $_SERVER['argv'] was not set).

With current master, I cannot reproduce it by any combination. Could you try it again?

[paranoiq]

ok, i am obviously an idiot. when i tried it again i finally find out what the problem was. i have disabled register_argc_argv directive in my testing php.ini

[milo]

@paranoiq It has been fixed. Moreover, now you can forgot on it and run TestCase just by $testCase->run(). Tester handles it internally.