The difference between Mockery and Phony

[enumag]

I'm not posting this to complain. It's just that I got the feeling like I'm missing some better way to test things or a point how Phony is intended to be used. I'd really appriciate some feedback what I'm doing wrong and what would you have done otherwise.

Let's have some code to test:

interface FooInterface
{
    public function bar();
}

class Strict
{
    public function call(FooInterface $foo)
    {
        $value = $foo->bar();
        if (!$value) {
            throw new RuntimeException();
        }
        return $value;
    }
}

---

Now let's test the simple case where no exception is thrown. With Mockery it's pretty straightforward:

    public function testStrictWithMockery()
    {
        $foo = Mockery::mock(FooInterface::class);
        $foo
            ->shouldReceive('bar')
            ->once()
            ->withNoArgs()
            ->andReturn('value');

        $strict = new Strict();
        self::assertSame('value', $strict->call($foo));
    }

Then I've tried to rewrite the same test with Phony without getting rid of any assertions and here is what I got:

    public function testStrictWithPhony()
    {
        $fooHandle = Phony::mock(FooInterface::class);
        $fooHandle
            ->bar
            ->with()
            ->returns('value');

        $strict = new Strict();
        self::assertSame('value', $strict->call($fooHandle->get()));

        $fooHandle
            ->bar
            ->once()
            ->called();

        $fooHandle
            ->bar
            ->calledWith();
    }

As you can see it's way longer which raised an exclamation mark in my head that I might be doing something wrong here. I'm unable to tell what though. But let's continue.

---

Next I need to test the case that raises the exception. Let's start with Mockery again:

    /**
     * @expectedException RuntimeException
     */
    public function testStrictExceptionWithMockery()
    {
        $foo = Mockery::mock(FooInterface::class);
        $foo
            ->shouldReceive('bar')
            ->withNoArgs()
            ->once()
            ->andReturn(null);

        $strict = new Strict();
        $strict->call($foo);
    }

Easy enough. Now again the same test with Phony:

    /**
     * @expectedException RuntimeException
     */
    public function testStrictExceptionWithPhony()
    {
        $fooHandle = Phony::mock(FooInterface::class);
        $fooHandle
            ->bar
            ->returns(null);

        $strict = new Strict();
        try {
            $strict->call($fooHandle->get());
        } catch (RuntimeException $e) {
        } finally {
            $fooHandle
                ->bar
                ->once()
                ->called();

            $fooHandle
                ->bar
                ->calledWith();
        }

        throw $e;
    }

... Yep I'm definitely doing something wrong here. Could you point me in the right direction?

[ezzatron]

Good question, and one that kind of cuts to the heart of the "philosophy" difference between the two frameworks. I'll write a proper response soon, but I'll at least give you some examples of what I would do.

I would test the non-error case like so:

    public function testStrictWithPhony()
    {
        $fooHandle = Phony::mock(FooInterface::class);
        $fooHandle->bar->returns('value');
        $strict = new Strict();

        $this->assertSame('value', $strict->call($fooHandle->get()));
    }

And the error case like this:

    public function testStrictExceptionWithPhony()
    {
        $foo = Phony::mock(FooInterface::class)->get();
        $strict = new Strict();

        $this->setExpectedException(RuntimeException::class);
        $strict->call($foo);
    }

This probably raises more questions. Sorry about that, I'm just about to head home for the week.

In the mean time, there's a pretty fundamental difference between Mockery's approach and Phony's. Mockery uses what is sometimes called the "expect-run-verify" pattern, which some mocking frameworks, like Phake, Prophecy, and Phony deliberately avoid.

You can read about the original coining of the term at expect-run-verify… Goodbye!. This is by the person who wrote Mockito, a massively popular Java mocking library, that introduced a lot of the ideas that influenced Phony. There's also some useful info on the Mockito Wikipedia page.

Hope that helps for now.

[enumag]

@ezzatron I think I get the gist of what you mean but I'll still really appriciate the proper response. :-)

[ezzatron]

So, it seems the first time I read this question, I missed the "without getting rid of any assertions" part of the question. That makes it a much more difficult question to answer.

One of the big differences between Mockery and Phony is that Mockery uses "expectations", whereas Phony uses "stubbing" and "verification". Expectations are essentially stubbing and verification mashed together, so there's a lot of overlap in their features.

Expectations force you to define every interaction that will happen before the actual body of test takes place. In contrast, Phony splits this into rules about how to respond when called (stubbing), and ways to assert what did happen once the body of the test is complete (verification).

Personally, my first experience with mocking was PHPUnit's mocks, which use expectations. After writing a lot of tests, I eventually realized that PHPUnit's mocks had some big problems:

• Sometimes I'd have to "expect" a method to be called, even when it had nothing to do with a test.
• Adding new functionality that required additional calls could break entire test suites, because none of them "expected" the new call.
• It was very tricky to set up a mock in a setUp() method, and override its behavior for one or two tests.
• When an expectation failed, the failure message was often very confusing, and wouldn't point me to where the problem really was.
• Using expectations just felt like it was in the wrong order. That is, I'd have some setup code, then expectations, then the actual code under test, then maybe some more assertions afterward. Ideally, all the setup code could be grouped together, and all the assertions could be grouped together.

Once I realized this, I started looking for alternatives, and found other mocking frameworks, which completely abandoned expectations in lieu of stubbing and verification. It took quite a while to really get it, but getting rid of expectations solved pretty much all of the above problems for me.

Unfortunately I don't have enough experience with Mockery to say with confidence whether it manages to avoid these issues better than PHPUnit's mocks did.

Regardless, along the way, I learned other ways to make my tests less "brittle". There's so many aspects to the concept of test "brittleness", but one thing that always seems to help, is to reduce the amount of assertions I'm making to the bare minimum.

Mocking frameworks that provide stubbing and verification fit well into this approach, because they give you the power to make decisions about what assertions you're making in each test. If some aspect of a system's behavior is irrelevant to your current test, you don't have to make additional expectations just to get the test working.

You also get the benefit of being able to push some of your stubbing off into the shared "set up" stage of a test suite, without requiring that every test in the suite makes use of it.

The trade-off for these benefits is that you have to be more explicit about what you're asserting. Generally speaking, this isn't a bad thing. If you've got a really complicated test, with lots of assertions, it's usually a sign that you're trying to do too much in one test, or that your code is overly complex.

At the end of the day, you have to decide which tool fits your needs better. Personally, I'll always avoid expectations because I feel that they cause more problems than they solve.

In addition to all of that, I have some more specific comments on your original examples. In both examples you use these two verifications:

$fooHandle->bar->once()->called();
$fooHandle->bar->calledWith();

Which could probably be collapsed into one:

$fooHandle->bar->once()->calledWith();

Because all Phony verifications support cardinality modifiers like once(). Note that these two snippets are not exactly equivalent. The second will not verify that bar was never called with other arguments. However, the same is also true of your original Mockery example.

Also note that with Phony, unless you explicitly care about the number of times
something happened, it is safe to leave off the cardinality modifier:

$fooHandle->bar->calledWith();

This is because Phony has a default cardinality of atLeast(1). That is, if it never happened, the verification is guaranteed to fail. From what I understand, this is different to Mockery, where an unmet expectation won't fail unless you explicitly specify something like once().

There are also a couple of non-Phony things I would suggest. Here is a revised version of the error-case, that does not remove any assertions:

    public function testStrictExceptionWithPhony()
    {
        $fooHandle = Phony::mock(FooInterface::class);
        $fooHandle
            ->bar
            ->returns(null);

        $strict = new Strict();

        $this->expectException(RuntimeException::class);
        try {
            $strict->call($fooHandle->get());
        } catch (RuntimeException $e) {
            $fooHandle
                ->bar
                ->once()
                ->calledWith();

            throw $e;
        }
    }

There are two things of note here:

• It's usually better to use expectException() (setExpectedException() in older versions of PHPUnit) instead of the @expectedException annotation, and to use it as late as possible in the test. This helps prevent a false positive when something else throws a matching exception.
• I also put the verifications and re-throw inside catch rather than finally, because the test will still fail if the expected exception is not thrown.

Beyond these changes, for these particular cases there's probably nothing you can do make the Phony examples as simple as the Mockery examples unless you make less assertions. But from my point of view at least, the amount of assertions being made here can only lead to very brittle tests, which is something you should seek to avoid in any case.

[ezzatron]

Closing to clean up active issues. Please feel free to continue discussion.