Feature request: expected exception

[robocoder]

Something like PHPUnit's @ExpectedException, e.g., when a scenario is expected to fail.

[stof]

a scenario expected to fail ?

Btw, Behat is about testing the expected output. And the output of the application will not be a PHP exception. If an exception occurs, you will receive a blank page or an error page.

[robocoder]

I have some negative test cases in my .feature file to test step definitions in my custom sub context are throwing exceptions, e.g., AmbiguousException.

[everzet]

What are you trying to achieve? And yeah, Behat is not unit testing framework.

[robocoder]

As an example, I have a step definition that takes an optional TableNode.

In the abnormal case, the step definition throws AmbiguousException if ($conditionA xor $conditionB) i.e., both or neither conditions must be satisfied.

[everzet]

I've asked you to provide scenario, you're concentrating on implementation :)

[zombor]

@stof It's also important to keep in mind that Behat isn't (or should not be, imo) completely focused on web UI testing. There's a lot more to BDD than testing html output. That may not change the scenario here, but I think it's an important distinction.

It should be a common situation where you can do BDD development with raw code with Behat.

[zombor]

@robocoder You can use any kind of phpunit assertions in your step definitions. Just use them :)

[stof]

@zombor I know it is not limited to web. but if you are testing that you expect an exception, you are at the unit testing level, not at the behavior testing level IMO.

[everzet]

@zombor of course. Behat could/should be used to describe everything, that has business value in the project.

But exceptions are too much low-level to describe as business rules (in most cases). In 99% cases, most deep level for Behat is "some action can not be done" (web UI, cli, web API or anything), not "... exception has been thrown" as @stof said. Business people make observations around application input/output, not classes and their exceptions - it's developers area.

[everzet]

I will try to clarify this as much as possible in my symfony_live talk.

[zombor]

but if you are testing that you expect an exception, you are at the unit testing level, not at the behavior testing level

Surely throwing an exception is behavior, just as any other output.

But exceptions are too much low-level to describe as business rules (in most cases).

Sure. I would not have a gherkin step that says "Then the class will throw ExceptionThing". I would say "Then I should see an error" or maybe something more specific with an error message string or something.

The step definition hides the actual implementation details (or rather, can emulate a delivery mechanism), and I don't think it's particularly wrong to assert an exception being thrown for this kind of behavior in the step definition if that's what the behavior calls for.

This probably isn't the right forum for this kind of discussion though :)

[robocoder]

Sorry, it looks like I wasn't clear. @ExpectedException means the scenario expects a Behat exception.

But having looked at the Behat/Behat's features/ directory, it's evidently better to use:

  Given a file named ...
  When I run "behat ..."
  Then it should fail
  And the output should contain:
    """
    ...
    """