Stories that time out are reported as passing

[csabakoncz]

If a story exceeds its timeout but handles InterruptedException and completes without further exceptions, then it is reported as passing.

On the other hand, the Maven build fails due to CancellationException.

Sample project: https://github.com/csabakoncz/serenity-projects/tree/master/serenity-timeouts
Invoking mvn clean verify -Djbehave.threads=1 results in

Reports view generated with 3 stories (of which 0 pending) containing 2 scenarios (of which 0 pending)
Tests run: 9, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 33.994 sec <<< FAILURE! - in com.ck.bdd.JUnitRunnerFeature
com.ck.bdd.JUnitRunnerFeature  Time elapsed: 1.522 sec  <<< ERROR!
java.lang.RuntimeException: org.jbehave.core.embedder.Embedder$RunningStoriesFailed: Failures in running stories:
stories/time20.feature: java.util.concurrent.CancellationException
stories/time10.feature: java.util.concurrent.CancellationException
        at org.jbehave.core.embedder.Embedder$ThrowingRunningStoriesFailed.handleFailures(Embedder.java:553)
        at org.jbehave.core.embedder.Embedder.handleFailures(Embedder.java:238)
        at org.jbehave.core.embedder.Embedder.runStoriesAsPaths(Embedder.java:216)
        at net.serenitybdd.jbehave.embedders.ExtendedEmbedder.runStoriesAsPaths(ExtendedEmbedder.java:60)
        at net.serenitybdd.jbehave.runners.SerenityReportingRunner.run(SerenityReportingRunner.java:175)
        at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:283)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:173)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:128)


Results :


Tests in error:
  JUnit4Provider.invoke:128->executeTestSet:153->executeWithRerun:173->execute:283 » Runtime


Tests run: 9, Failures: 0, Errors: 1, Skipped: 0
[...]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[...]

This might be due to SerenityReporter ignoring storyCancelled notifications from JBehave:

    public void storyCancelled(Story story, StoryDuration storyDuration) {
        logger.debug("cancelled story ".concat(story.getName()));
    }

I am using Serenity 1.1.34 and JBehave 4.0.5

[csabakoncz]

Sorry, it seems that StoryReporter.storyCancelled() is only invoked when the executing thread throws an InterruptedException. In my example I catch this exception without re-trowing it.

Looking into the JBehave code it seems that it is only EmbedderMonitor and PerformableTree.RunContext that get notified when the main thread cancels a story due to a timeout.

[YamStranger]

@csabakoncz thanks for you issue, and it is great that you find solution! Will close this ticket.