Support for @RunWith #9

Closed
Bochenski opened this Issue Mar 22, 2012 · 17 comments

Comments

Projects
None yet
6 participants

I'm trying to use cucumber-jvm with SBT, and it would be great if this tool could support @runwith

package cucumber.examples.scalacalculator

import org.junit.runner.RunWith
import org.junit.Test
import cucumber.junit.Cucumber

@runwith(classOf[Cucumber])
class RunCukesTest {

}

only is detected if you annotate it with @test above @runwith - but then it doesn't return correctly after executing the Cucumber tests.

Just a +1 as I'm also quite interested in having cucumber-jvm work with sbt

Contributor

szeiger commented May 1, 2012

The extra @test annotation should not be an issue for JUnit or junit-interface. Does it have an effect on Cucumber, thus causing the different behavior?

I have added some code to master for discovering tests with @runwith but without @test annotations. That should allow you to omit the @test annotation.

Now the tests run by just having the @runwith but there are still a few
problems:

  • The result of the test does not affect the result of the test run,
    i.e. if the tests fail the test run still passes
  • The number of cucumber tests is not added to the total test count,
    success count or failure count - probably related to the issue above
  • Only scala files will be picked up when using the @runwith

On Tue, May 1, 2012 at 2:36 PM, Stefan Zeiger <
reply@reply.github.com

wrote:

The extra @test annotation should not be an issue for JUnit or
junit-interface. Does it have an effect on Cucumber, thus causing the
different behavior?

I have added some code to master for discovering tests with @runwith but
without @test annotations. That should allow you to omit the @test
annotation.


Reply to this email directly or view it on GitHub:
szeiger#9 (comment)

Yes that's what I find also. The issue with the exception being thrown was fixed in version 1.0.0 but the output from cucumber doesn't seem to come back to the SBT console.

Contributor

szeiger commented May 2, 2012

Can you send me a log of a test run with a failed test which is not reported, using junit-interface options -v +q -n -a?

Do other test runners successfully get the failures?

Right now I'm at a loss as to what could possibly go wrong to cause such a behaviour, unless Cucumber wasn't reporting test failures to the RunNotifier in the first place.

@gnorsilva What non-Scala files do you expect to be picked up?

@Bochenski Both stdout and stderr go to the sbt console. Make sure you're running with +q (or rather: without -q) so that stdout is not swallowed by junit-interface.

Thanks for implementing the @runwith. That works great now on 0.9-SNAPSHOT :-)

Still having the issue with the output - but as you say could well be Cucumber not reporting the test failures back (too many moving parts!) I've run a failing test with -v +q -n -a as requested, the output to the console was as follows:

[stack] $ test
[info] RunCukesTest
Starting Server
when I visit the site
Stopping Server

junit.framework.ComparisonFailure: null expected:<Welcome to Stack[s]> but was:<Welcome to Stack[]>
at junit.framework.Assert.assertEquals(Assert.java:81)
at junit.framework.Assert.assertEquals(Assert.java:87)
at fundamentalStepDefinitions$$anonfun$2.apply$mcV$sp(fundamentalStepDefinitions.scala:19)
at fundamentalStepDefinitions$$anonfun$2.apply(fundamentalStepDefinitions.scala:17)
at fundamentalStepDefinitions$$anonfun$2.apply(fundamentalStepDefinitions.scala:17)
at cucumber.runtime.ScalaDsl$StepBody$$anonfun$apply$1.apply(ScalaDsl.scala:50)
at cucumber.runtime.ScalaDsl$StepBody$$anonfun$apply$1.apply(ScalaDsl.scala:50)
at cucumber.runtime.ScalaStepDefinition.execute(ScalaStepDefinition.scala:24)
at cucumber.runtime.StepDefinitionMatch.runStep(StepDefinitionMatch.java:44)
at cucumber.runtime.Runtime.runStep(Runtime.java:223)
at cucumber.runtime.model.StepContainer.runStep(StepContainer.java:44)
at cucumber.runtime.model.StepContainer.runSteps(StepContainer.java:39)
at cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:36)
at cucumber.junit.ExecutionUnitRunner.run(ExecutionUnitRunner.java:76)
at cucumber.junit.FeatureRunner.runChild(FeatureRunner.java:65)
at cucumber.junit.FeatureRunner.runChild(FeatureRunner.java:20)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at cucumber.junit.FeatureRunner.run(FeatureRunner.java:72)
at cucumber.junit.Cucumber.runChild(Cucumber.java:75)
at cucumber.junit.Cucumber.runChild(Cucumber.java:36)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at cucumber.junit.Cucumber.run(Cucumber.java:80)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:24)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
at com.novocode.junit.JUnitRunner.run(JUnitRunner.java:80)
at sbt.TestRunner.delegateRun(TestFramework.scala:62)
at sbt.TestRunner.run(TestFramework.scala:55)
at sbt.TestRunner.runTest$1(TestFramework.scala:75)
at sbt.TestRunner.run(TestFramework.scala:84)
at sbt.TestFramework$$anonfun$6$$anonfun$apply$8$$anonfun$7$$anonfun$apply$9.apply(TestFramework.scala:183)
at sbt.TestFramework$$anonfun$6$$anonfun$apply$8$$anonfun$7$$anonfun$apply$9.apply(TestFramework.scala:183)
at sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:195)
at sbt.TestFramework$$anonfun$6$$anonfun$apply$8$$anonfun$7.apply(TestFramework.scala:183)
at sbt.TestFramework$$anonfun$6$$anonfun$apply$8$$anonfun$7.apply(TestFramework.scala:183)
at sbt.Tests$$anonfun$makeSerial$1$$anonfun$apply$8.apply(Tests.scala:115)
at sbt.Tests$$anonfun$makeSerial$1$$anonfun$apply$8.apply(Tests.scala:115)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
at scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
at scala.collection.immutable.List.foreach(List.scala:45)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:194)
at scala.collection.immutable.List.map(List.scala:45)
at sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:115)
at sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:115)
at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:47)
at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:47)
at sbt.std.Transform$$anon$5.work(System.scala:67)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:221)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:221)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
at sbt.Execute.work(Execute.scala:227)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:221)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:221)
at sbt.CompletionService$$anon$1$$anon$2.call(CompletionService.scala:26)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
at ?.Then I should see the home page(fundamental.feature:5)

[info] Test run started
[info]
[info]
[info] Total for test RunCukesTest
[info] Finished in 10.065 seconds
[info] 0 tests, 0 failures, 0 errors
[info] Passed: : Total 0, Failed 0, Errors 0, Passed 0, Skipped 0
[success] Total time: 11 s, completed May 2, 2012 7:33:57 PM

Same goes if you have passing tests - always seems to report 0 tests back to SBT.

Test run with the following added to build.sbt:

testOptions += Tests.Argument(TestFrameworks.JUnit, "-v", "+q", "-n", "-a")

[info] Test run started
[info] Test cucumber.examples.java.calculator.TestTheTruth.isItTrue started
[info] Test run finished: 0 failed, 0 ignored, 1 total, 0.017s

java.lang.AssertionError: expected:<4> but was:<1>
at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
at org.junit.Assert.assertEquals(Assert.java:454)
at
cucumber.examples.java.calculator.ShoppingStepdefs.my_change_should_be_(ShoppingStepdefs.java:29)
at ✽.Then my change should be
1(cucumber/examples/java/calculator/shopping.feature:9)

[info] Test run started
[info] Passed: : Total 1, Failed 0, Errors 0, Passed 1, Skipped 0
[success] Total time: 2 s, completed 02-May-2012 21:50:02

To run the cucumber tests I need to add a scala class:

package cucumber.examples.java.calculator;

import cucumber.junit.Cucumber
import org.junit.runner.RunWith

@runwith(classOf[Cucumber])
class RunCukesTest

If I replace it with a java class, the cucumber tests do not run in sbt
(they run fine in a maven setup). This is the java class:

package cucumber.examples.java.calculator;

import cucumber.junit.Cucumber;
import org.junit.runner.RunWith;

@runwith(value = Cucumber.class)
class RunCukesTest {}

On Wed, May 2, 2012 at 7:38 PM, David Bochenski <
reply@reply.github.com

wrote:

Thanks for implementing the @runwith. That works great now on
0.9-SNAPSHOT :-)

Still having the issue with the output - but as you say could well be
Cucumber not reporting the test failures back (too many moving parts!)
I've run a failing test with -v +q -n -a as requested, the output to the
console was as follows:

[stack] $ test
[info] RunCukesTest
Starting Server
when I visit the site
Stopping Server

junit.framework.ComparisonFailure: null expected:<Welcome to Stack[s]> but
was:<Welcome to Stack[]>
at junit.framework.Assert.assertEquals(Assert.java:81)
at junit.framework.Assert.assertEquals(Assert.java:87)
at
fundamentalStepDefinitions$$anonfun$2.apply$mcV$sp(fundamentalStepDefinitions.scala:19)
at
fundamentalStepDefinitions$$anonfun$2.apply(fundamentalStepDefinitions.scala:17)
at
fundamentalStepDefinitions$$anonfun$2.apply(fundamentalStepDefinitions.scala:17)
at
cucumber.runtime.ScalaDsl$StepBody$$anonfun$apply$1.apply(ScalaDsl.scala:50)
at
cucumber.runtime.ScalaDsl$StepBody$$anonfun$apply$1.apply(ScalaDsl.scala:50)
at
cucumber.runtime.ScalaStepDefinition.execute(ScalaStepDefinition.scala:24)
at
cucumber.runtime.StepDefinitionMatch.runStep(StepDefinitionMatch.java:44)
at cucumber.runtime.Runtime.runStep(Runtime.java:223)
at
cucumber.runtime.model.StepContainer.runStep(StepContainer.java:44)
at
cucumber.runtime.model.StepContainer.runSteps(StepContainer.java:39)
at
cucumber.runtime.model.CucumberScenario.run(CucumberScenario.java:36)
at
cucumber.junit.ExecutionUnitRunner.run(ExecutionUnitRunner.java:76)
at cucumber.junit.FeatureRunner.runChild(FeatureRunner.java:65)
at cucumber.junit.FeatureRunner.runChild(FeatureRunner.java:20)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at cucumber.junit.FeatureRunner.run(FeatureRunner.java:72)
at cucumber.junit.Cucumber.runChild(Cucumber.java:75)
at cucumber.junit.Cucumber.runChild(Cucumber.java:36)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at cucumber.junit.Cucumber.run(Cucumber.java:80)
at org.junit.runners.Suite.runChild(Suite.java:128)
at org.junit.runners.Suite.runChild(Suite.java:24)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:193)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:52)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:191)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:42)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:184)
at org.junit.runners.ParentRunner.run(ParentRunner.java:236)
at org.junit.runner.JUnitCore.run(JUnitCore.java:157)
at org.junit.runner.JUnitCore.run(JUnitCore.java:136)
at com.novocode.junit.JUnitRunner.run(JUnitRunner.java:80)
at sbt.TestRunner.delegateRun(TestFramework.scala:62)
at sbt.TestRunner.run(TestFramework.scala:55)
at sbt.TestRunner.runTest$1(TestFramework.scala:75)
at sbt.TestRunner.run(TestFramework.scala:84)
at
sbt.TestFramework$$anonfun$6$$anonfun$apply$8$$anonfun$7$$anonfun$apply$9.apply(TestFramework.scala:183)
at
sbt.TestFramework$$anonfun$6$$anonfun$apply$8$$anonfun$7$$anonfun$apply$9.apply(TestFramework.scala:183)
at
sbt.TestFramework$.sbt$TestFramework$$withContextLoader(TestFramework.scala:195)
at
sbt.TestFramework$$anonfun$6$$anonfun$apply$8$$anonfun$7.apply(TestFramework.scala:183)
at
sbt.TestFramework$$anonfun$6$$anonfun$apply$8$$anonfun$7.apply(TestFramework.scala:183)
at
sbt.Tests$$anonfun$makeSerial$1$$anonfun$apply$8.apply(Tests.scala:115)
at
sbt.Tests$$anonfun$makeSerial$1$$anonfun$apply$8.apply(Tests.scala:115)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:194)
at
scala.collection.LinearSeqOptimized$class.foreach(LinearSeqOptimized.scala:59)
at scala.collection.immutable.List.foreach(List.scala:45)
at
scala.collection.TraversableLike$class.map(TraversableLike.scala:194)
at scala.collection.immutable.List.map(List.scala:45)
at sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:115)
at sbt.Tests$$anonfun$makeSerial$1.apply(Tests.scala:115)
at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:47)
at sbt.std.Transform$$anon$3$$anonfun$apply$2.apply(System.scala:47)
at sbt.std.Transform$$anon$5.work(System.scala:67)
at
sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:221)
at
sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:221)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
at sbt.Execute.work(Execute.scala:227)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:221)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:221)
at
sbt.CompletionService$$anon$1$$anon$2.call(CompletionService.scala:26)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
at ?.Then I should see the home page(fundamental.feature:5)

[info] Test run started
[info]
[info]
[info] Total for test RunCukesTest
[info] Finished in 10.065 seconds
[info] 0 tests, 0 failures, 0 errors
[info] Passed: : Total 0, Failed 0, Errors 0, Passed 0, Skipped 0
[success] Total time: 11 s, completed May 2, 2012 7:33:57 PM

Same goes if you have passing tests - always seems to report 0 tests back
to SBT.


Reply to this email directly or view it on GitHub:
szeiger#9 (comment)

Contributor

szeiger commented May 3, 2012

@Bochenski The exception in your test is not thrown on the thread that launched the test and apparently it is not reported to JUnit, so you see no results.

@gnorsilva That exception also does not come from JUnit/junit-interface/sbt but is printed directly by Cucumber. Apparently Cucumber reports the test as successful in the end so the "total" count gets incremented.

I expect you will get the same result with any JUnit runner (e.g. Eclipse or Maven) and Cucumber needs to fix the events its sending to JUnit.

@gnorsilva Java tests not being discovered looks like a bug in sbt 0.11. It seems to be fixed in 0.12.0-Beta2.

I'm closing this issue because RunWith is now supported and the other issues are not in junit-interface AFAICT.

szeiger closed this May 3, 2012

"Apparently Cucumber reports the test as successful in the end so the "total" count gets incremented."

The total count does not get incremented.

This demo project has two test files, "cucumber.examples.java.calculator.TestTheTruth.isItTrue" and "cucumber.examples.java.calculator.ShoppingStepdefs".

as you can see, this first one runs fine:

[info] Test run started
[info] Test cucumber.examples.java.calculator.TestTheTruth.isItTrue started
[info] Test run finished: 0 failed, 0 ignored, 1 total, 0.017s

then the cucumber one fails

java.lang.AssertionError: expected:<4> but was:<1>
 at org.junit.Assert.fail(Assert.java:91)
at org.junit.Assert.failNotEquals(Assert.java:645)
 at org.junit.Assert.assertEquals(Assert.java:126)
at org.junit.Assert.assertEquals(Assert.java:470)
 at org.junit.Assert.assertEquals(Assert.java:454)
at
cucumber.examples.java.calculator.ShoppingStepdefs.my_change_should_be_(ShoppingStepdefs.java:29)
 at ✽.Then my change should be
1(cucumber/examples/java/calculator/shopping.feature:9)

But then the total does not get incremented

[info] Test run started
[info] Passed: : Total 1, Failed 0, Errors 0, Passed 1, Skipped 0
[success] Total time: 2 s, completed 02-May-2012 21:50:02

Running the same project with maven yields the following

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running cucumber.examples.java.calculator.TestTheTruth
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.097 sec
Running cucumber.examples.java.calculator.RunCukesTest

java.lang.AssertionError: expected:<4> but was:<1>
    at org.junit.Assert.fail(Assert.java:93)
    at org.junit.Assert.failNotEquals(Assert.java:647)
    at org.junit.Assert.assertEquals(Assert.java:128)
    at org.junit.Assert.assertEquals(Assert.java:472)
    at org.junit.Assert.assertEquals(Assert.java:456)
    at cucumber.examples.java.calculator.ShoppingStepdefs.my_change_should_be_(ShoppingStepdefs.java:29)
    at ✽.Then my change should be 1(cucumber/examples/java/calculator/shopping.feature:9)

Tests run: 4, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 0.943 sec <<< FAILURE!

Results :

Failed tests: 
  Then my change should be 1     : expected:<4> but was:<1>
  Scenario: Give correct change  : expected:<4> but was:<1>

Tests run: 5, Failures: 2, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------

We can see that it actually picks up 5 tests to run (4 of which are all part of the same cucumber file)

Contributor

szeiger commented May 4, 2012

OK, reopening this issue. If Maven can do it, we should be able to do it, too. Can you provide a test case for this (e.g. a fork of https://github.com/szeiger/junit-interface-test with an additional Cucumber test)?

szeiger reopened this May 4, 2012

I experience the same problem. I can't find anything here to add myself to this issue as a watcher, so I'll comment and hope to receive further notifications :-)

ms-tg commented Jun 8, 2012

+1

We have the same issue trying to run cucumber-jvm tests from play test -- exception is shown and results are not aggregated with other test results

If it helps, I've put up the smallest example I can find that reproduces this issue

https://github.com/Gintellect/cucumberjvm-sbt

It's basically the scala example from the cucumber-jvm project:
https://github.com/cucumber/cucumber-jvm/tree/master/examples/scala-calculator

but with the pom.xml removed and SBT put in its place

Just run sbt and then 'test'.

You'll notice that it says Total 0, Failed 0, Errors 0, Passed 0, Skipped 0

but rest assured it is working, if you change the feature so the last scenario expects 2 (say) then test again it'll give you the (ugly looking) stack trace but included within it is the 'The the result is [whatever you changed it to] 'text

So the question is how to get it to report the correct number of passed, failed .... tests.

Contributor

szeiger commented Aug 16, 2012

With 539df52 I get correct test output. When I update to latest cucumber-scala, I still get nothing in junit-interface but I do see an error in a plain JUnit runner (apparently because of non-matching Scala versions). So the root cause seems to be that exceptions during test initialization get lost. Investigating further.

Contributor

szeiger commented Aug 16, 2012

539df52 seems to fix the problem with wrong test counts for normal test failures. The ugly stack traces must be printed by Cucumber. I also get them when I run the tests directly with JUnitCore, so there's nothing I can do about it. All output that is printed by junit-interface goes through the sbt logger, so it should be prefixed with the log level.

It looks like the strange behavior was caused by cucumber itself printing the stack trace but junit-interface running into an exception in the event handler which caused the events to be omitted from the test result and then got swallowed by sbt. https://github.com/Gintellect/cucumberjvm-sbt runs correctly now, in the original form, with an error in one of the tests, and with an initialization error.

szeiger closed this Aug 16, 2012

Thanks :-)

On 16 Aug 2012, at 13:49, Stefan Zeiger notifications@github.com wrote:

539df52 seems to fix the problem with wrong test counts for normal test failures. The ugly stack traces must be printed by Cucumber. I also get them when I run the tests directly with JUnitCore, so there's nothing I can do about it. All output that is printed by junit-interface goes through the sbt logger, so it should be prefixed with the log level.

It looks like the strange behavior was caused by cucumber itself printing the stack trace but junit-interface running into an exception in the event handler which caused the events to be omitted from the test result and then got swallowed by sbt. https://github.com/Gintellect/cucumberjvm-sbt runs correctly now, in the original form, with an error in one of the tests, and with an initialization error.


Reply to this email directly or view it on GitHub.

Contributor

benmccann commented Sep 24, 2012

hmmm, not working for me :-/ i've filed a new issue since this one's getting long

szeiger#12

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment