New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problems running integration tests in parallel using SpringClassRule and SpringMethodRule [SPR-14112] #18684

Closed
spring-issuemaster opened this Issue Apr 4, 2016 · 4 comments

Comments

Projects
None yet
2 participants
@spring-issuemaster
Copy link
Collaborator

spring-issuemaster commented Apr 4, 2016

Andrew Toolan opened SPR-14112 and commented

Background:

I am trying to run JUnit integration tests in parallel and repeatedly using the SpringClassRule and SpringMethodRule. To do this I am using a custom runner which executes the tests in parallel and using a repeat annotation to run each test repeatedly. However I have noticed that after multiple iterations the tests will begin failing. One test always appears to succeed. The resulting stack trace seems to indicate that the Application Context has been changed. The tests do not make use of the @DirtiesContext annotation however some of the errors received appear to complain about the test context being null. I am attaching a few examples of the stack traces, below is an example:

java.lang.IllegalArgumentException: The test method of the supplied TestContext must not be null
	at org.springframework.util.Assert.notNull(Assert.java:115)
	at org.springframework.test.context.support.AbstractDirtiesContextTestExecutionListener.beforeOrAfterTestMethod(AbstractDirtiesContextTestExecutionListener.java:95)
	at org.springframework.test.context.support.DirtiesContextBeforeModesTestExecutionListener.beforeTestMethod(DirtiesContextBeforeModesTestExecutionListener.java:95)
	at org.springframework.test.context.TestContextManager.beforeTestMethod(TestContextManager.java:265)
	at org.springframework.test.context.junit4.statements.RunBeforeTestMethodCallbacks.evaluate(RunBeforeTestMethodCallbacks.java:74)
	at org.springframework.test.context.junit4.statements.RunAfterTestMethodCallbacks.evaluate(RunAfterTestMethodCallbacks.java:86)
	at org.springframework.test.context.junit4.statements.RunPrepareTestInstanceCallbacks.evaluate(RunPrepareTestInstanceCallbacks.java:64)
	at org.springframework.test.context.junit4.statements.SpringRepeat.evaluate(SpringRepeat.java:84)
	at org.springframework.test.context.junit4.statements.SpringFailOnTimeout.evaluate(SpringFailOnTimeout.java:87)
	at org.springframework.test.context.junit4.statements.ProfileValueChecker.evaluate(ProfileValueChecker.java:103)
	at com.ibm.toscana.util.junit.repeat.RunRepeatedly.evaluateStatement(RunRepeatedly.java:55)
	at com.ibm.toscana.util.junit.repeat.RunRepeatedly.evaluate(RunRepeatedly.java:34)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.experimental.theories.Theories$TheoryAnchor$1$1.evaluate(Theories.java:232)
	at org.junit.experimental.theories.Theories$TheoryAnchor.runWithCompleteAssignment(Theories.java:218)
	at org.junit.experimental.theories.Theories$TheoryAnchor.runWithAssignment(Theories.java:204)
	at com.ibm.toscana.util.junit.runner.ToscanaRunner$ParallelTheoryAnchor.runWithAssignment(ToscanaRunner.java:70)
	at org.junit.experimental.theories.Theories$TheoryAnchor.evaluate(Theories.java:187)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1397)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:300)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1067)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1703)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:168)

I have tried changing from the default Spring Repeat annotation to another implementation but I am seeing the same behavior. All tests are within the same test class. I was wondering if there was any reason why using SpringClassRule and SpringMethodRule in parallel might cause problems?

I was hoping to try run the tests without the SpringMethodRule being applied as I do not need to leverage any of Springs Method level annotations but I noticed that there is an explicit dependency between them.

I am using spring-boot version 1.3.2.RELEASE and JUnit 4.12.


Attachments:

Issue Links:

  • #10532 Allow for concurrent test execution in the TestContext framework ("duplicates")
@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator

spring-issuemaster commented Apr 11, 2016

Andrew Toolan commented

After further looking into the issue it appears that the problem I am having is related to #10532. @Sam Brannen are there any plans to resolve the issues with TestContext when running tests in parallel in the short term? I see that issue has been open for many years. Thanks!

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator

spring-issuemaster commented Jun 9, 2016

abdulmuqueet mohammed commented

Is there any update on this issue. I am facing same issue. I am using spring-boot-start-web 1.3.0 and spring-boot-starer-test -1.3.0 and spring-test 4.2.6 and Junit 4.11

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator

spring-issuemaster commented Jun 10, 2016

Sam Brannen commented

This is a duplicate of #10532.

@spring-issuemaster

This comment has been minimized.

Copy link
Collaborator

spring-issuemaster commented Jun 10, 2016

Sam Brannen commented

are there any plans to resolve the issues with TestContext when running tests in parallel in the short term?

#10532 is currently assigned to the 5.0 backlog. So we hope to address concurrency issues in the 5.0 time frame.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment