New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parallelised unit tests #350

Closed
wants to merge 2 commits into
base: trunk
from

Conversation

Projects
None yet
4 participants
@ham1

ham1 commented Dec 6, 2017

Description

introduced JMeterSerialTest interface for tests which need to be run in sequence not concurrently.
Re-wrote TestFileServer in Spock to use private constructor to get around singleton and make clearer to understand.
Added a test listener to time and log to file time taken for each unit test.
Also refactored/re-formatted some code while investigating parallel test failures.

ant _test time on a 4 core laptop:
before: 1m 19s
after: 51s

on an 8 core PC:
before: 59s
after: 31s

I'd welcome more testing on different configurations and review.

Motivation and Context

Speed up feedback

How Has This Been Tested?

100 iterations of ant _test on an 8 core machine
50 iterations of ant _test on a 4 core machine

Screenshots (if appropriate):

Types of changes

  • Dev improvement

Checklist:

  • My code follows the code style of this project.
  • I have updated the documentation accordingly.
@codecov-io

This comment has been minimized.

codecov-io commented Dec 6, 2017

Codecov Report

Merging #350 into trunk will increase coverage by 0.01%.
The diff coverage is 89.28%.

Impacted file tree graph

@@             Coverage Diff              @@
##              trunk     #350      +/-   ##
============================================
+ Coverage     58.28%   58.29%   +0.01%     
- Complexity    10217    10228      +11     
============================================
  Files          1160     1160              
  Lines         74426    74424       -2     
  Branches       7343     7362      +19     
============================================
+ Hits          43378    43388      +10     
+ Misses        28552    28535      -17     
- Partials       2496     2501       +5
Impacted Files Coverage Δ Complexity Δ
...ter/protocol/http/control/TestCacheManagerHC4.java 78.18% <ø> (ø) 12 <0> (ø) ⬇️
...isualizers/backend/SamplerMetricFixedModeTest.java 100% <ø> (ø) 9 <0> (ø) ⬇️
...isualizers/backend/SamplerMetricTimedModeTest.java 100% <ø> (ø) 7 <0> (ø) ⬇️
...g/apache/jmeter/functions/CSVReadFunctionTest.java 89.88% <100%> (-1.63%) 11 <4> (ø)
...rg/apache/jmeter/functions/TestGroovyFunction.java 92.3% <100%> (ø) 10 <1> (ø) ⬇️
...ache/jmeter/functions/TestFileRowColContainer.java 82.71% <100%> (ø) 6 <3> (ø) ⬇️
...t/src/org/apache/jmeter/functions/PackageTest.java 91.63% <100%> (ø) 25 <1> (ø) ⬇️
...c/org/apache/jmeter/samplers/TestSampleResult.java 93.85% <100%> (ø) 24 <4> (ø) ⬇️
...g/apache/jmeter/assertions/SMIMEAssertionTest.java 100% <100%> (ø) 9 <1> (ø) ⬇️
...c/org/apache/jmeter/reporters/TestResultSaver.java 100% <100%> (ø) 7 <1> (ø) ⬇️
... and 98 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 286a89e...693da2f. Read the comment docs.

ham1 added some commits Dec 6, 2017

Parallelised unit tests.
Introduced JMeterSerialTest interface for tests which need to be run
individually in sequence not concurrently.
Re-wrote TestFileServer in Spock to use private constructor to get around singleton.
Also refactored/re-formatted some code while investigating parallel test failures.

@asfgit asfgit closed this in 32c5b11 Dec 7, 2017

asfgit pushed a commit that referenced this pull request Dec 7, 2017

Parallelised unit tests
Contributed by Graham Russell
This closes #350

git-svn-id: https://svn.apache.org/repos/asf/jmeter/trunk@1817425 13f79535-47bb-0310-9956-ffa450edef68

asfgit pushed a commit that referenced this pull request Dec 7, 2017

Parallelised unit tests
Contributed by Graham Russell
This closes #350
svn:eol

git-svn-id: https://svn.apache.org/repos/asf/jmeter/trunk@1817426 13f79535-47bb-0310-9956-ffa450edef68
@pmouawad

This comment has been minimized.

Contributor

pmouawad commented Dec 7, 2017

Hi @ham1 ,
It seems there are issues in headless mode following my merge of this PR.

See https://travis-ci.org/apache/jmeter/jobs/313214205

I'll have a look this week-end but if you have an idea before, feel free to propose a PR.

Regards

@ham1

This comment has been minimized.

ham1 commented Dec 7, 2017

Not sure. I can reproduce on trunk but It doesn't happen on this PR for me. Should have time at the weekend to investigate,

@FSchumacher

This comment has been minimized.

Contributor

FSchumacher commented Dec 8, 2017

Maybe this is the culprit:

URL: http://svn.apache.org/viewvc?rev=1817421&view=rev
Log:
Make mock react correctly on setText/getText

@FSchumacher

This comment has been minimized.

Contributor

FSchumacher commented Dec 8, 2017

If we initialize the string text to an empty string in the anonymous inner class, the tests pass.

@ham1 ham1 deleted the ham1:parallelise_unit_tests branch Dec 11, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment