New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build time overhead when running with surefire smart test provider #44

Closed
bartoszmajsak opened this Issue Jun 27, 2017 · 4 comments

Comments

Projects
None yet
1 participant
@bartoszmajsak
Member

bartoszmajsak commented Jun 27, 2017

Issue Overview

When playing with arquillian-core I've noticed interesting "anomaly". It seems that by simply adding our provider significantly increases the build time (I would rather say it adds 20s rather than doubles it, but it is still a huge cost for relatively small projects).

I added "affected" and "selection" mode and here are the numbers:

Regular build against master branch
[INFO] Total time: 27.764 s
[INFO] Finished at: 2017-06-27T19:52:54+02:00
[INFO] Final Memory: 69M/1678M
[INFO] ------------------------------------------------------------------------
arquillian-core [mvn clean install]
Build against affected strategy (so no tests executed)
[INFO] Total time: 44.933 s
[INFO] Finished at: 2017-06-27T19:54:24+02:00
[INFO] Final Memory: 96M/3387M
[INFO] ------------------------------------------------------------------------
arquillian-core [mvn clean install]

Configuration can be found here: https://github.com/bartoszmajsak/arquillian-core/tree/smart_testing_poc

Additional Information

I ruled out possibility that we are in fact silently running the tests without notifying maven. For a peek into the maven execution time you can use https://github.com/jcgay/maven-profiler

My maven configuration:

Apache Maven 3.5.0 (ff8f5e7444045639af65f6095c62210b5713f426; 2017-04-03T21:39:06+02:00)
Maven home: /usr/bin/mvn
Java version: 1.8.0_121, vendor: Oracle Corporation
Java home: /usr/java/jdk1.8.0_121/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "4.11.5-100.fc24.x86_64", arch: "amd64", family: "unix"
@bartoszmajsak

This comment has been minimized.

Show comment
Hide comment
@bartoszmajsak

bartoszmajsak Jul 6, 2017

Member

I think it also applies to other strategies (new and changed).

Member

bartoszmajsak commented Jul 6, 2017

I think it also applies to other strategies (new and changed).

@bartoszmajsak

This comment has been minimized.

Show comment
Hide comment
@bartoszmajsak

bartoszmajsak Jul 7, 2017

Member

Observartions:

  • logic for calculating changes is invoked every time surefire is executed (that means for each and every module having tests - this leads to excessive IO and repeated git operations for example
  • we should pre-calculate this before we even start running the build
  • one of the possible approaches would be to move this logic to maven extension and implement services which will be returning particular tests for each strategy during surfire build, based on pre-calculated model
Member

bartoszmajsak commented Jul 7, 2017

Observartions:

  • logic for calculating changes is invoked every time surefire is executed (that means for each and every module having tests - this leads to excessive IO and repeated git operations for example
  • we should pre-calculate this before we even start running the build
  • one of the possible approaches would be to move this logic to maven extension and implement services which will be returning particular tests for each strategy during surfire build, based on pre-calculated model
@bartoszmajsak

This comment has been minimized.

Show comment
Hide comment
@bartoszmajsak

bartoszmajsak Jul 7, 2017

Member

We are investigating usage of Maven Extension model to solve this problem

Member

bartoszmajsak commented Jul 7, 2017

We are investigating usage of Maven Extension model to solve this problem

@bartoszmajsak

This comment has been minimized.

Show comment
Hide comment
@bartoszmajsak

bartoszmajsak Jul 28, 2017

Member

Closed via #57 and ChangeStorage implementation.

Member

bartoszmajsak commented Jul 28, 2017

Closed via #57 and ChangeStorage implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment