not fast

[KatyConnor]

The project build speed is not fast, the same as mvn

[gnodet]

Could you measure the time of the first, second and third build with mvn and mvnd and report ?

[conderls]

I also did a benchmark with two scala projects, which have about 30 subprojects:

• flow-all: without maven-shade-plugin,
• offline-all: with maven-shade-plugin to build fat-jar which required for spark task.

using commands for both mvn/mvnd: mvn -pl $PROJECT -am spotless:apply clean --no-snapshot-updates $PHASE, while extra opts for:

• mvn: -T 1C
• mvnd: -Dquickly -Dmvnd.maxHeapSize=8G

reuslt

PROJECT	PHASE	mvn(s)	mvnd(s)
flow-all	install	215/253/238/219/208/205(avg=223)	254/259/245/205/230/209(avg=233)
offline-all	package	416/433/431/443(avg=430)	412/431/418/398/411/409(avg=413)

the performance of compilation and build is quite close to each other.

by the way, it is time-consuming to build fat-jar with maven-shade-plugin, is there better way to create fat-jar for spark jobs?

[gnodet]

I also did a benchmark with two scala projects, which have about 30 subprojects:

• flow-all: without maven-shade-plugin,
• offline-all: with maven-shade-plugin to build fat-jar which required for spark task.

using commands for both mvn/mvnd: mvn -pl $PROJECT -am spotless:apply clean --no-snapshot-updates $PHASE, while extra opts for:

• mvn: -T 1C
• mvnd: -Dquickly -Dmvnd.maxHeapSize=8G

reuslt

PROJECT PHASE mvn(s) mvnd(s)
flow-all install 215/253/238/219/208/205(avg=223) 254/259/245/205/230/209(avg=233)
offline-all package 416/433/431/443(avg=430) 412/431/418/398/411/409(avg=413)
the performance of compilation and build is quite close to each other.

by the way, it is time-consuming to build fat-jar with maven-shade-plugin, is there better way to create fat-jar for spark jobs?

How come the duration for mvn is decreasing run after run ? Is that a public project ?

[gnodet]

How come the duration for mvn is decreasing run after run ?

Well, it doesn't really. At least it's not really meaningful.
Do you have lots of test execution ? mvnd is not magic so if most of your time is already spent compiling and running tests and if you compare with a concurrent maven build, the only remaining benefit is to warm-up the JVM. The benefit becomes nearly insignificant if you spend 5 minutes running tests...

[conderls]

by default, I turned off test during compiling and packaging by

        <scoverage.skip>true</scoverage.skip>
        <scalatest.skip>true</scalatest.skip>
        <surefire.skip>true</surefire.skip>

for offline-all, it tooks about 50% time spent to build fat-jar for maven-shade-plugin.

[gnodet]

So mvnd speeds up by

• enabling concurrent build by default
• cacheing a few things and help speeding up the time spent by maven itself when loading the poms, the plugins
• keeping the JVM warm
  If if the plugins themselves use 95% of the build time and you're comparing to a multithreaded build, mvnd won't help much...

[conderls]

So mvnd speeds up by

* enabling concurrent build by default

* cacheing a few things and help speeding up the time spent by maven _itself_ when loading the poms, the plugins

* keeping the JVM warm
  If if the plugins themselves use 95% of the build time and you're comparing to a multithreaded build, `mvnd` help much...

Thanks for your time.

• Considering "enabling concurrent build by default", it seems mvn with -T 1C opts can also achieve multi-threaded build?
• only for offline-all project which requires maven-shade-plugin to build fat-jar, the shade-plugin uses most of the build time. for flow-all, there are no time-consuming plugins observed.

[gnodet]

So mvnd speeds up by

* enabling concurrent build by default

* cacheing a few things and help speeding up the time spent by maven _itself_ when loading the poms, the plugins

* keeping the JVM warm
  If if the plugins themselves use 95% of the build time and you're comparing to a multithreaded build, `mvnd` help much...

Thanks for your time.

• Considering "enabling concurrent build by default", it seems mvn with -T 1C opts can also achieve multi-threaded build?

Yes, though the output is imho much more usable in mvnd.

• only for offline-all project which requires maven-shade-plugin to build fat-jar, the shade-plugin uses most of the build time. for flow-all, there are no time-consuming plugins observed.

Sure, but again, mvnd does not speed up plugin execution (or only to some degree because the JVM can be kept warm and JIT can kick in). You can also investigate your build adding -Dmvnd.buildTime=true on the command line, the report at the end should tell you where time was spent in plugins.

[conderls]

with -Dmvnd.buildTime=true options, the report shows that, the time spent in plugins is negligible, e.g.

[gnodet]

Then there may be a problem. Can you provide a project for investigation ?

[conderls]

Then there may be a problem. Can you provide a project for investigation ?

sorry for that, the projects are accessible in intranet only now.

[gnodet]

Well, I can't do much without any data. Please open a new issue when/if you have something that can be reproduced.