Add a test class to measure heap allocation level on a range of Maven commits

[jeanbisutti]

Why

Today, MvnValidateAllocationByMaven3VersionTest can be used to evaluate the heap allocation level for a range of Maven distributions.
To more easily spot the changes (the commits) responsible for an allocation increase, we would like to have a test bench to measure the heap allocation level between two Maven commits.

Possible implementation

We could add commit.first.hash and commit.last.hash properties in the maven-bench.properties file.

The new test could execute the following steps:

1. Clone Maven with the hash of the first commit
   You may execute Git commands with the help of a Java ProcessBuilder. A ProcessBuilder example can be found here.

2. Build Maven
   mvn -DdistributionTargetDir="{location}/apache-maven-head" clean package
   You may execute this Maven command line from Java with the help of a MvnVerifier object (see MvnValidateAllocationByMaven3VersionTest) or with the help of a Java ProcessBuilder. A ProcessBuilder example can be found here.

3. Measure heap allocation level in the same way it is done in MvnValidateAllocationByMaven3VersionTest

4. Update the local repository of Maven with the next commit

5. Build Maven

6. Measure heap allocation

...

N) Update the local repository of Maven with the last commit
N+1) Build Maven
N+2) Measure heap allocation

Don't hesitate to leave a comment to discuss about this feature!

[albertotn]

I'd like to work on this, more challenging for sure

[jeanbisutti]

@albertotn Great! :) Don't hesitate if I can help.

[albertotn]

What I've done

I have created a new test class MvnValidateAllocationByMaven3HashTest and added some configuration parameters in maven-bench.properties file.
Right now it's possible to specify an hash value from a maven git repository, checkout source code and clean package using local maven installation.

Problems

In my commit you can take a look in my implementation.
My questions:

• how to "register" an "hash" version so MvnValidate works ? See here in particular
• I'm not satisfied with this platform-depentent code, but I don't know project target system and I need to have an entry point for maven in order to clean+package source code, so any idea is appreciated
• this is a first implementation where source code must be downloaded every time, I think should be more fast to checkout maven git repository and checkout a commit, but of cource it's "heavy" for a single test classes handle all this stuff, so I have created a GitDelegate class. I think this feature could be a separated issue once this is settled

If you want, I'm able to create a PR right now, I have only added new classes, so nothing should be broken, instead to talk on code on my fork.

[hboutemy]

Hi Alberto,
regarding code to launch Maven, you can use Verifier like maven-test-bench does in its test: have a look at tests code, and eventually the reference documentation for Verifier is https://maven.apache.org/shared/maven-verifier/

thanks for your help

[jeanbisutti]

Thanks a lot for your great work Alberto!
Regarding the third point, I completely agree.
About the first, this evening I will think about this.
It's OK for the PR.
Huge thanks @albertotn for your this important step in the development of this feature.

[albertotn]

thansk @hboutemy and @jeanbisutti for your support!

[jeanbisutti]

About the way to register an "hash" version...
In MvnValidateAllocationByMaven3VersionTest the code iterates between two Maven distributions, in this feature we would like to iterate between two commits (two hash versions).
In saveMavenVersion method we could replace the Maven3Version argument by a String argument representing the hash of the commit for which allocation measure is going to be performed.

[albertotn]

Back again here for hacktoberfest, I see this still open, maybe you can merge it and close ?

[jeanbisutti]

Hi @albertotn,
The #10 PR can't be merged because of conflicts. Indeed, the code has changed last months on master. Perhaps you may be interested in adapting the PR code with the new master.