[MBUILDCACHE-78] Maven reactor doesn't work well for a multimodule project when enabling build cache

[jira-importer]

Hongxin Liang opened MBUILDCACHE-78 and commented

In a multimodule project, as a simplest example: https://github.com/honnix/cache , "mvn compile" with build cache enabled doesn't play well with Maven reactor. A few scenarios I have tested in the provided repro example:

1. With a clean environment (no build cache, no "mvn install" have been done)
   1. Run "mvn compile" and everything works
   2. Modify (putting a random empty line will suffice) class "B" in module "b"
   3. Run "mvn compile" again, and now it fails when trying to download snapshot of module "a" from maven central (after failing to find that in my local repo)
2. With a clean environment (no build cache)
   1. Run "mvn install" and everything works
   2. Modify class "A" in module "a" to rename the method
   3. Run "mvn compile" for the first time, and it fails to compile due to renaming of the method in class "A"
   4. Run "mvn compile" for the second time, nothing fails but apparent the code is still wrong; run more "mvn compile" and still nothing fails
   5. Running "mvn install" fails the build due to compilation error

I'm not entirely sure I am doing things properly and I'm not sure whether the config is correct either (I took the example as https://maven.apache.org/extensions/maven-build-cache-extension/maven-build-cache-config.xml with modification to disable remote cache).

---

Affects: 1.1.0

[jira-importer]

Johan Parent commented

Hi,

I confirm. The checksum for the pom seems to  be different each run.

Having looked briefly at the code I suppose the effecitve pom representation contains something that "doesn't hash" consistently. A missing hashCode() implementation, but I speculate.

Regards,

Johan

[jira-importer]

Johan Parent commented

Hello Hongxin Liang ,

I tried on your cache project. Thing seems to work, no?  The checksums match and files get restored. Or is their content wrong?

Regards,

Johan

 INFO] --------------------------< io.honnix:cache >---------------------------
[INFO] Building cache 1.0-SNAPSHOT                                        [1/3]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] ----------------------------< io.honnix:a >-----------------------------
[INFO] Building a 1.0-SNAPSHOT                                            [2/3]
[INFO]   from a/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 6, plugins: 4 millis
[INFO] Project inputs calculated in 25 ms. SHA-256 checksum [585351936dd02c1a4d84a3384e83fa66698e678148b14c141a3fb8855e32df6c] calculated in 24 ms.
[INFO] Attempting to restore project io.honnix:a from build cache
[INFO] Local build found by checksum 585351936dd02c1a4d84a3384e83fa66698e678148b14c141a3fb8855e32df6c
[INFO] Found cached build, restoring io.honnix:a from cache by checksum 585351936dd02c1a4d84a3384e83fa66698e678148b14c141a3fb8855e32df6c
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO] Skipping plugin execution (cached): resources:testResources
[INFO] Skipping plugin execution (cached): compiler:testCompile
[INFO] Skipping plugin execution (cached): surefire:test
[INFO] 
[INFO] ----------------------------< io.honnix:b >-----------------------------
[INFO] Building b 1.0-SNAPSHOT                                            [3/3]
[INFO]   from b/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=b]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 2, plugins: 1 millis
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Project inputs calculated in 6 ms. SHA-256 checksum [4fded1cb646674a216944e06b31754cb5a90c70ea90b8c1acc18f56fd11a6a7b] calculated in 2 ms.
[INFO] Attempting to restore project io.honnix:b from build cache
[INFO] Local build found by checksum 4fded1cb646674a216944e06b31754cb5a90c70ea90b8c1acc18f56fd11a6a7b
[INFO] Found cached build, restoring io.honnix:b from cache by checksum 4fded1cb646674a216944e06b31754cb5a90c70ea90b8c1acc18f56fd11a6a7b
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO] Skipping plugin execution (cached): resources:testResources
[INFO] Skipping plugin execution (cached): compiler:testCompile
[INFO] Skipping plugin execution (cached): surefire:test
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for cache 1.0-SNAPSHOT:
[INFO] 
[INFO] cache .............................................. SUCCESS [  0.005 s]
[INFO] a .................................................. SUCCESS [  0.279 s]
[INFO] b .................................................. SUCCESS [  0.019 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.390 s
[INFO] Finished at: 2024-05-17T13:14:50+02:00
[INFO] ------------------------------------------------------------------------

[jira-importer]

Hongxin Liang commented

Hi parent_johan,

Thank you for looking into this. It has been a while and I don't remember details. Have you tried following the repro steps I put in the description? I can try it out again later asap.

/H

[jira-importer]

Hongxin Liang commented

I could still produce these two using 1.2.0 version.

For 1, I got this log:

[INFO] --------------------------< io.honnix:cache >---------------------------
[INFO] Building cache 1.0-SNAPSHOT                                        [1/3]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] ----------------------------< io.honnix:a >-----------------------------
[INFO] Building a 1.0-SNAPSHOT                                            [2/3]
[INFO]   from a/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Loading cache configuration from /Users/honnix/Developer/github/cache/.mvn/maven-build-cache-config.xml
[INFO] Using SHA-256 hash algorithm for cache
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 1, plugins: 1 millis
[INFO] Project inputs calculated in 3 ms. SHA-256 checksum [d94bcab16e7817cbb356a288016b80130c30e87d3c0e96f63347697334c206f5] calculated in 1 ms.
[INFO] Attempting to restore project io.honnix:a from build cache
[INFO] Local build found by checksum d94bcab16e7817cbb356a288016b80130c30e87d3c0e96f63347697334c206f5
[INFO] Found cached build, restoring io.honnix:a from cache by checksum d94bcab16e7817cbb356a288016b80130c30e87d3c0e96f63347697334c206f5
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO]
[INFO] ----------------------------< io.honnix:b >-----------------------------
[INFO] Building b 1.0-SNAPSHOT                                            [3/3]
[INFO]   from b/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=b]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 1, plugins: 1 millis
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Project inputs calculated in 2 ms. SHA-256 checksum [8a63d23bd4e2fb7162d1e6ef5d415a8b35527fc13c1b58122e2af25f52d53de0] calculated in 0 ms.
[INFO] Attempting to restore project io.honnix:b from build cache
[INFO] Local build was not found by checksum 8a63d23bd4e2fb7162d1e6ef5d415a8b35527fc13c1b58122e2af25f52d53de0 for io.honnix:b
[INFO] Downloading from repo: ...
[INFO] Downloading from repo: ...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for cache 1.0-SNAPSHOT:
[INFO]
[INFO] cache .............................................. SUCCESS [  0.000 s]
[INFO] a .................................................. SUCCESS [  0.012 s]
[INFO] b .................................................. FAILURE [ 16.822 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  16.845 s (Wall Clock)
[INFO] Finished at: 2024-05-17T14:34:19+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project b: Could not resolve dependencies for project io.honnix:b:jar:1.0-SNAPSHOT: The following artifacts could not be resolved: io.honnix:a:jar:1.0-SNAPSHOT (absent): Could not find artifact io.honnix:a:jar:1.0-SNAPSHOT in repo (...) -> [Help 1]

For 2, I got this log:

$ mvn compile
[INFO] Scanning for projects...
[INFO] BuildTimeEventSpy is registered.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] cache                                                              [pom]
[INFO] a                                                                  [jar]
[INFO] b                                                                  [jar]
[INFO]
[INFO] Using the SmartBuilder implementation with a thread count of 11
[INFO]
[INFO] --------------------------< io.honnix:cache >---------------------------
[INFO] Building cache 1.0-SNAPSHOT                                        [1/3]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] ----------------------------< io.honnix:a >-----------------------------
[INFO] Building a 1.0-SNAPSHOT                                            [2/3]
[INFO]   from a/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Loading cache configuration from /Users/honnix/Developer/github/cache/.mvn/maven-build-cache-config.xml
[INFO] Using SHA-256 hash algorithm for cache
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 2, plugins: 0 millis
[INFO] Project inputs calculated in 2 ms. SHA-256 checksum [c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c] calculated in 0 ms.
[INFO] Attempting to restore project io.honnix:a from build cache
[INFO] Local build was not found by checksum c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c for io.honnix:a
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ a ---
[INFO] skip non existing resourceDirectory /Users/honnix/Developer/github/cache/a/src/main/resources
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ a ---
[INFO] Changes detected - recompiling the module! :source
[INFO] Compiling 1 source file with javac [debug release 17] to target/classes
[INFO] Saved Build to local file: /Users/honnix/.m2/build-cache/v1/io.honnix/a/c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c/local/buildinfo.xml
[INFO]
[INFO] ----------------------------< io.honnix:b >-----------------------------
[INFO] Building b 1.0-SNAPSHOT                                            [3/3]
[INFO]   from b/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=b]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 0, plugins: 1 millis
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Project inputs calculated in 2 ms. SHA-256 checksum [8bffbe36857d63a0897bc4a9b69f2e2beb50c5170043ad7de09a01e2d790da85] calculated in 0 ms.
[INFO] Attempting to restore project io.honnix:b from build cache
[INFO] Local build was not found by checksum 8bffbe36857d63a0897bc4a9b69f2e2beb50c5170043ad7de09a01e2d790da85 for io.honnix:b
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ b ---
[INFO] skip non existing resourceDirectory /Users/honnix/Developer/github/cache/b/src/main/resources
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ b ---
[INFO] Changes detected - recompiling the module! :dependency
[INFO] Compiling 1 source file with javac [debug release 17] to target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /Users/honnix/Developer/github/cache/b/src/main/java/io/honnix/B.java:[7,13] cannot find symbol
  symbol:   method a()
  location: class io.honnix.A
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for cache 1.0-SNAPSHOT:
[INFO]
[INFO] cache .............................................. SUCCESS [  0.001 s]
[INFO] a .................................................. SUCCESS [  0.039 s]
[INFO] b .................................................. FAILURE [  0.025 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.075 s (Wall Clock)
[INFO] Finished at: 2024-05-17T14:39:43+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:compile (default-compile) on project b: Compilation failure
[ERROR] /Users/honnix/Developer/github/cache/b/src/main/java/io/honnix/B.java:[7,13] cannot find symbol
[ERROR]   symbol:   method a()
[ERROR]   location: class io.honnix.A
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <args> -r

$ mvn compile
[INFO] Scanning for projects...
[INFO] BuildTimeEventSpy is registered.
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] cache                                                              [pom]
[INFO] a                                                                  [jar]
[INFO] b                                                                  [jar]
[INFO]
[INFO] Using the SmartBuilder implementation with a thread count of 11
[INFO]
[INFO] --------------------------< io.honnix:cache >---------------------------
[INFO] Building cache 1.0-SNAPSHOT                                        [1/3]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO]
[INFO] ----------------------------< io.honnix:a >-----------------------------
[INFO] Building a 1.0-SNAPSHOT                                            [2/3]
[INFO]   from a/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Loading cache configuration from /Users/honnix/Developer/github/cache/.mvn/maven-build-cache-config.xml
[INFO] Using SHA-256 hash algorithm for cache
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 1, plugins: 0 millis
[INFO] Project inputs calculated in 2 ms. SHA-256 checksum [c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c] calculated in 1 ms.
[INFO] Attempting to restore project io.honnix:a from build cache
[INFO] Local build found by checksum c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c
[INFO] Found cached build, restoring io.honnix:a from cache by checksum c314c8ae60babb75456ed81056b4c11fe52c0b74d713a623cee87361768a672c
[INFO] Skipping plugin execution (cached): resources:resources
[INFO] Skipping plugin execution (cached): compiler:compile
[INFO]
[INFO] ----------------------------< io.honnix:b >-----------------------------
[INFO] Building b 1.0-SNAPSHOT                                            [3/3]
[INFO]   from b/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=b]
[INFO] Scanning plugins configurations to find input files. Probing is enabled, values will be checked for presence in file system
[INFO] Found 1 input files. Project dir processing: 1, plugins: 0 millis
[INFO] Going to calculate checksum for project [groupId=io.honnix, artifactId=a]
[INFO] Project inputs calculated in 2 ms. SHA-256 checksum [8bffbe36857d63a0897bc4a9b69f2e2beb50c5170043ad7de09a01e2d790da85] calculated in 1 ms.
[INFO] Attempting to restore project io.honnix:b from build cache
[INFO] Local build was not found by checksum 8bffbe36857d63a0897bc4a9b69f2e2beb50c5170043ad7de09a01e2d790da85 for io.honnix:b
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ b ---
[INFO] skip non existing resourceDirectory /Users/honnix/Developer/github/cache/b/src/main/resources
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ b ---
[INFO] Changes detected - recompiling the module! :source
[INFO] Compiling 1 source file with javac [debug release 17] to target/classes
[INFO] Saved Build to local file: /Users/honnix/.m2/build-cache/v1/io.honnix/b/8bffbe36857d63a0897bc4a9b69f2e2beb50c5170043ad7de09a01e2d790da85/local/buildinfo.xml
[INFO] ------------------------------------------------------------------------
[INFO] Average project wall time: 0,00s
[INFO] Total concurrency: 9 %
[INFO] Bottleneck projects that decrease concurrency: (run build with -Dsmartbuilder.profiling=true for further details)
[INFO] 	- io.honnix:b:1.0-SNAPSHOT
[INFO] 	- io.honnix:a:1.0-SNAPSHOT
[INFO] 	- io.honnix:cache:1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for cache 1.0-SNAPSHOT:
[INFO]
[INFO] cache .............................................. SUCCESS [  0.000 s]
[INFO] a .................................................. SUCCESS [  0.008 s]
[INFO] b .................................................. SUCCESS [  0.037 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.054 s (Wall Clock)
[INFO] Finished at: 2024-05-17T14:39:51+02:00
[INFO] ------------------------------------------------------------------------

In general I have a feeling it doesn't play well with maven reactor.

[olamy]

this is actually per design see documentation

maven-build-cache-extension/src/site/markdown/usage.md

Line 20 in 21d9bef

Once the extension is activated, the cache automatically kicks in on every `package` or higher phase.