"JAVA_HOME is not correctly set" when building `docker-maven-plugin:0.36.0` from https://github.com/jvm-repo-rebuild/reproducible-central

[wtwhite]

To work on #30 (comment) I tried building docker-maven-plugin:0.36.0 from https://github.com/jvm-repo-rebuild/reproducible-central on wtwhite-vuw-vm:

wtwhite@wtwhite-vuw-vm:~/code/reproducible-central$ time ./rebuild.sh content/io/fabric8/docker-maven-plugin/docker-maven-plugin-0.36.0.buildspec 
--snip--
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  01:48 min
[INFO] Finished at: 2023-11-09T23:48:45Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.2.0:jar (attach-javadocs) on project docker-maven-plugin: MavenReportException: Error while generating Javadoc: Unable to find javadoc command: The environment variable JAVA_HOME is not correctly set. -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:3.2.0:jar (attach-javadocs) on project docker-maven-plugin: MavenReportException: Error while generating Javadoc: Unable to find javadoc command: The environment variable JAVA_HOME is not correctly set.
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:215)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.plugin.MojoExecutionException: MavenReportException: Error while generating Javadoc: Unable to find javadoc command: The environment variable JAVA_HOME is not correctly set.
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.failOnError (AbstractJavadocMojo.java:6814)
    at org.apache.maven.plugins.javadoc.JavadocJar.doExecute (JavadocJar.java:203)
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.execute (AbstractJavadocMojo.java:1935)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: org.apache.maven.reporting.MavenReportException: Unable to find javadoc command: The environment variable JAVA_HOME is not correctly set.
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.executeReport (AbstractJavadocMojo.java:2019)
    at org.apache.maven.plugins.javadoc.JavadocJar.doExecute (JavadocJar.java:199)
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.execute (AbstractJavadocMojo.java:1935)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.io.IOException: The environment variable JAVA_HOME is not correctly set.
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.getJavadocExecutable (AbstractJavadocMojo.java:3818)
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.executeReport (AbstractJavadocMojo.java:2015)
    at org.apache.maven.plugins.javadoc.JavadocJar.doExecute (JavadocJar.java:199)
    at org.apache.maven.plugins.javadoc.AbstractJavadocMojo.execute (AbstractJavadocMojo.java:1935)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

[FAIL] /========================================================================
[FAIL] |  failed to build 
[FAIL] \========================================================================
[FAIL]  ---------->>> PROCESS WAS ABORTED <<<---------- 


real	4m11.880s
user	0m2.148s
sys	0m1.830s

No open or closed issues mention JAVA_HOME.

Presumably the error occurred inside a docker container, since AFAICT the logic for choosing how to build the project chooses docker in this case (jdk=11 and RB_SHELL unset), though no containers were running immediately afterwards:

wtwhite@wtwhite-vuw-vm:~/code/reproducible-central$ docker ps -a
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES

[wtwhite]

Surprising that any attempt is made to build javadoc, since it is explicitly disabled.

[wtwhite]

The same attempt failed at an earlier stage on piccolo:

[whitewa@piccolo ~/code/reproducible-central]$ time ./rebuild.sh content/io/fabric8/docker-maven-plugin/docker-maven-plugin-0.36.0.buildspec
--snip--
[INFO] Adding layer to rebuilder Docker image for environment settings (umask, timezone, ...)
[RUN ] docker build -t rb-ubuntu-2204-jdk11-mvn3.6.3-toolchains-11-whitewa-utc-en_us-0002 -f /home/whitewa/code/reproducible-central/bin/docker/tmp/Dockerfile-rb-ubuntu-2204-jdk11-mvn3.6.3-toolchains-11-whitewa-utc-en_us-0002 /home/whitewa/code/reproducible-central/bin 
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Sending build context to Docker daemon  101.4kB
Step 1/23 : FROM rb-ubuntu-2204-jdk11-mvn3.6.3-toolchains-11-whitewa
 ---> b79f3a1f07a0
Step 2/23 : WORKDIR /root
 ---> Running in 68224da4cdf8
Removing intermediate container 68224da4cdf8
 ---> 2d3d58064eac
Step 3/23 : RUN [ -d /scripts ] || mkdir /scripts
 ---> Running in 41537e9f45c7
Removing intermediate container 41537e9f45c7
 ---> 97ca7ea45d7c
Step 4/23 : ADD docker/*.sh /scripts/
 ---> 2b1b4202b291
Step 5/23 : ADD includes/bashcolors.sh /scripts/
failed to export image: failed to create image: failed to get layer sha256:f5ef264bdcf8d0b7d8f205e48614f78be7fb81472bd33dd13eed262521669471: layer does not exist

[FAIL] /========================================================================
[FAIL] |  Unable to build rb-ubuntu-2204-jdk11-mvn3.6.3-toolchains-11-whitewa-utc-en_us-0002 using Dockerfile-rb-ubuntu-2204-jdk11-mvn3.6.3-toolchains-11-whitewa-utc-en_us-0002 
[FAIL] \========================================================================
[FAIL]  ---------->>> PROCESS WAS ABORTED <<<---------- 


real	3m18.699s
user	0m3.602s
sys	0m0.753s

A second run gets further but hits a third distinct error:

[INFO] Rebuilder Docker image is ready for use: rb-ubuntu-2204-jdk11-mvn3.6.3-toolchains-11-whitewa-utc-en_us-0002
[INFO] Rebuilding io.fabric8:docker-maven-plugin:0.36.0 release
[RUN ] docker run -it --rm --name rebuild-central -v /home/whitewa/code/reproducible-central/content/io/fabric8/docker-maven-plugin/buildcache/docker-maven-plugin:/var/maven/app -v /home/whitewa/code/reproducible-central/m2:/var/maven/.m2 -v /home/whitewa/code/reproducible-central/.sbt:/var/maven/.sbt -v /home/whitewa/code/reproducible-central/.npm:/.npm -v /home/whitewa/code/reproducible-central/.bnd:/.bnd -u whitewa:23 -e MAVEN_CONFIG=/var/maven/.m2 -e MVN_UMASK=0002 -w /var/maven/app rb-ubuntu-2204-jdk11-mvn3.6.3-toolchains-11-whitewa-utc-en_us-0002 mvn -Prelease clean package -DskipTests -Dmaven.javadoc.skip -Dgpg.skip -V -e org.apache.maven.plugins:maven-artifact-plugin:3.5.0:compare -Dbuildinfo.reproducible -Dcompare.fail=false 
[INFO] Generating /var/maven/.m2/toolchains.xml
/usr/local/bin/mvn: line 7: /var/maven/.m2/toolchains.xml: Permission denied
[WARN] The UMASK    has been set to: 0002
[WARN] The Timezone has been set to:

Current default time zone: 'Etc/UTC'
Local time is now:      Fri Nov 10 00:16:17 UTC 2023.
Universal Time is now:  Fri Nov 10 00:16:17 UTC 2023.


Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/apache-maven
Java version: 11.0.20.1, vendor: Ubuntu, runtime: /usr/lib/jvm/java-11-openjdk-amd64
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "6.3.8-arch1-1", arch: "amd64", family: "unix"
[INFO] Error stacktraces are turned on.
[ERROR] Could not create local repository at /var/maven/.m2/repository-0002 -> [Help 1]
org.apache.maven.repository.LocalRepositoryNotAccessibleException: Could not create local repository at /var/maven/.m2/repository-0002
    at org.apache.maven.DefaultMaven.validateLocalRepository (DefaultMaven.java:364)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:169)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:566)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/LocalRepositoryNotAccessibleException

[FAIL] /========================================================================
[FAIL] |  failed to build 
[FAIL] \========================================================================
[FAIL]  ---------->>> PROCESS WAS ABORTED <<<---------- 


real	0m32.223s
user	0m0.209s
sys	0m0.159s

A third attempt produces the same error again.

Meanwhile, back on wtwhite-vuw-vm, a second run produces the same error about JAVA_HOME.

[wtwhite]

I can make the build succeed on wtwhite-vuw-vm by running bash instead of mvn in the final docker command, setting JAVA_HOME manually, and then running mvn:

wtwhite@wtwhite-vuw-vm:~/code/reproducible-central$ docker run -it --name rebuild-central -v /home/wtwhite/code/reproducible-central/content/io/fabric8/docker-maven-plugin/buildcache/docker-maven-plugin:/var/maven/app -v /home/wtwhite/code/reproducible-central/m2:/var/maven/.m2 -v /home/wtwhite/code/reproducible-central/.sbt:/var/maven/.sbt -v /home/wtwhite/code/reproducible-central/.npm:/.npm -v /home/wtwhite/code/reproducible-central/.bnd:/.bnd -u wtwhite:1000 -e MAVEN_CONFIG=/var/maven/.m2 -e MVN_UMASK=0002 -w /var/maven/app rb-ubuntu-2204-jdk11-mvn3.6.3-toolchains-11-wtwhite-utc-en_us-0002 bash
    ____                            __           _ __    __        ____        _ __    __
   / __ \___  ____  _________  ____/ /_  _______(_) /_  / /__     / __ )__  __(_) /___/ /__  _____
  / /_/ / _ \/ __ \/ ___/ __ \/ __  / / / / ___/ / __ \/ / _ \   / __  / / / / / / __  / _ \/ ___/
 / _, _/  __/ /_/ / /  / /_/ / /_/ / /_/ / /__/ / /_/ / /  __/  / /_/ / /_/ / / / /_/ /  __/ /
/_/ |_|\___/ .___/_/   \____/\__,_/\__,_/\___/_/_.___/_/\___/  /_____/\__,_/_/_/\__,_/\___/_/
          /_/

Use (for example)   SwitchJDK 11   to select the desired JDK to build with.
 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ mvn -Prelease clean package -DskipTests -Dmaven.javadoc.skip -Dgpg.skip -V -e org.apache.maven.plugins:maven-artifact-plugin:3.5.0:compare -Dbuildinfo.reproducible -Dcompare.fail=false
--snip--
[INFO] Saved info on build to /var/maven/app/target/docker-maven-plugin-0.36.0.buildinfo
[INFO] Checking against reference build from central...
Downloading from central: https://repo.maven.apache.org/maven2/io/fabric8/docker-maven-plugin/0.36.0/docker-maven-plugin-0.36.0.buildinfo
[INFO] Reference buildinfo file not found: it will be generated from downloaded reference artifacts
Downloading from central: https://repo.maven.apache.org/maven2/io/fabric8/docker-maven-plugin/0.36.0/docker-maven-plugin-0.36.0.pom
Downloaded from central: https://repo.maven.apache.org/maven2/io/fabric8/docker-maven-plugin/0.36.0/docker-maven-plugin-0.36.0.pom (20 kB at 49 kB/s)
Downloading from central: https://repo.maven.apache.org/maven2/io/fabric8/docker-maven-plugin/0.36.0/docker-maven-plugin-0.36.0.jar
Downloaded from central: https://repo.maven.apache.org/maven2/io/fabric8/docker-maven-plugin/0.36.0/docker-maven-plugin-0.36.0.jar (691 kB at 1.5 MB/s)
Downloading from central: https://repo.maven.apache.org/maven2/io/fabric8/docker-maven-plugin/0.36.0/docker-maven-plugin-0.36.0-sources.jar
Downloaded from central: https://repo.maven.apache.org/maven2/io/fabric8/docker-maven-plugin/0.36.0/docker-maven-plugin-0.36.0-sources.jar (310 kB at 476 kB/s)
[INFO] Reference build java.version: 11 (from MANIFEST.MF Build-Jdk-Spec)
[INFO] Reference build os.name: Unix (from pom.properties newline)
[INFO] Minimal buildinfo generated from downloaded artifacts: /var/maven/app/target/reference/docker-maven-plugin-0.36.0.buildinfo
[ERROR] size mismatch docker-maven-plugin-0.36.0.jar: investigate with diffoscope target/reference/io.fabric8/docker-maven-plugin-0.36.0.jar target/docker-maven-plugin-0.36.0.jar
[ERROR] Reproducible Build output summary: 2 files ok, 1 different
[ERROR] see diff target/reference/docker-maven-plugin-0.36.0.buildinfo target/docker-maven-plugin-0.36.0.buildinfo
[ERROR] see also https://maven.apache.org/guides/mini/guide-reproducible-builds.html
[INFO] Reproducible Build output comparison saved to /var/maven/app/target/docker-maven-plugin-0.36.0.buildcompare
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  16.455 s
[INFO] Finished at: 2023-11-10T01:15:51Z
[INFO] ------------------------------------------------------------------------

This results in bytecode differences in HelpMojo.class, similar to the original but more extensive:

 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ diffoscope target/reference/io.fabric8/docker-maven-plugin-0.36.0.jar target/docker-maven-plugin-0.36.0.jar | grep -C 10 javap
│ -[INFO] --- docker-maven-plugin:0.33-SNAPSHOT:tag (default-cli) @ demp-sample-zero-config ---
│ +[INFO] --- docker-maven-plugin:dmpversion:tag (default-cli) @ demp-sample-zero-config ---
│  [INFO] DOCKER> Tagging image samples/demp-sample-zero-config:0.9.0 successful!
│  [INFO] ------------------------------------------------------------------------
│  [INFO] BUILD SUCCESS
│  [INFO] ------------------------------------------------------------------------
│  [INFO] Total time:  1.155 s
│  [INFO] Finished at: 2020-06-27T14:05:33+05:30
│  [INFO] ------------------------------------------------------------------------
├── io/fabric8/maven/docker/HelpMojo.class
│ ├── javap -verbose -constants -s -l -private {}
│ │ @@ -1,495 +1,495 @@
│ │    Compiled from "HelpMojo.java"
│ │  public class io.fabric8.maven.docker.HelpMojo extends org.apache.maven.plugin.AbstractMojo
│ │    minor version: 0
│ │    major version: 52
│ │    flags: (0x0021) ACC_PUBLIC, ACC_SUPER
│ │ -  this_class: #3                          // io/fabric8/maven/docker/HelpMojo
│ │ -  super_class: #121                       // org/apache/maven/plugin/AbstractMojo
│ │ +  this_class: #2                          // io/fabric8/maven/docker/HelpMojo
│ │ +  super_class: #4                         // org/apache/maven/plugin/AbstractMojo
 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ diffoscope target/reference/io.fabric8/docker-maven-plugin-0.36.0.jar target/docker-maven-plugin-0.36.0.jar | wc -l
3464

[wtwhite]

But maybe the differences are actually due to small differences in versions:

 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ diffoscope target/reference/io.fabric8/docker-maven-plugin-0.36.0.jar target/docker-maven-plugin-0.36.0.jar | grep -C 5 demp-sample-zero-config
│  [source]
│  ----
│  ~/work/repos/docker-maven-plugin/samples/zero-config : $ mvn docker:tag -Ddocker.image.tag=0.9.0
│  [INFO] Scanning for projects...
│  [INFO]
│  [INFO] -----------< io.fabric8.dmp.samples:demp-sample-zero-config >-----------
│ -[INFO] Building demp-sample-zero-config 0.33-SNAPSHOT
│ +[INFO] Building demp-sample-zero-config 
│  [INFO] --------------------------------[ jar ]---------------------------------
│  [INFO]
│ -[INFO] --- docker-maven-plugin:0.33-SNAPSHOT:tag (default-cli) @ demp-sample-zero-config ---
│ +[INFO] --- docker-maven-plugin:dmpversion:tag (default-cli) @ demp-sample-zero-config ---
│  [INFO] DOCKER> Tagging image samples/demp-sample-zero-config:0.9.0 successful!
│  [INFO] ------------------------------------------------------------------------
│  [INFO] BUILD SUCCESS
│  [INFO] ------------------------------------------------------------------------
│  [INFO] Total time:  1.155 s
│  [INFO] Finished at: 2020-06-27T14:05:33+05:30

The mvn output suggests diffing the .buildinfo files, and this reveals differences though it's not yet clear whether they are significant:

 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ sdiff target/reference/docker-maven-plugin-0.36.0.buildinfo target/docker-maven-plugin-0.36.0.buildinfo
# effective build environment information		      |	# https://reproducible-builds.org/docs/jvm/
java.version=11 (from MANIFEST.MF Build-Jdk-Spec)	      |	buildinfo.version=1.0-SNAPSHOT
os.name=Unix (from pom.properties newline)		      |
							      >	name=docker-maven-plugin
							      >	group-id=io.fabric8
							      >	artifact-id=docker-maven-plugin
							      >	version=0.36.0
							      >
							      >	# source information
							      >	source.scm.uri=scm:git:git://github.com/fabric8io/docker-mave
							      >	source.scm.tag=HEAD
							      >
							      >	# build instructions
							      >	build-tool=mvn
							      >
							      >	# build environment information (simplified for reproducibili
							      >	java.version=11
							      >	os.name=Unix
							      >
							      >	# Maven rebuild instructions and effective environment
							      >	mvn.minimum.version=3.0.3
							      >
							      >	# output

outputs.0.groupId=io.fabric8					outputs.0.groupId=io.fabric8
outputs.0.filename=docker-maven-plugin-0.36.0.pom		outputs.0.filename=docker-maven-plugin-0.36.0.pom
outputs.0.length=19603						outputs.0.length=19603
outputs.0.checksums.sha512=1129c0d9290d5d78f0138a62ddb2ac8f40	outputs.0.checksums.sha512=1129c0d9290d5d78f0138a62ddb2ac8f40

outputs.1.groupId=io.fabric8					outputs.1.groupId=io.fabric8
outputs.1.filename=docker-maven-plugin-0.36.0.jar		outputs.1.filename=docker-maven-plugin-0.36.0.jar
outputs.1.length=690857					      |	outputs.1.length=690877
outputs.1.checksums.sha512=b00b5c0cce203e6da32c16f2a9b45ff085 |	outputs.1.checksums.sha512=c6cc9693cbd37776895904e69ed6afeb89

outputs.2.groupId=io.fabric8					outputs.2.groupId=io.fabric8
outputs.2.filename=docker-maven-plugin-0.36.0-sources.jar	outputs.2.filename=docker-maven-plugin-0.36.0-sources.jar
outputs.2.length=309659						outputs.2.length=309659
outputs.2.checksums.sha512=9f1cab445bf5e2cc6d2c4f08212834936f	outputs.2.checksums.sha512=9f1cab445bf5e2cc6d2c4f08212834936f

[wtwhite]

In fact HelpMojo.java is the only generated source file:

 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ find target/generated-sources/
target/generated-sources/
target/generated-sources/plugin
target/generated-sources/plugin/io
target/generated-sources/plugin/io/fabric8
target/generated-sources/plugin/io/fabric8/maven
target/generated-sources/plugin/io/fabric8/maven/docker
target/generated-sources/plugin/io/fabric8/maven/docker/HelpMojo.java
target/generated-sources/annotations

It appears to be generated by maven-plugin-plugin based on pom.xml lines 223-246:

      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-plugin-plugin</artifactId>
        <version>3.6.0</version>
        <configuration>
          <goalPrefix>docker</goalPrefix>
          <skipErrorNoDescriptorsFound>true</skipErrorNoDescriptorsFound>
        </configuration>
        <executions>
          <execution>
            <id>mojo-descriptor</id>
            <phase>process-classes</phase>
            <goals>
              <goal>descriptor</goal>
            </goals>
          </execution>
          <execution>
            <id>help-goal</id>
            <goals>
              <goal>helpmojo</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

Also mentioned in the generated file itself:

 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ grep @author target/generated-sources/plugin/io/fabric8/maven/docker/HelpMojo.java
 * @author maven-plugin-tools

I thought it must be that maven-plugin-plugin is somehow generating different source code now than it did for the reference build, but HelpMojo.java is included in both -source.jars with identical CRC-32s:

 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ unzip -l -v target/docker-maven-plugin-0.36.0-sources.jar |grep HelpMojo
   15231  Defl:N     3537  77% 2020-11-14 11:57 b6a0c98a  io/fabric8/maven/docker/HelpMojo.java
 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ unzip -l -v target/reference/io.fabric8/docker-maven-plugin-0.36.0-sources.jar |grep HelpMojo
   15231  Defl:N     3537  77% 2020-11-14 11:57 b6a0c98a  io/fabric8/maven/docker/HelpMojo.java

Investigating this further will mean figuring out exactly where the reference build is coming from. Although p. 6 of the paper says:

To demonstrate the effectiveness of our approach, we have to compare the two deliverable packages built in our local environment, instead of comparing the deliverable packages against the ones in the central repository.

, based on #32 (comment), it looks like the "reference" build is actually being downloaded from the regular Maven Central repo, not built locally.

It's not obvious how different versions could result, since <version>3.6.0</version> is there now and was presumably also there in the original reference build.

Hypothesis: The bytecode differences in HelpMojo.class are due to some difference between the build environments used to build the local binary jar (tightly controlled) and the reference binary jar on Maven Central Repo (TTBOMK we know nothing about this).

[jensdietrich]

@wtwhite -- just to clarify -- differences you have observed when building locally on wtwhite-vuw-vm -- did you run multiple builds and compare results ?

[wtwhite]

@jensdietrich no, I just ran their rebuild.sh script once (and "fixing" it by manually setting JAVA_HOME and rerunning mvn). I thought (based on the paper) this would do multiple local builds itself, but AFAICT it actually just runs 1 local build, and compares that with what's on Maven Central repo.

I think that's what rebuild.sh does based on the wording [INFO] Checking against reference build from central... from #32 (comment). But I'm not 100% sure -- I tried looking for that message in their repo but could only find it in the docs, so I'm not sure what's actually generating it:

wtwhite@wtwhite-vuw-vm:~/code/reproducible-central$ git grep -n "Checking against"
doc/HISTORY.md:51:[INFO] Checking against reference build from central...

Just now I checked out a separate source tree into ~/code/reproducible-central-FRESH and built that the same way. It results in a binary jar that is identical to the first build:

wtwhite@wtwhite-vuw-vm:~/code/reproducible-central$ md5sum {.,~/code/reproducible-central-FRESH}/content/io/fabric8/docker-maven-plugin/buildcache/docker-maven-plugin/target/docker-maven-plugin-0.36.0.jar 
45e8bc0e51f1bf598eff3d64b1d09f75  ./content/io/fabric8/docker-maven-plugin/buildcache/docker-maven-plugin/target/docker-maven-plugin-0.36.0.jar
45e8bc0e51f1bf598eff3d64b1d09f75  /home/wtwhite/code/reproducible-central-FRESH/content/io/fabric8/docker-maven-plugin/buildcache/docker-maven-plugin/target/docker-maven-plugin-0.36.0.jar

(I don't consider that definitive -- maybe seeing no differences in 100 such builds would be, but right now that requires me typing export JAVA_HOME=... 100x.)

[wtwhite]

@jensdietrich I just ran the build another 99x (inside a loop in the original docker container, so admittedly not as independent as separate containers would be) and found no differences in the resulting built jars:

 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ for ((i = 1; i < 100; ++i)); do echo $i; mkdir -p builds/$i; mvn -Prelease clean package -DskipTests -Dmaven.javadoc.skip -Dgpg.skip -V -e org.apache.maven.plugins:maven-artifact-plugin:3.5.0:compare -Dbuildinfo.reproducible -Dcompare.fail=false ; mv target/docker-maven-plugin-0.36.0.jar builds/$i/; done
--snip--
 wtwhite@[Reproducible Builder <JDK 11.0.20.1>]:~/app {(v0.36.0)} ]
$ md5sum builds/*/docker-maven-plugin-0.36.0.jar|cut -d' ' -f1|uniq -c
     99 45e8bc0e51f1bf598eff3d64b1d09f75

Based on that I would conclude that there is no run-to-run variabiliity for building docker-maven-plugin:0.36.0.

Also I missed that the docs actually do say that rebuild.sh compares against Maven Central repo:

rebuild.sh script will use the build specification file (= .buildspec file) to choose a Docker image to rebuild the project and check output against Central Repository reference binaries.