EBR 1.3.0-SNAPSHOTS has regression issues due to port to Tycho 2.0.0

[nikhilnanivadekar]

Bugzilla: https://bugs.eclipse.org/bugs/show_bug.cgi?id=567399

2020-09-27T00:46:34.0289786Z 00:46:34:025 [ERROR] Failed to execute goal org.eclipse.ebr:ebr-maven-plugin:1.3.0-SNAPSHOT:bundle (default-bundle) on project org.eclipse.collections: Unable to execute mojo: Execution null of goal org.eclipse.ebr:ebr-tycho-extras-plugin:1.3.0-SNAPSHOT:assemble-bundle-p2-repository failed: An API incompatibility was encountered while executing org.eclipse.ebr:ebr-tycho-extras-plugin:1.3.0-SNAPSHOT:assemble-bundle-p2-repository: java.lang.NoSuchMethodError: 'void org.eclipse.tycho.core.ee.ExecutionEnvironmentConfigurationImpl.<init>(org.codehaus.plexus.logging.Logger, boolean, org.apache.maven.toolchain.ToolchainManager, org.apache.maven.execution.MavenSession)'
2020-09-27T00:46:34.0336708Z 00:46:34:028 [ERROR] -----------------------------------------------------
2020-09-27T00:46:34.0337674Z 00:46:34:028 [ERROR] realm =    plugin>org.eclipse.ebr:ebr-tycho-extras-plugin:1.3.0-SNAPSHOT
2020-09-27T00:46:34.0338906Z 00:46:34:029 [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
2020-09-27T00:46:34.0340653Z 00:46:34:029 [ERROR] urls[0] = file:/home/runner/.m2/repository/org/eclipse/ebr/ebr-tycho-extras-plugin/1.3.0-SNAPSHOT/ebr-tycho-extras-plugin-1.3.0-SNAPSHOT.jar
2020-09-27T00:46:34.0342722Z 00:46:34:029 [ERROR] urls[1] = file:/home/runner/.m2/repository/org/eclipse/ebr/ebr-maven-shared/1.3.0-SNAPSHOT/ebr-maven-shared-1.3.0-SNAPSHOT.jar
2020-09-27T00:46:34.0344130Z 00:46:34:029 [ERROR] urls[2] = file:/home/runner/.m2/repository/org/apache/commons/commons-lang3/3.10/commons-lang3-3.10.jar
2020-09-27T00:46:34.0345317Z 00:46:34:029 [ERROR] urls[3] = file:/home/runner/.m2/repository/org/apache/commons/commons-text/1.8/commons-text-1.8.jar
2020-09-27T00:46:34.0346440Z 00:46:34:029 [ERROR] urls[4] = file:/home/runner/.m2/repository/commons-io/commons-io/2.7/commons-io-2.7.jar
2020-09-27T00:46:34.0347863Z 00:46:34:029 [ERROR] urls[5] = file:/home/runner/.m2/repository/org/eclipse/tycho/tycho-p2-repository-plugin/2.0.0/tycho-p2-repository-plugin-2.0.0.jar
2020-09-27T00:46:34.0349317Z 00:46:34:030 [ERROR] urls[6] = file:/home/runner/.m2/repository/org/codehaus/plexus/plexus-utils/3.2.1/plexus-utils-3.2.1.jar
2020-09-27T00:46:34.0351293Z 00:46:34:030 [ERROR] urls[7] = file:/home/runner/.m2/repository/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.jar
2020-09-27T00:46:34.0353157Z 00:46:34:030 [ERROR] urls[8] = file:/home/runner/.m2/repository/org/codehaus/plexus/plexus-archiver/4.2.2/plexus-archiver-4.2.2.jar
2020-09-27T00:46:34.0354498Z 00:46:34:030 [ERROR] urls[9] = file:/home/runner/.m2/repository/org/codehaus/plexus/plexus-io/3.2.0/plexus-io-3.2.0.jar
2020-09-27T00:46:34.0355718Z 00:46:34:030 [ERROR] urls[10] = file:/home/runner/.m2/repository/org/apache/commons/commons-compress/1.20/commons-compress-1.20.jar
2020-09-27T00:46:34.0356862Z 00:46:34:030 [ERROR] urls[11] = file:/home/runner/.m2/repository/org/iq80/snappy/snappy/0.4/snappy-0.4.jar
2020-09-27T00:46:34.0357763Z 00:46:34:030 [ERROR] urls[12] = file:/home/runner/.m2/repository/org/tukaani/xz/1.8/xz-1.8.jar
2020-09-27T00:46:34.0360204Z 00:46:34:031 [ERROR] urls[13] = file:/home/runner/.m2/repository/org/eclipse/platform/org.eclipse.osgi.compatibility.state/1.1.800/org.eclipse.osgi.compatibility.state-1.1.800.jar
2020-09-27T00:46:34.0362049Z 00:46:34:031 [ERROR] urls[14] = file:/home/runner/.m2/repository/org/eclipse/platform/org.eclipse.osgi/3.15.300/org.eclipse.osgi-3.15.300.jar
2020-09-27T00:46:34.0362813Z 00:46:34:031 [ERROR] Number of foreign imports: 1
2020-09-27T00:46:34.0363677Z 00:46:34:031 [ERROR] import: Entry[import  from realm ClassRealm[project>org.eclipse.collections:p2-repository:11.0.0-SNAPSHOT, parent: ClassRealm[maven.api, parent: null]]]
2020-09-27T00:46:34.0364259Z 00:46:34:031 [ERROR] 
2020-09-27T00:46:34.0364586Z 00:46:34:031 [ERROR] -----------------------------------------------------
2020-09-27T00:46:34.0364898Z 00:46:34:031 [ERROR] -> [Help 1]
2020-09-27T00:46:34.0366311Z org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.eclipse.ebr:ebr-maven-plugin:1.3.0-SNAPSHOT:bundle (default-bundle) on project org.eclipse.collections: Unable to execute mojo

https://pipelines.actions.githubusercontent.com/JkyRYJ2TyKWwyWVC64gcjuCucC48E3LHwbAHn3mhzMN5NnNwUC/_apis/pipelines/1/runs/4476/signedlogcontent/3?urlExpires=2020-09-27T22%3A14%3A37.2435567Z&urlSigningMethod=HMACV1&urlSignature=rcRrLZvPeQxNhzbcoar03niBURjKkqu5zmm11jPRV7c%3D

[nikhilnanivadekar]

Tycho 2.0.0 is bumping up min requirement to JDK-11.
Bugzilla: https://bugs.eclipse.org/bugs/show_bug.cgi?id=567400
This might mean we will have to remove p2 publishing as it will break the compatibility of the project.

I am open to alternatives and workarounds if anyone has any ideas.

Cc: @fipro78 @guw @ujhelyiz

[fipro78]

@nikhilnanivadekar

AFAIK Tycho only requires Java 11 to run. That should not affect the build results. So if you are able to run your build with Java 11 and have the maven-compiler-plugin correctly configured to produce Java 8 compatible code, there should be no issue. Of coures the BREE in the MANIFEST needs to be verified and it should be tested in an environment executed with Java 8. But from my experience this should be ok.

[nikhilnanivadekar]

@fipro78
I would prefer not to do that. It adds maintenance work, we have to look out for regression breaks and it exposes a risk especially during release process.
That said, if you have an idea how to do it safely, I’ll be happy to try it out. I have never done a release build built using jdk-8 and jdk-11 in the same run. We can do the setup and create a milestone release.

[guw]

@nikhilnanivadekar Can you run the p2 repo creation in a separate process/step with its own JRE?

[nikhilnanivadekar]

@guw will need a bit of guidance and expertise here. Doesn’t the p2 repo require the EC jars to be available? The EC jars will have to be created using JDK 8 and then p2 repo will have to be created using JDK-11. The second part is where I am not following how it’ll come together?

Should we script something in a different repo, where the EC jars can be fetched from maven central and then bundled to a p2 repo? This will separate the project, process for p2 creation and actual jar creation.
Thoughts?

[fipro78]

@nikhilnanivadekar
IIRC right now the build is setup like this.

1. Maven build for Eclipse Collections (now containing correct MANIFEST @guw)
2. As part of this Maven build an EBR Maven module is triggered at the end of the process, that creates the bundles and a p2 update site containing only the bundles.

Actually you are not publishing that p2 update site anymore, as it did not contain features.

So someone created an additional build that consumes the temporary created p2 update site to create another p2 update site based on features. This build step is AFAIK separated already. And that result is published by you.

The question now is, do you still need the p2 repo build based on EBR at the end of the Eclipse Collections build, or would it work that the p2 repo build based on features can be executed by consuming the Eclipse Collections from Maven and not from a locally build artefact.

[nikhilnanivadekar]

@fipro78 I think there might be a disconnect in my understanding.
The module p2-feature is not registered in the main maven project. We only register p2-repository. p2-repository prepares the artifacts, which are picked up by the second step which uses the pom in p2-feature module. As you mentioned this step is actually the one which creates the p2 features which are pushed to the p2 repo.

The problem here is this, the first build p2-repository itself is failing, so it is not creating the artifacts. Now if it means that p2-repository does not need any Tycho dependencies, then we should clean up the module. But I am in no way an expert in this integration.
I regard all of you as experts in this part of integration, so will look forward to direction from you.

Cc: @guw @ujhelyiz @kthoms

[guw]

@nikhilnanivadekar Does the build still fail for you? The SNAPSHOT should be back to the old version as I reverted the Tycho 2.0.0 PR. We'll revisit this one, potentially for a different stream/version of the EBR plug-ins.

https://ci.eclipse.org/ebr/job/ebr-maven-plugins/422/console

[nikhilnanivadekar]

@guw triggered our pipelines. It will most likely succeed, as things were fine till the port to 2.0.0 was merged.
But in order to retain longevity of the project we should be able to upgrade the dependencies in a fairly trivial way. So, I am just looking at ways to minimize the impact for the eventual port of EBR to Tycho 2.0.0.

[guw]

Is EBR still required for building the OSGi MANIFEST.MF or can this be done as part of the jar build?

[nikhilnanivadekar]

@guw I dont know how to create the OSGi MANIFEST.MF any other way. If there are other ways, happy to change it. Would appreciate all the help I can get in this.

[fipro78]

@guw @nikhilnanivadekar the manifest is already generated in the jar build. I did this with my contributions lately. You should see an OSGi manifest now also in the Maven published artifacts

[guw]

You can always fall back to using the OSGi bundle manifest with the bnd Maven plug-in directly. The subsequent Tycho build could then work with pom dependencies consider.

As these are two reactors, each reactor can use a different JVM.

The first Maven build needs to "mvn install" artifacts into the local Maven repository (on the CI machine). The Tycho can then consume those artifacts directly from the build machine.

[prathasirisha]

@guw @nikhilnanivadekar can confirm the builds that were failing the last 2-3 days are working fine now.