Mirroring remote artifacts into local cache loses P2 artifact properties (eg PGP, maven info...)

[mbooth101]

I was investigating eclipse-linuxtools/org.eclipse.linuxtools.eclipse-build#3 and found this bug in Tycho.

It looks like when tycho is building the target platform, GPG signatures are dropped or not preserved when bundles are resolved from the upstream repo configured in the target platform.

Consider this target platform:

<target includeMode="feature" name="target-platform">
        <locations>
                <location includeAllPlatforms="false" includeConfigurePhase="true" includeMode="planner" includeSource="true" type="InstallableUnit">
                        <repository location="https://download.eclipse.org/eclipse/updates/4.23-I-builds/"/>
                        <unit id="org.mockito.mockito-core" version="0.0.0"/>

In the upstream repository at https://download.eclipse.org/eclipse/updates/4.23-I-builds/ the mockito bundle has these properties set in the artifacts.xml:

<artifact classifier="osgi.bundle" id="org.mockito.mockito-core" version="4.1.0">
    <properties size="7">
        <property name="...." value="...."/>
        <property name="pgp.signatures" value="-----BEGIN PGP SIGNATURE----- ...."/>
        <property name="pgp.publicKeys" value="-----BEGIN PGP PUBLIC KEY BLOCK----- ...."/>

When we look at the p2artifacts.xml in the tycho target platform cache in ~/.m2/repository/p2/osgi/.... we see these properties are missing:

<artifact classifier='osgi.bundle' id='org.mockito.mockito-core' version='4.1.0'/>

This means that tycho-generated p2 repositories will contain unsigned bundles when these bundles are resolved from the tycho target platform.

This affects some Eclipse Platform deliverables, for example in the test framework repo mockito and friends are unsigned causing Eclipse to prompt with the "Trust" dialog when you try to install it:

• https://download.eclipse.org/eclipse/downloads/drops4/I20220216-1800/eclipse-test-framework-I20220216-1800.zip
  I filed a bug about this with platform releng, see bug: The eclipse-test-framework deliverable contains unsigned bundles eclipse-platform/eclipse.platform.releng.aggregator#32

The eclipse-test-framework deliverable is generated using the tycho mirror mojo from p2-extras-plugin, so I wrote you some integration tests using the mirror mojo to demonstrate the problem. The integration tests demonstrate:

• how it works perfectly when you by-pass the target platform by specifying explicit source repos for the mirror mojo
• how it fails when you try to resolve the same bundle from the target platform

[laeubi]

Mirro mojo and target platform are two completely different things. Beside that PGP signatures artifacts are not signed-bundles!

So one should really distinguish all these cases. A target platform could for example also contain file based items that do not have a signature at all.

[mbooth101]

@laeubi Well, the overriding point is (and I hope the integration tests demonstate it) that tycho is generating artifact repos that contain bundles that are "untrusted" by Eclipse when they absolutely should be trustable.

[laeubi]

@laeubi Well, the overriding point is (and I hope the integration tests demonstate it) that tycho is generating artifact repos that contain bundles that are "untrusted" by Eclipse when they absolutely should be trustable.

Sure, I just wanted to note that this is specific at least to the IU location. The current "workflow" is that platform resigns bundles with an eclipse PGP key.

[mbooth101]

@laeubi Well, the overriding point is (and I hope the integration tests demonstate it) that tycho is generating artifact repos that contain bundles that are "untrusted" by Eclipse when they absolutely should be trustable.

Sure, I just wanted to note that this is specific at least to the IU location. The current "workflow" is that platform resigns bundles with an eclipse PGP key.

That is done, one assumes, for publication to maven central. That seems orthogonal to this use-case (generating and publishing p2 repositories.)

[laeubi]

@laeubi Well, the overriding point is (and I hope the integration tests demonstate it) that tycho is generating artifact repos that contain bundles that are "untrusted" by Eclipse when they absolutely should be trustable.

Sure, I just wanted to note that this is specific at least to the IU location. The current "workflow" is that platform resigns bundles with an eclipse PGP key.

That is done, one assumes, for publication to maven central. That seems orthogonal to this use-case (generating and publishing p2 repositories.)

Nope, it explicitly re-signs artifacts from maven-central (org.mockito.mockito-core is a maven-central artifact)

[mbooth101]

@laeubi Well, the overriding point is (and I hope the integration tests demonstate it) that tycho is generating artifact repos that contain bundles that are "untrusted" by Eclipse when they absolutely should be trustable.

Sure, I just wanted to note that this is specific at least to the IU location. The current "workflow" is that platform resigns bundles with an eclipse PGP key.

That is done, one assumes, for publication to maven central. That seems orthogonal to this use-case (generating and publishing p2 repositories.)

Nope, it explicitly re-signs artifacts from maven-central (org.mockito.mockito-core is a maven-central artifact)

These are the signatures that tycho is omitting :-)

[laeubi]

No these are additional new created signatures not the ones from maven.central. Anyways they seem missing regardless of their origin...

[mbooth101]

No these are additional new created signatures not the ones from maven.central. Anyways they seem missing regardless of their origin...

I actually think we are talking about the same thing.... Sorry if I wasn't clear in the original bug description and caused confusion.

[laeubi]

No problem, do you like to take a look at this? If not I can try to investigate a bit here in the next days. @mickaelistria @akurtakov should this be considered as a blocker for 1.7.0 release?

[mickaelistria]

should this be considered as a blocker for 1.7.0 release?

I don't think so. It's an annoyance for sure, but so far, no one complained and the use-case brought by @mbooth101 isn't too critical. However, it's a serious bug in Tycho that is worth being investigated with high prirority, particularly as we rely more and more on artifact metadata for various purposes. Once we have a fix, we can maybe consider it as important enough to backport to branch and release some 2.7.1, but let's first wait to have a fix ready before delaying an awaited release.

[laeubi]

I'm just curious as actually the properties are propagated at least for maven GAV properties ... maybe this is not a general problem but specific to the usage here.

[mickaelistria]

@laeubi have you verified both metadata and artifact properties?

[laeubi]

@laeubi have you verified both metadata and artifact properties?

No idea, have to recheck this I hardly work with the artifact metadata so there is a good chance I simply have missed this.

[laeubi]

I can at least confirm that at the Target level (URITargetDefinitionContent) the information is still present.

[laeubi]

The root cause is how we mirror remote artifacts here, I'll try to prepare a fix.

[laeubi]

Cherry picked to 2.7.0 as well