Provide source archives as part of the releases

[jerboaa]

It would be great to have a "source" link on the releases page. This helps users figure out what changed and get access to sources of a particular release right away.

Related issues:
Build support: #2698
API: adoptium/api.adoptium.net#176

TODO:

• Enable the --create-source-archive in https://github.com/adoptium/ci-jenkins-pipelines
• Figure out a way how to publish/promote the source archive snapshot (I'd expect it would need some deduplication).
• Actually use the new API on the website and provide source links akin to the upstream releases page

[sxa]

NOTE: As per discussion earlier we should consider the impact of enabling this option on the space required on the jenkins server. I would suggest that we only produce the source archive on one platform (e.g. Linux/x64) and ship from that build only. Another option would be to have a separate "Create source bundle" job to pull from. The former seems a bit easier to me and utilises the existing options, albeit slightly less "clean" by having to do it on a particular source archive.

Given the above, we should also consider when / whether to include it when doing point releases (many of which are repackaging on a subset of platforms)

[jerboaa]

Enable --create-source-archive on Linux x64: adoptium/ci-jenkins-pipelines#198
Tag sources tarball as such in metadata: adoptium/ci-jenkins-pipelines#199

[jerboaa]

OK, both pipeline script updates have been shipped. Sources artefacts are being produced. But for some reason the publishing to Github step seems to miss uploading *sources-*.tar.gz files. Not sure why that is.

[sxa]

Looks like it's some issue in the release job. From looking at last nights JDK8 builds the source artefacts are being correcly copied back to the top level openjdk8-pipeline job but the release job hasn't published it to the expected location.

The log for the release job has lines that are "processing" the files, so it knows about them, they just aren't included in the "Uploading" phase

Processing OpenJDK8U-sources_2021-10-11-18-05.tar.gz
Processing OpenJDK8U-sources_2021-10-11-18-05.tar.gz.json
Processing OpenJDK8U-sources_2021-10-11-18-05.tar.gz.sha256.txt

There was nothing set as ARTEFACTS_TO_SKIP in that release job, so the failure to publish is presumably somewhere in https://github.com/adoptium/github-release-scripts/blob/master/sbin/Release.sh although at a quick glance the initial regex in there doesn't seem to include a match for the debugimage files which do get shipped.

At the end of this we'll need to add a pipelines FAQ item for "How to add a new artefact to the releases".

[jerboaa]

@sxa Thanks for the info. I don't have access to see any of those jobs so I'm somewhat flying blind.

My testing of Release.sh does include *-sources* artefacts in the $files. So the issue seems to be in the groovy code. And it indeed seems to be the case. This line seems to put files containing hotspot in the file name into the adopt group. For a sources artefact this doesn't match. Then this upload looking for the adopt group won't include the sources artefact.

[jerboaa]

Proposed PR for a fix of #2728 (comment):
adoptium/github-release-scripts#83

[sxa]

@sxa Thanks for the info. I don't have access to see any of those jobs so I'm somewhat flying blind.

Yep figured that might be the case so I tried to eliminate any issues in the bits that weren't publicly visible 👍
That fix looks like it makes sense, and backs up the need to have the "end to end" process for adding new artefacts documented somewhere!

I've re-run last nights JDK8 publish job (restricted) and it seems to have added the source artefact the the release at https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u-2021-10-12-04-24-beta

[jerboaa]

[... ] and backs up the need to have the "end to end" process for adding new artefacts documented somewhere!

Agreed.

I've re-run last nights JDK8 publish job (restricted) and it seems to have added the source artefact the the release at https://github.com/adoptium/temurin8-binaries/releases/tag/jdk8u-2021-10-12-04-24-beta

Great, thanks! Could you do the same for JDK 11/17 pipelines, please?

[jerboaa]

Final piece of this is getting a link added on the main page. Tracked with: adoptium/adoptium.net#213

[sxa]

Great, thanks! Could you do the same for JDK 11/17 pipelines, please?

Sorry, missed this first time round, but it seems to be there in openjdk11-pipeline (visible in the release) and it looks like the openjdk17-pipeline that's running at the moment has picked it up too. (That one's running slow due to AIX).

[jerboaa]

Final piece of this is getting a link added on the main page. Tracked with: adoptium/adoptium.net#213

This is now done. Closing this epic.