Create a debug symbols-only package #2042

milderhc · 2020-08-28T19:04:39Z

Option to create debug symbols in a different archive than within the JDK one.

We can either create the JDK archive with or without the debug symbols but there isn't an option to create them separately.

smlambert · 2020-12-08T16:18:05Z

@milderhc - it looks like you have made great progress on this (with #2043 merged), thanks!

Are the next steps to:

update the build pipeline scripts, so that these artifacts are produced and pushed to the releases repo
served up via the API
visible on the website for download

Will you be taking that on? Do you need any support to move this forward?

milderhc · 2020-12-11T01:05:20Z

@smlambert Sure. I can take those.

Could you tell me where the pipeline scripts are? And also, the idea in the API would be a new method that serves the debug symbols exclusively, as well as in the website?

I'm not familiar with any of those repos, it would take a while.

smlambert · 2020-12-11T02:18:24Z

Thanks @milderhc! Pipeline scripts are in this repo. Though, as I look at one of the PRs that added support for debugimages for OpenJ9, I think a bunch of the work may already be in place... but defer to build team to give guidance.

@andrew-m-leonard @sxa - Would one of you be able to assist @milderhc ?
Is all that is needed to actually update the pipeline config to build the debug images?

API work: (work is done) AdoptOpenJDK/openjdk-api-v3#130
Website work: (still outstanding?) AdoptOpenJDK/openjdk-website#696 (this is less critical than being able to serve up debuginfo through the API)

andrew-m-leonard · 2020-12-11T08:40:28Z

Sure I can help. @smlambert @milderhc I wanted to check if the following bug I helped integrate into OpenJDK for jdk-16 is also relevant here?
https://bugs.openjdk.java.net/browse/JDK-8252233 Put debug symbols in symbols-image

smlambert · 2020-12-11T15:05:35Z

I would guess it is relevant @andrew-m-leonard - can you outline what, if any further changes are needed to produce these artifacts at the project (for hotspot builds)?

andrew-m-leonard · 2020-12-11T15:20:31Z

So the change https://bugs.openjdk.java.net/browse/JDK-8252233 enabled the production of a pure debug symbols image via the target:

make symbols-image

Which will result in a "build/*/images/symbols" image
The default "make product-images" which the Adopt builds use includes "symbols-image", so you should find we already have a symbols image folder being produced, we just need to zip it up... and archive.

smlambert · 2020-12-11T18:14:14Z

@andrew-m-leonard - is that make target only for jdk16+ or for all versions (is it possible for all versions, or what is the debuginfo artifact that should be produced for pre-jdk16)? perhaps you can point to the place in the code where one would make the change to enable this given #2042 (comment)

andrew-m-leonard · 2021-01-07T18:36:37Z

@milderhc hi Milder, i've been having a look at what upstream openjdk supports and currently they vary by version:
jdk16+ : With support in https://bugs.openjdk.java.net/browse/JDK-8252233 you can now simply issue "make symbols-image". The "symbols-image" is part of the "product-images" make target, and for jdk8+ Adopt already builds that target: https://github.com/AdoptOpenJDK/openjdk-build/blob/87f799024a58d392255eead1cd83d93515410b2e/configureBuild.sh#L206 . This means for jdk16+ we already build the "build/*/symbols/.." image, we just need to archive it at the end of the build, so this line needs to be moving the "symbols" folder: https://github.com/AdoptOpenJDK/openjdk-build/blob/88c0d77255264cc92423d534de9c3563d4fb0903/sbin/build.sh#L632 ie."${BUILD_CONFIG[DEBUG_IMAGE_PATH]}" for jdk16+ needs to be "symbols".

jdk11u & jdk8u: So both of these support make "symbols-image", however they do not have JDK-8252233, and so it does not produce an image bundle, but just places the symbols in the jdk image "lib" folder. This means we have two options here:

Get JDK-8252233 backported and use the same method as jdk16+. This is probably ideal, but will take time...
Add logic to sbin/build.sh to move the symbols from lib to a new location, it looks like you have already done this? https://github.com/AdoptOpenJDK/openjdk-build/blob/88c0d77255264cc92423d534de9c3563d4fb0903/sbin/build.sh#L651 Does this work for jdk16+ ? if so then maybe ignore JDK-8252233 for the moment.

I see now you added a new build arg: --create-debug-symbols-package, so I think the question to ask is when do we want this set to true? always? if so it's just a matter of setting it in: https://github.com/AdoptOpenJDK/openjdk-build/blob/master/build-farm/make-adopt-build-farm.sh
If it only wants setting on specific platforms or versions then in here: https://github.com/AdoptOpenJDK/openjdk-build/tree/master/build-farm/platform-specific-configurations

andrew-m-leonard · 2021-01-07T18:38:57Z

@milderhc another important fact that needs evaluating before we enable --create-debug-symbols-package, is how much more space is the symbols archive going to add to our builds, for capacity planning? As this is going to add xMB per osbuild x variant x version... ?
@sxa fyi

milderhc · 2021-01-08T09:26:02Z

Hi @andrew-m-leonard, thanks for looking into this and for the detailed explanation.

The --create-debug-symbols-package option works in jdk8u, jdk11u, and jdk16+. We can include it directly in make-adopt-build-farm.sh which I believe is called by the build pipeline?

The space in MB for jdk11u in Linux is around 230MB and around 40MB for macOS and Windows. In jdk8u is around 100MB for LInux and 5MB for Windows. Sizes for jdk16 are pretty similar to jdk11u.

andrew-m-leonard · 2021-01-08T09:35:06Z

@milderhc hi, one quick question, what is the difference between the current openj9 "-debug-image" and the new "-debug-symbols" ?

andrew-m-leonard · 2021-01-08T09:44:17Z

isn't openj9 "debug-image" the same thing? https://github.com/ibmruntimes/openj9-openjdk-jdk11/blob/cf49ddabb9e46e90af63e85f03e3e1e71338f6a2/closed/DebugImage.gmk

https://github.com/AdoptOpenJDK/openjdk-build/blob/88c0d77255264cc92423d534de9c3563d4fb0903/sbin/build.sh#L410

andrew-m-leonard · 2021-01-08T09:44:59Z

it would make sense to call Hotspot one "debug-image" as well?

andrew-m-leonard · 2021-01-08T10:00:32Z

So some rough calculations on size:
Platforms that cache builds for each version:

arm : 5x230MB = +1.15Gb
pLinux : 5x230MB = +1.15Gb
mac : 5x10MB = +50MB

Platforms that do not cache currently:
AIX : ws cleared
zLinux : ws cleared
Win : only caches last build in tmp ws
xLinux : Docker
aarch64 : Docker

"master" node archive storage for all versions x platforms:

jdk8:
- 11 Linux = 1100MB
- 5 mac & win = 25MB
jdk11:
- 9 Linux = 2160MB
- 4 mac & win = 160MB
jdk15:
- 7 Linux = 1680MB
- 3 mac & win = 120MB
jdk16:
- 7 Linux =1680MB
- 4 mac & win = 160MB
jdk17:
- 7 Linux = 1680MB
- 4 mac & win = 160MB

So adding roughly an extra 1.1-2.3Gb increase in size of each new archived pipeline build on Jenkins "master"

@sxa fyi

milderhc · 2021-01-08T10:01:04Z

I was asking the same myself. I think they are the same. In fact, if we call the script for openj9 along with --with-debug-symbols-package we might get two packages. It makes sense to rename debug-symbols to debug-image as well.

andrew-m-leonard · 2021-01-08T10:01:38Z

I was asking the same myself. I think they are the same. In fact, if we call the script for openj9 along with --with-debug-symbols-package we might get two packages. It makes sense to rename debug-symbols to debug-image as well.

yes, agree

andrew-m-leonard · 2021-01-08T10:03:49Z

I was asking the same myself. I think they are the same. In fact, if we call the script for openj9 along with --with-debug-symbols-package we might get two packages. It makes sense to rename debug-symbols to debug-image as well.

So I think we need to update the logic so for openj9 it uses the current "debug-image" make target as it currently does, and for the other variants uses the new logic you have added to create their "debug-image".

milderhc · 2021-01-08T10:04:24Z

Ok, I will update that first.

andrew-m-leonard · 2021-01-14T15:03:03Z

@milderhc hi Milder, how is it going?

milderhc · 2021-01-19T17:01:18Z

@andrew-m-leonard hi, sorry for the late answer. I created the pull request: #2393

andrew-m-leonard · 2021-01-20T09:45:03Z

@milderhc thanks, that PR looks good. Once that is merged it's just a matter of:

Perform Jenkins infrastructure capacity evaluation/impact: Create a debug symbols-only package #2042 (comment)
Having completed (1), adding --create-debug-image if variant != "openj9" somewhere around here: https://github.com/AdoptOpenJDK/openjdk-build/blob/102237341c7f0737f0dd4dc57fcc7e9e3ffe3bd5/build-farm/make-adopt-build-farm.sh#L164

andrew-m-leonard · 2021-02-03T09:37:12Z

The build output cleaning PR is now merged, and needs to propagate across the build nodes to take affect. Currently the build nodes with the most marginal space are the 2 arm nodes, with spare space of 3Gb, 4Gb respectively as of now. Introducing at this at the moment would marginalize this 2 machines by about another 1.2Gb.
I will examine each to check for any dedundent usage and monitor nodes once the clean build output has propagated across them.

Having just examined build-scaleway-ubuntu1604-armv7-1, the build output cleaning should release 8Gb from it.

@sxa fyi

andrew-m-leonard · 2021-02-03T09:51:02Z

Submitted build pipelines for arm on jdk8/11/16/17
New arm node space:

build-scaleway-ubuntu1604-armv7-1 : 18Gb
build-scaleway-ubuntu1604-armv7-2 : 19Gb
So we've freed up at least 6Gb
This easily gives us enough headroom for debug-images.

andrew-m-leonard · 2021-02-03T16:38:24Z

@milderhc Would you like to propose a PR to enable --create-debug-image ?
https://github.com/AdoptOpenJDK/openjdk-build/blob/ecb4ce1182b079fb88cc77ce871e3ad1216e4612/build-farm/make-adopt-build-farm.sh#L163

milderhc · 2021-02-03T17:16:53Z

Thanks, @andrew-m-leonard, would that be enough for the debug-images to be downloadable?

milderhc · 2021-02-04T09:09:07Z

Here it is: #2443

andrew-m-leonard · 2021-02-25T14:11:46Z

Nightly builds for hotspot with debug-image's are now available from the website: https://adoptopenjdk.net/nightly.html?variant=openjdk8&jvmVariant=hotspot

andrew-m-leonard · 2021-02-25T14:13:39Z

Also via the API eg.: https://api.adoptopenjdk.net/v3/assets/feature_releases/8/ea?architecture=x64&heap_size=normal&image_type=debugimage&jvm_impl=hotspot&os=linux&page=0&page_size=10&project=jdk&sort_method=DEFAULT&sort_order=DESC&vendor=adoptopenjdk

andrew-m-leonard · 2021-02-25T14:14:25Z

Things remaining:

Upstream mac debug jdk8 patch
Upstream win debug jdk8 fix
Upstream AIX debug fix for all versions

andrew-m-leonard · 2021-05-24T08:26:54Z

AIX debug symbol contributions now merged into all upstream versions, just awaiting official build levels

sxa · 2021-07-12T17:55:26Z

@andrew-m-leonard Is this now present in the July versions?

andrew-m-leonard · 2021-07-12T19:35:11Z

@sxa yes, complete

lwahonen · 2023-04-17T12:26:44Z

Nightly builds for hotspot with debug-image's are now available from the website: https://adoptopenjdk.net/nightly.html?variant=openjdk8&jvmVariant=hotspot

The debug images don't seem to be available from this page anymore.

milderhc mentioned this issue Aug 28, 2020

Add option to create debug-symbols in a separate file #2043

Merged

karianna added this to TODO in temurin-build via automation Sep 5, 2020

karianna added the enhancement Issues that enhance the code or documentation of the repo in any way label Sep 5, 2020

karianna added this to the September 2020 milestone Sep 5, 2020

karianna moved this from TODO to In Progress in temurin-build Sep 5, 2020

karianna modified the milestones: September 2020, November 2020 Nov 3, 2020

karianna assigned milderhc Dec 8, 2020

karianna modified the milestones: November 2020, December 2020 Jan 3, 2021

smlambert added this to To do in Top Priorities via automation Jan 5, 2021

karianna moved this from To do to In progress in Top Priorities Jan 12, 2021

karianna modified the milestones: December 2020, February 2021 Feb 4, 2021

M-Davies linked a pull request Feb 18, 2021 that will close this issue

Add option to generate debug images #2443

Closed

andrew-m-leonard self-assigned this Mar 3, 2021

karianna modified the milestones: February 2021, March 2021 Mar 29, 2021

smlambert moved this from In progress to Reviewer approved in Top Priorities Apr 6, 2021

gdams removed this from Reviewer approved in Top Priorities Apr 27, 2021

karianna modified the milestones: March 2021, May 2021 May 24, 2021

karianna modified the milestones: May 2021, June 2021 Jun 6, 2021

andrew-m-leonard closed this as completed Jul 12, 2021

karianna mentioned this issue Apr 20, 2023

Show nightly debug builds adoptium/adoptium.net#1707

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a debug symbols-only package #2042

Create a debug symbols-only package #2042

milderhc commented Aug 28, 2020

smlambert commented Dec 8, 2020

milderhc commented Dec 11, 2020 •

edited

smlambert commented Dec 11, 2020 •

edited

andrew-m-leonard commented Dec 11, 2020 •

edited

smlambert commented Dec 11, 2020

andrew-m-leonard commented Dec 11, 2020

smlambert commented Dec 11, 2020

andrew-m-leonard commented Jan 7, 2021

andrew-m-leonard commented Jan 7, 2021

milderhc commented Jan 8, 2021

andrew-m-leonard commented Jan 8, 2021

andrew-m-leonard commented Jan 8, 2021 •

edited

andrew-m-leonard commented Jan 8, 2021

andrew-m-leonard commented Jan 8, 2021 •

edited

milderhc commented Jan 8, 2021

andrew-m-leonard commented Jan 8, 2021

andrew-m-leonard commented Jan 8, 2021 •

edited

milderhc commented Jan 8, 2021

andrew-m-leonard commented Jan 14, 2021

milderhc commented Jan 19, 2021

andrew-m-leonard commented Jan 20, 2021

andrew-m-leonard commented Feb 3, 2021 •

edited

andrew-m-leonard commented Feb 3, 2021 •

edited

andrew-m-leonard commented Feb 3, 2021 •

edited

milderhc commented Feb 3, 2021

milderhc commented Feb 4, 2021

andrew-m-leonard commented Feb 25, 2021

andrew-m-leonard commented Feb 25, 2021

andrew-m-leonard commented Feb 25, 2021 •

edited

andrew-m-leonard commented May 24, 2021

sxa commented Jul 12, 2021

andrew-m-leonard commented Jul 12, 2021

lwahonen commented Apr 17, 2023

Create a debug symbols-only package #2042

Create a debug symbols-only package #2042

Comments

milderhc commented Aug 28, 2020

smlambert commented Dec 8, 2020

milderhc commented Dec 11, 2020 • edited

smlambert commented Dec 11, 2020 • edited

andrew-m-leonard commented Dec 11, 2020 • edited

smlambert commented Dec 11, 2020

andrew-m-leonard commented Dec 11, 2020

smlambert commented Dec 11, 2020

andrew-m-leonard commented Jan 7, 2021

andrew-m-leonard commented Jan 7, 2021

milderhc commented Jan 8, 2021

andrew-m-leonard commented Jan 8, 2021

andrew-m-leonard commented Jan 8, 2021 • edited

andrew-m-leonard commented Jan 8, 2021

andrew-m-leonard commented Jan 8, 2021 • edited

milderhc commented Jan 8, 2021

andrew-m-leonard commented Jan 8, 2021

andrew-m-leonard commented Jan 8, 2021 • edited

milderhc commented Jan 8, 2021

andrew-m-leonard commented Jan 14, 2021

milderhc commented Jan 19, 2021

andrew-m-leonard commented Jan 20, 2021

andrew-m-leonard commented Feb 3, 2021 • edited

andrew-m-leonard commented Feb 3, 2021 • edited

andrew-m-leonard commented Feb 3, 2021 • edited

milderhc commented Feb 3, 2021

milderhc commented Feb 4, 2021

andrew-m-leonard commented Feb 25, 2021

andrew-m-leonard commented Feb 25, 2021

andrew-m-leonard commented Feb 25, 2021 • edited

andrew-m-leonard commented May 24, 2021

sxa commented Jul 12, 2021

andrew-m-leonard commented Jul 12, 2021

lwahonen commented Apr 17, 2023

milderhc commented Dec 11, 2020 •

edited

smlambert commented Dec 11, 2020 •

edited

andrew-m-leonard commented Dec 11, 2020 •

edited

andrew-m-leonard commented Jan 8, 2021 •

edited

andrew-m-leonard commented Jan 8, 2021 •

edited

andrew-m-leonard commented Jan 8, 2021 •

edited

andrew-m-leonard commented Feb 3, 2021 •

edited

andrew-m-leonard commented Feb 3, 2021 •

edited

andrew-m-leonard commented Feb 3, 2021 •

edited

andrew-m-leonard commented Feb 25, 2021 •

edited