8286582: Build fails on macos aarch64 when using --with-zlib=bundled #8651

jaikiran · 2022-05-11T11:38:31Z

Can I please get a review of this change which fixes build failures on macos when using --with-zlib=bundled?

With this change the build now passes (tested both with bundled and system zlib variants).

tier1, tier2 and tier3 testing has been done and no related failures have been noticed.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed (1 review required, with at least 1 reviewer)

Issue

JDK-8286582: Build fails on macos aarch64 when using --with-zlib=bundled

Reviewers

Magnus Ihse Bursie (@magicus - Reviewer)
Lance Andersen (@LanceAndersen - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/8651/head:pull/8651
$ git checkout pull/8651

Update a local copy of the PR:
$ git checkout pull/8651
$ git pull https://git.openjdk.java.net/jdk pull/8651/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 8651

View PR using the GUI difftool:
$ git pr show -t 8651

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/8651.diff

bridgekeeper · 2022-05-11T11:40:21Z

👋 Welcome back jpai! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2022-05-11T11:42:38Z

@jaikiran The following labels will be automatically applied to this pull request:

build
core-libs

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

mlbridge · 2022-05-11T11:46:00Z

Webrevs

01: Full - Incremental (081a7d34)
00: Full (eee12c25)

magicus · 2022-05-11T11:52:55Z

make/autoconf/lib-bundled.m4

@@ -216,7 +221,7 @@ AC_DEFUN_ONCE([LIB_SETUP_ZLIB],
  LIBZ_CFLAGS=""
  LIBZ_LIBS=""
  if test "x$USE_EXTERNAL_LIBZ" = "xfalse"; then
-    LIBZ_CFLAGS="$LIBZ_CFLAGS -I$TOPDIR/src/java.base/share/native/libzip/zlib"
+    LIBZ_CFLAGS="$LIBZ_CFLAGS $APPLE_LIBZ_CFLAGS -I$TOPDIR/src/java.base/share/native/libzip/zlib"


Declaring APPLE_LIBZ_CFLAGS far away (and only conditionally) and then using it once here just makes for hard-to-read code.

Suggested change

LIBZ_CFLAGS="$LIBZ_CFLAGS $APPLE_LIBZ_CFLAGS -I$TOPDIR/src/java.base/share/native/libzip/zlib"

LIBZ_CFLAGS="$LIBZ_CFLAGS -I$TOPDIR/src/java.base/share/native/libzip/zlib"

if test "x$OPENJDK_TARGET_OS" = xmacosx; then

LIBZ_CFLAGS="$LIBZ_CFLAGS -DHAVE_UNISTD_H"

fi

... and remove the assignment above.

Updated the PR to implement this suggestion

magicus · 2022-05-11T11:56:30Z

src/java.base/share/native/libzip/zlib/gzwrite.c

@@ -449,7 +449,14 @@ int ZEXPORTVA gzvprintf(gzFile file, const char *format, va_list va)
    (void)vsnprintf(next, state->size, format, va);
    len = strlen(next);
 #  else
-    len = vsnprintf(next, state->size, format, va);
+#   ifdef __APPLE__ // ignore format-nonliteral warning on macOS


Instead of patching 3rd party code to fix a compilation warning, you should disable that warning instead.

In make/modules/java.base/lib/CoreLibraries.gmk, add

DISABLED_WARNINGS_clang := format-nonliteral, \

as line 138.

Thank you for these useful inputs Magnus. I did these changes locally but for some reason this format-nonliteral is not getting picked up while building that library. I will investigate and see what's going on. Will update the PR once I figure it out.

I agree with Magnus and try to avoid changing the imported zlib code.

I did these changes locally but for some reason this format-nonliteral is not getting picked up while building that library.

Turns out that was slightly inaccurate. What was actually happening was that, that setting you suggested in that build file did indeed work and got picked up. But it ran into an error which looked like:

src/java.base/share/native/libzip/zlib/gzwrite.c:452:40: error: format string is not a string literal [-Werror,-Wformat-nonliteral] len = vsnprintf(next, state->size, format, va); ^~~~~~ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.1.sdk/usr/include/secure/_stdio.h:75:63: note: expanded from macro 'vsnprintf' __builtin___vsnprintf_chk (str, len, 0, __darwin_obsz(str), format, ap) ^~~~~~ 1 error generated. make[3]: *** [build/macosx-aarch64-server-release/support/native/java.desktop/libsplashscreen/gzwrite.o] Error 1

I hadn't paid close attention initially and upon seeing that error, I thought it was coming while building LIBZIP. However, what's actually happening is, it appears that while building LIBSPLASHSCREEN if bundled zlib is used then the build process recompiles the bundled zlib, so I had to include that setting even in the Awt2dLibraries.gmk.

I have now updated this PR with the changes you suggested and this additional change. Local build now works fine both for bundled and system zlib. I will trigger more extensive tests once these changes look OK.

I agree with Magnus and try to avoid changing the imported zlib code.

We already had modified gzwrite.c in our port so I thought keeping the changes narrowed to apple made sense here.

…ib instead of source code

…file

…bundled zlib

LanceAndersen

Hi Jai,

thank you for continuing the work to allow us to build/use the bundled zlib on macOS

we should also update: open/src/java.base/share/native/libzip/zlib/ChangeLog to add a comment regarding why the build changes were required

LanceAndersen · 2022-05-11T15:03:56Z

make/autoconf/lib-bundled.m4

@@ -217,6 +217,9 @@ AC_DEFUN_ONCE([LIB_SETUP_ZLIB],
  LIBZ_LIBS=""
  if test "x$USE_EXTERNAL_LIBZ" = "xfalse"; then
    LIBZ_CFLAGS="$LIBZ_CFLAGS -I$TOPDIR/src/java.base/share/native/libzip/zlib"
+    if test "x$OPENJDK_TARGET_OS" = xmacosx; then


Please add a comment here as to why we are doing this

@LanceAndersen Is that really needed? We normally don't comment on the reason why certain code needs certain defines. We tried to document some compiler flags in the beginning, but it turned out that most command lines ended up as essays, and this were not helpful. I think it's quite obvious what this code does: libz requires the define HAVE_UNISTD_H on macOS. I'm not even sure how you'd formulate a "why" -- "otherwise it does not work properly"?

@LanceAndersen Is that really needed? We normally don't comment on the reason why certain code needs certain defines. We tried to document some compiler flags in the beginning, but it turned out that most command lines ended up as essays, and this were not helpful. I think it's quite obvious what this code does: libz requires the define HAVE_UNISTD_H on macOS. I'm not even sure how you'd formulate a "why" -- "otherwise it does not work properly"?

The zlib configure script, which needs to be run prior running make to build the upstream repository, will determine if HAVE_UNISTD_H is needed and generate zconf.h replacing the macro with "1". My reason for adding a comment as this is a variant from how zlib is built upstream. Perhaps just updating open/src/java.base/share/native/libzip/zlib/ChangeLog is enough. I was just trying to document why we are doing this.

Another question would it make sense to set LIBZ_DISABLED_WARNINGS_CLANG in make/autoconf/lib-bundled.m4 so that the value in the case of zlib is set in one location? In addition, I can log a request to address this in the upstream project so we do not need to worry about this warning going forward.

It would not make sense to set the disabled warning in the configure script, no. The current code looks perfectly fine. Disabled warnings per module are set in the makefiles.

If you feel strongly that this needs to be documented more than in the JBS issue and this PR review, updating the zlib ChangeLog file is probably the way to go. I have no strong opinion on that. But from the build system PoV, the current code is fine as it is to be committed.

It would not make sense to set the disabled warning in the configure script, no. The current code looks perfectly fine. Disabled warnings per module are set in the makefiles.

OK, that you for your feedback regarding the makefile changes vs. configure script.

If you feel strongly that this needs to be documented more than in the JBS issue and this PR review, updating the zlib ChangeLog file is probably the way to go. I have no strong opinion on that. But from the build system PoV, the current code is fine as it is to be committed.

OK, I am probably overthinking the need to document this

LanceAndersen · 2022-05-11T15:04:41Z

make/modules/java.base/lib/CoreLibraries.gmk

@@ -135,6 +135,7 @@ $(eval $(call SetupJdkLibrary, BUILD_LIBZIP, \
        $(LIBZ_CFLAGS), \
    CFLAGS_unix := $(BUILD_LIBZIP_MMAP) -UDEBUG, \
    DISABLED_WARNINGS_gcc := unused-function implicit-fallthrough, \
+    DISABLED_WARNINGS_clang := format-nonliteral, \
    LDFLAGS := $(LDFLAGS_JDKLIB) \


A comment would be good here also as to the reasoning

And once again, we never comment on why we disable warnings. The context is clear enough -- there is a compiler warning that is not applicable to this module. Especially for 3rd party code, which is not nearly as stringent as we are about enabling warnings.

LanceAndersen · 2022-05-11T15:05:51Z

make/modules/java.desktop/lib/Awt2dLibraries.gmk

@@ -680,6 +680,7 @@ ifeq ($(ENABLE_HEADLESS_ONLY), false)

  ifeq ($(USE_EXTERNAL_LIBZ), false)
    LIBSPLASHSCREEN_EXTRA_SRC += java.base:libzip/zlib
+    LIBZ_DISABLED_WARNINGS_CLANG := format-nonliteral


Same here for a comment

magicus

Looks good to me.

openjdk · 2022-05-11T18:10:33Z

@jaikiran This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8286582: Build fails on macos aarch64 when using --with-zlib=bundled

Reviewed-by: ihse, lancea

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 15 new commits pushed to the master branch:

1586bf8: 8286401: Address possibly lossy conversions in Microbenchmarks
1c50ea3: 8282607: runtime/ErrorHandling/MachCodeFramesInErrorFile.java failed with "RuntimeException: 0 < 2"
fcf49f4: 8286441: Remove mode parameter from jdk.internal.perf.Perf.attach()
46a775a: 8286540: Build failure caused by missing DefaultPollerProvider implementation on AIX
4ad8cfa: 8286002: Add support for intel syntax to capstone hsdis
cec23b1: 8286556: Remove EagerInitialization develop option
89de756: 8286387: Remove unused FreeListAllocator::reduce_free_list
ae695d6: 8286476: x86_32: Fix crashes with non-preview mode after JDK-8284161 (Virtual Threads)
87f3d2b: 8286446: PPC64: fix crashes after JDK-8284161 (virtual threads preview)
7a2bbbb: 8286396: Address possibly lossy conversions in jdk.management.jfr
... and 5 more: https://git.openjdk.java.net/jdk/compare/73c5e993e17f7435553edae79a1e8d70ece5493d...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

LanceAndersen

Thanks Jai for the latest tweaks to the our original patch.

I think we should be good to go

jaikiran · 2022-05-12T02:49:29Z

Thank you Lance and Magnus for the reviews and inputs. I've triggered various build and test runs with this state of the PR. Once those complete satisfactorily, I'll go ahead and integrate this.

jaikiran · 2022-05-12T08:08:53Z

/integrate

openjdk · 2022-05-12T08:10:06Z

Going to push as commit 50d47de.
Since your change was applied there have been 23 commits pushed to the master branch:

89392fb: 8285820: C2: LCM prioritizes locally dependent CreateEx nodes over projections after 8270090
96d48f3: 8286433: Cache certificates decoded from TLS session tickets
7567627: 8286467: G1: Collection set pruning adds one region too many
82d2570: 8283001: windows-x86-cmp-baseline fails in some jvmti native libs
e9f45bb: 8282966: AArch64: Optimize VectorMask.toLong with SVE2
57a7670: 8285612: Remove jtreg tag manual=yesno for java/awt/print/PrinterJob/ImagePrinting/ClippedImages.java
44a60ce: 8285867: Convert applet manual tests SelectionVisible.java to Frame and automate
ccbe8fa: 8282772: JButton text set as HTML content has unwanted padding
1586bf8: 8286401: Address possibly lossy conversions in Microbenchmarks
1c50ea3: 8282607: runtime/ErrorHandling/MachCodeFramesInErrorFile.java failed with "RuntimeException: 0 < 2"
... and 13 more: https://git.openjdk.java.net/jdk/compare/73c5e993e17f7435553edae79a1e8d70ece5493d...master

Your commit was automatically rebased without conflicts.

openjdk · 2022-05-12T08:10:26Z

@jaikiran Pushed as commit 50d47de.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

fix build issues with bundled zlib on macosx

eee12c2

openjdk bot added the rfr Pull request is ready for review label May 11, 2022

openjdk bot added build build-dev@openjdk.org core-libs core-libs-dev@openjdk.org labels May 11, 2022

magicus suggested changes May 11, 2022

View reviewed changes

jaikiran added 4 commits May 11, 2022 17:52

Magnus' suggestion - Disable format-nonliteral in build section of zl…

89f8050

…ib instead of source code

Magnus' suggestion - make the LIBZ_CFLAGS more readable in the build …

1ab4674

…file

disable format-nonliteral warning when building LIBSPLASHSCREEN with …

ccb1edb

…bundled zlib

copyright years

081a7d3

LanceAndersen reviewed May 11, 2022

View reviewed changes

magicus approved these changes May 11, 2022

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label May 11, 2022

LanceAndersen approved these changes May 11, 2022

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label May 12, 2022

openjdk bot closed this May 12, 2022

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels May 12, 2022

jaikiran deleted the 8286582 branch May 12, 2022 08:10

8286582: Build fails on macos aarch64 when using --with-zlib=bundled #8651

8286582: Build fails on macos aarch64 when using --with-zlib=bundled #8651

Uh oh!

Conversation

jaikiran commented May 11, 2022 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Reviewing

Uh oh!

bridgekeeper bot commented May 11, 2022

Uh oh!

openjdk bot commented May 11, 2022

Uh oh!

mlbridge bot commented May 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jaikiran May 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jaikiran May 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LanceAndersen left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

magicus left a comment

Choose a reason for hiding this comment

Uh oh!

openjdk bot commented May 11, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LanceAndersen left a comment

Choose a reason for hiding this comment

Uh oh!

jaikiran commented May 12, 2022

Uh oh!

jaikiran commented May 12, 2022

Uh oh!

openjdk bot commented May 12, 2022

Uh oh!

openjdk bot commented May 12, 2022

Uh oh!

Uh oh!

jaikiran commented May 11, 2022 •

edited by openjdk bot

Loading

mlbridge bot commented May 11, 2022 •

edited

Loading

jaikiran May 11, 2022 •

edited

Loading

jaikiran May 11, 2022 •

edited

Loading

openjdk bot commented May 11, 2022 •

edited

Loading