Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8327460: Compile tests with the same visibility rules as product code #18135

Closed
wants to merge 2 commits into from

Conversation

magicus
Copy link
Member

@magicus magicus commented Mar 6, 2024

Currently, our symbol visibility handling for tests are sloppy; we only handle it properly on Windows. We need to bring it up to the same levels as product code. This is a prerequisite for JDK-8327045, which in turn is a building block for Hermetic Java.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

  • JDK-8327460: Compile tests with the same visibility rules as product code (Bug - P3)

Reviewers

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/18135/head:pull/18135
$ git checkout pull/18135

Update a local copy of the PR:
$ git checkout pull/18135
$ git pull https://git.openjdk.org/jdk.git pull/18135/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 18135

View PR using the GUI difftool:
$ git pr show -t 18135

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/18135.diff

Webrev

Link to Webrev Comment

@bridgekeeper
Copy link

bridgekeeper bot commented Mar 6, 2024

👋 Welcome back ihse! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk
Copy link

openjdk bot commented Mar 6, 2024

@magicus The following labels will be automatically applied to this pull request:

  • build
  • core-libs
  • hotspot
  • shenandoah

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added build build-dev@openjdk.org hotspot hotspot-dev@openjdk.org core-libs core-libs-dev@openjdk.org shenandoah shenandoah-dev@openjdk.org labels Mar 6, 2024
@magicus magicus marked this pull request as ready for review March 6, 2024 12:00
@openjdk openjdk bot added the rfr Pull request is ready for review label Mar 6, 2024
@mlbridge
Copy link

mlbridge bot commented Mar 6, 2024

Webrevs

@magicus
Copy link
Member Author

magicus commented Mar 6, 2024

Most of these changes is just putting the EXPORT define in export.h instead (where support for __attribute__((visibility("default"))) is also added). However, there are a few other changes worth mentioning:

  • test/jdk/java/lang/Thread/jni/AttachCurrentThread/libImplicitAttach.c was missing an export. This had not been discovered before since that file was not compiled on Windows.
  • On macOS, the main function of a Java launcher needs to be exported. This is since on macOS, the main thread is reserved for Cocoa events, so we do not attach the main thread, but instead launch a new thread as the Java main thread, and re-run the mainfunction. (This is utterly weird behavior and I'm not sure this is properly documented by the JNI specification.)
  • Output referencing the dereference_null function from libTestDwarfHelper.h is checked in the Hotspot hs_err stack trace in test/hotspot/jtreg/runtime/ErrorHandling/TestDwarf.java. When compiling with -fvisibility=hidden, it became inlined and the test failed. I solved this by exporting the function, thus restoring the same situation as was before. I'm not sure this is the correct or best solution though, since it depends on what you expect TestDwarf.java to really achieve. (You can't expect of it to perform miracles and show functions that has been inlined in stack traces, though...)

@magicus magicus marked this pull request as draft March 6, 2024 12:14
@openjdk openjdk bot removed the rfr Pull request is ready for review label Mar 6, 2024
@magicus magicus marked this pull request as ready for review March 6, 2024 13:38
@openjdk openjdk bot added the rfr Pull request is ready for review label Mar 6, 2024
Copy link
Member

@erikj79 erikj79 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build system changes look good to me. The specific test changes probably needs other reviewers.

I assume you have run all the affected tests on a wide variety of platforms?

/reviewers 2

@openjdk
Copy link

openjdk bot commented Mar 6, 2024

@magicus This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8327460: Compile tests with the same visibility rules as product code

Reviewed-by: erikj, jvernee, dholmes, alanb

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 46 new commits pushed to the master branch:

  • d451f81: 8326381: com.sun.net.httpserver.HttpsParameters and SSLStreams incorrectly handle needClientAuth and wantClientAuth
  • 243cb09: 8327389: Remove use of HOTSPOT_BUILD_USER
  • 2d4c757: 8327041: Incorrect lane size references in avx512 instructions.
  • 761ed25: 8327138: Clean up status management in cdsConfig.hpp and CDS.java
  • 53628f2: 8327492: Remove applet usage and update DisposeInActionEventTest.html
  • 2627470: 8325567: jspawnhelper without args fails with segfault
  • a6dc4bc: 8326332: Unclosed inline tags cause misalignment in summary tables
  • 33aa4b2: 8256314: JVM TI GetCurrentContendedMonitor is implemented incorrectly
  • 87b40c6: 8327167: Clarify the handling of Leap year by Calendar
  • 6efdaf8: 8326908: DecimalFormat::toPattern throws OutOfMemoryError when pattern is empty string
  • ... and 36 more: https://git.openjdk.org/jdk/compare/fbb422ece7ff61bc10ebafe48ecb7f17ea315682...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Mar 6, 2024
@openjdk
Copy link

openjdk bot commented Mar 6, 2024

@erikj79
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

@openjdk openjdk bot removed the ready Pull request is ready to be integrated label Mar 6, 2024
@magicus
Copy link
Member Author

magicus commented Mar 6, 2024

I have run the tests in tier1, tier2 and tier3 on the Oracle internal CI, which includes linux/x64, linux/aarch64, windows/x64, macosx/x64 and macosx/aarch64.

I am currently running tier 4-10; this will take a while (and I won't integrate until this has finished running). When running multiple high-tier tests like this, the likelihood that I encounter intermittent problems increase. I will ignore test failures that seem likely to be intermittent and not caused by this patch. I might be overdoing the testing, but since this change affects all native tests, I want to be absolutely sure I don't break any tests that only execute as part of a high tier.

@magicus
Copy link
Member Author

magicus commented Mar 8, 2024

I am currently running tier 4-10

This is now done. A handful of tests failed, but all are due to transient environmental problems, known errors, or seemingly intermittent product errors -- none are due to symbol visibility.

So I'd argue that this is one of the most well-tested PRs posted ever. :-)

@magicus
Copy link
Member Author

magicus commented Mar 8, 2024

@JornVernee Are you okay with this solution? No JNI in foreign tests.

Copy link
Member

@JornVernee JornVernee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this looks good to me.

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Mar 8, 2024
Copy link
Member

@dholmes-ora dholmes-ora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a nice cleanup - great to see all the duplicated EXPORT code gone!

@@ -21,9 +21,11 @@
* questions.
*/

#include "jni.h"
#include <stdio.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems unneeded.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what I did here was change:

#include "jni.h"
#include <stdio.h>

into

#include <stdio.h>

#include "export.h"
#include "jni.h"

The reordering was strictly not needed, but putting user includes in front of system ones looked like bad coding to me, and put me in a difficult spot in where to add the new #include "export.h" -- next to the current user include even if I thought that was wrong, or have the system includes "sandwitched" between two user includes.

Or do you mean that it seems unneeded to include jni.h at all? Yes, I agree, but it was there before, and I don't want to make any other changes to the tests. This change is scary enough as it is :-). If you want, I can file a follow-up to remove this include instead.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I meant the include is actually unnecessary, but fine to not make that change here.

#include <stdio.h>

#include "export.h"
#include "jni.h"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems unneeded.

@AlanBateman
Copy link
Contributor

  • test/jdk/java/lang/Thread/jni/AttachCurrentThread/libImplicitAttach.c was missing an export. This had not been discovered before since that file was not compiled on Windows.

It's a Linux/macOS specific test so it wasn't needed.

Copy link
Contributor

@AlanBateman AlanBateman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good cleanup.

@magicus
Copy link
Member Author

magicus commented Mar 12, 2024

test/jdk/java/lang/Thread/jni/AttachCurrentThread/libImplicitAttach.c was missing an export. This had not been discovered before since that file was not compiled on Windows.

It's a Linux/macOS specific test so it wasn't needed.

Yeah. I understand. I just wanted to give an explanation for why this particular test needed changes that were not present elsewhere (since other exported methods all were tested on Windows).

@magicus
Copy link
Member Author

magicus commented Mar 12, 2024

Good cleanup.

Thanks!

@magicus
Copy link
Member Author

magicus commented Mar 13, 2024

/integrate

@openjdk
Copy link

openjdk bot commented Mar 13, 2024

Going to push as commit cc9a8ab.
Since your change was applied there have been 88 commits pushed to the master branch:

  • 3b18c5d: 8323605: Java source launcher should not require --source ... to enable preview
  • 5d4bfad: 8327693: C1: LIRGenerator::_instruction_for_operand is only read by assertion code
  • f3d0c45: 8327829: [JVMCI] runtime/ClassUnload/ConstantPoolDependsTest.java fails on libgraal
  • d5b95a0: 8327631: Update IANA Language Subtag Registry to Version 2024-03-07
  • 966a42f: 8324868: debug agent does not properly handle interrupts of a virtual thread
  • 22f10e0: 8327856: Convert applet test SpanishDiacriticsTest.java to a main program
  • 7283c8b: 8327972: Convert java/awt/FileDialog/SaveFileNameOverrideTest/SaveFileNameOverrideTest.html applet test to main
  • 30249c4: 8327838: Convert java/awt/FileDialog/MultipleMode/MultipleMode.html applet test to main
  • 94b4ed5: 8327096: (fc) java/nio/channels/FileChannel/Size.java fails on partition incapable of creating large files
  • b9c3dc3: 8327738: Remove unused internal method sun.n.w.p.h.HttpURLConnection.setDefaultAuthenticator
  • ... and 78 more: https://git.openjdk.org/jdk/compare/fbb422ece7ff61bc10ebafe48ecb7f17ea315682...master

Your commit was automatically rebased without conflicts.

@openjdk openjdk bot added the integrated Pull request has been integrated label Mar 13, 2024
@openjdk openjdk bot closed this Mar 13, 2024
@openjdk openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Mar 13, 2024
@openjdk
Copy link

openjdk bot commented Mar 13, 2024

@magicus Pushed as commit cc9a8ab.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@magicus magicus deleted the hidden-visibility-in-tests branch March 13, 2024 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build build-dev@openjdk.org core-libs core-libs-dev@openjdk.org hotspot hotspot-dev@openjdk.org integrated Pull request has been integrated shenandoah shenandoah-dev@openjdk.org
Development

Successfully merging this pull request may close these issues.

5 participants