8311661: Resolve duplicate symbol of StringTable::StringTable with JDK static linking

[jianglizhou]

Move StringTable to 'hotspot_jvm' namespace.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8311661: Resolve duplicate symbol of StringTable::StringTable with JDK static linking (Enhancement - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/14808/head:pull/14808
$ git checkout pull/14808

Update a local copy of the PR:
$ git checkout pull/14808
$ git pull https://git.openjdk.org/jdk.git pull/14808/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 14808

View PR using the GUI difftool:
$ git pr show -t 14808

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/14808.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back jiangli! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@jianglizhou The following labels will be automatically applied to this pull request:

• graal
• hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing lists. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 03: Full - Incremental (db681ecf)
• 02: Full - Incremental (3385b743)
• 01: Full - Incremental (c31ad972)
• 00: Full (cb9a4a22)

[coleenp]

This seems fine with me. We don't seem to have a namespace name convention in the style guide (was expecting lower case). But this would be an appropriate name to move SymbolTable to also if need be.

[openjdk]

@jianglizhou This change is no longer ready for integration - check the PR body for details.

[jianglizhou]

This seems fine with me. We don't seem to have a namespace name convention in the style guide (was expecting lower case). But this would be an appropriate name to move SymbolTable to also if need be.

Thanks for the review, Coleen.

[JornVernee]

The JBS issue says: "That avoids having to fix the application or libraries code, which may not be practical." but I'm not sure how changing hotspot every time an application defines a conflicting symbol is more practical? This patch seems like a band-aid to make one particular application work. I think a more principled solution is needed that avoids symbol conflicts going forward, if we are serious about supporting static linking of hotspot.

For instance, the symbols that should be exported from hotspot are found in several files in make/data/hotspot-symbols. Is it possible to limit the symbols exported/visible in the static library to those symbols instead?

[coleenp]

How about the namespace being HotSpotClassfile (StringTable is in the classfile directory)

[jianglizhou]

@JornVernee thanks for looking into this.

The JBS issue says: "That avoids having to fix the application or libraries code, which may not be practical." but I'm not sure how changing hotspot every time an application defines a conflicting symbol is more practical? This patch seems like a band-aid to make one particular application work. I think a more principled solution is needed that avoids symbol conflicts going forward, if we are serious about supporting static linking of hotspot.

Changing the application or libraries code is not practical in this case as we don't have the full control of what applications or libraries may be linked with. In some cases, developers may not have access to the actual code or required insights to resolve the linkage failures. Fixing the hotspot code using namespace in this case resolves the issue from the source at once, which makes this as a more principled approach.

For instance, the symbols that should be exported from hotspot are found in several files in make/data/hotspot-symbols. Is it possible to limit the symbols exported/visible in the static library to those symbols instead?

The linking failure involved with duplicate symbol issue concerns internal linkage vs. global linkage, which is different from symbol exporting, IIUC. When duplicating symbol with global linkages from different compilation units linked together, it causes the linking failure. On the other hand, exporting a symbol putting the symbol in the export table, so the symbol can be dynamically looked up. Please see some detailed discussions in:

#13451 (comment)
#13451 (comment)
#13451 (comment)

[jianglizhou]

How about the namespace being HotSpotClassfile (StringTable is in the classfile directory)

HotSpotClassfile sounds ok.

I did some search and HotSpotClassfile is not used by anyone.

[JornVernee]

Changing the application or libraries code is not practical in this case as we don't have the full control of what applications or libraries may be linked with. In some cases, developers may not have access to the actual code or required insights to resolve the linkage failures. Fixing the hotspot code using namespace in this case resolves the issue from the source at once, which makes this as a more principled approach.

I'm not suggesting that the application code should be changed. I'm suggesting that a broader solution should be found that avoids this issue in the future, rather than having to patch hotspot again for the next application that comes along and defines a symbol that conflicts. That's what I mean by "principled". I.e. we should be able to explain how to avoid symbol conflicts when statically linking applications against hotspot. I don't think that telling users that, if they have a symbol conflict: "just file a bug report and we will patch hotspot to use a different symbol" is good enough.

For instance, we could put all of hotspot into a HotSpot namespace, and then document this name space name and require that applications that want to statically link against hotspot don't use the HotSpot name space. That would also avoid having to fix up uses of these symbols inside hotspot itself, since things in the same namespace can refer to each other already, without needing a qualifier.

The linking failure involved with duplicate symbol issue concerns internal linkage vs. global linkage, which is different from symbol exporting, IIUC. When duplicating symbol with global linkages from different compilation units linked together, it causes the linking failure. On the other hand, exporting a symbol putting the symbol in the export table, so the symbol can be dynamically looked up. Please see some detailed discussions in:

#13451 (comment) #13451 (comment) #13451 (comment)

Thanks for the pointers. So it sounds like limiting the symbols 'exported' by a static library is not really possible.

[jianglizhou]

I'm not suggesting that the application code should be changed. I'm suggesting that a broader solution should be found that avoids this issue in the future, rather than having to patch hotspot again for the next application that comes along and defines a symbol that conflicts. That's what I mean by "principled". I.e. we should be able to explain how to avoid symbol conflicts when statically linking applications against hotspot.

Thanks for the clarification, @JornVernee.

I don't think that telling users that, if they have a symbol conflict: "just file a bug report and we will patch hotspot to use a different symbol" is good enough.

Agreed.

For instance, we could put all of hotspot into a HotSpot namespace, and then document this name space name and require that applications that want to statically link against hotspot don't use the HotSpot name space. That would also avoid having to fix up uses of these symbols inside hotspot itself, since things in the same namespace can refer to each other already, without needing a qualifier.

I think using a hotspot special namespace can be a good general solution for resolving and avoiding any potential duplication symbol issue like this. The good news is that with our prototype and testing on JDK 11, we didn't find many duplicate symbol issues with hotspot code specifically. So far we only observed issues with:

• StringTable::StringTable
• Thread
• ProfileData

Thanks for the pointers. So it sounds like limiting the symbols 'exported' by a static library is not really possible.

Right. We've looked into following solutions for linkage failures due to symbol conflicts and each solution may be used in different cases when applicable:

• Changing to static function/variable, so the symbol has internal linkage (as suggested by @AlanBateman in earlier code review for symbol issues in JDK library code)
• Use namespace
• Symbol (function, variable) renaming

[jianglizhou]

How about the namespace being HotSpotClassfile (StringTable is in the classfile directory)

HotSpotClassfile sounds ok.

I did some search and HotSpotClassfile is not used by anyone.

I'll change the namespace from JavaClassFile to HotSpotClassFile as Coleen suggested.

[coleenp]

What solutions do you have to resolve "Thread" and "ProfileData" ?

[jianglizhou]

What solutions do you have to resolve "Thread" and "ProfileData" ?

For the Thread symbol issue, we simply #defined the symbol to HotspotBaseThread in globalDefinitions.hpp in our prototype on JDK 11, to avoid large delta. It would a large change (in terms of touched files and lines) with namespace change. I'll file a bug for the Thread symbol for discussion.

[jianglizhou]

What solutions do you have to resolve "Thread" and "ProfileData" ?

For the Thread symbol issue, we simply #defined the symbol to HotspotBaseThread in globalDefinitions.hpp in our prototype on JDK 11, to avoid large delta. It would a large change (in terms of touched files and lines) with namespace change. I'll file a bug for the Thread symbol for discussion.

https://bugs.openjdk.org/browse/JDK-8311846

[coleenp]

Ok, yeah, I'd hate for Thread to get a namespace:: prepended to the million uses of it. Maybe the same approach should be used for StringTable too, since they're solving the same problem. Is there an #if STATIC_LINK or something that surrounds this #define?

[iklam]

If there are only 3 problematic symbols, wouldn't it be easier to configure the JDK with

bash configure --with-extra-cxxflags='-DStringTable=HotSpotStringTable -DThread=HotSpotThread -DProfileData=HotSpotProfileData'

That way, you don't need to submit a new PR every time there's a new conflict.

[jianglizhou]

Ok, yeah, I'd hate for Thread to get a namespace:: prepended to the million uses of it. Maybe the same approach should be used for StringTable too, since they're solving the same problem.

#define renaming is a good workaround to avoid editing larger number of files and reducing integration/merge issues. Using namespace seems to be a better choice in the JDK mainline. We can also reduce the needed edits with using.

Is there an #if STATIC_LINK or something that surrounds this #define?

We are proposing removing #ifdef STATIC_BUILD in https://github.com/jianglizhou/jdk/tree/static-java (not complete). That allows building .so and .a from the same set of object files.

[jianglizhou]

If there are only 3 problematic symbols, wouldn't it be easier to configure the JDK with

bash configure --with-extra-cxxflags='-DStringTable=HotSpotStringTable -DThread=HotSpotThread -DProfileData=HotSpotProfileData'

That way, you don't need to submit a new PR every time there's a new conflict.

Thanks, I haven't explored that option. Seems to be a useful quick fix when experimenting with JDK static linking. As we are working towards a longer term and more general solution, resolving the issues in hotspot code seems to be better.

[coleenp]

When you add in Thread, the namespace solution seems a lot less attractive unless you can do "using namespace hotspot;" around all the code in the JVM without altering names everywhere. Having sets of different HotSpotX namespaces seems a lot less attractive in the code and isn't really more general. So I don't think this is a good change anymore.

Maybe there is a build option to wrap names as Ioi suggested or some global namespace wrapping. But then we'd just want to wrap the whole thing in namespace hotspot. This code was written before namespaces existed everywhere.

[coleenp]

I think I've changed my mind. Since static build is a build time issue, then maybe a source solution shouldn't be done for this.

[coleenp]

It does look strange since you'd expect SymbolTable to be in the same namespace.

[tstuefe]

Hi,

to prevent clashes like this, libraries that support static linking tend to define a global namespace or a common prefix, usually switchable via defines. IIRC sqlite does this, zlib, jemalloc... If we could pull such a thing off with a minimum of invasiveness, it would be a much more robust solution.

If we introduce such a namespace, the name could be very short and non-descriptive, and then be defined (or completely switched off by default) via compile option. That way one could even link two hotspots if one wanted (only one usuable at a time for other reasons).

If combined with "using" somewhere, maybe in precompiled.hpp, this solution may not be that invasive.

Cheers, Thomas

[iklam]

Hi,

to prevent clashes like this, libraries that support static linking tend to define a global namespace or a common prefix, usually switchable via defines. IIRC sqlite does this, zlib, jemalloc... If we could pull such a thing off with a minimum of invasiveness, it would be a much more robust solution.

Do the libraries that you mention just enclose every .cpp and .hpp files with namespace xxx { and }. Something like

// thread.hpp
#include <....>
#include <....>
#include <....>

namespace hotspot {   // <<< add
class Thread {....}
} // <<< add

That doesn't look too bad to me, as we just need to do this once, and it can (probably) be done with a script.

If we introduce such a namespace, the name could be very short and non-descriptive, and then be defined (or completely switched off by default) via compile option. That way one could even link two hotspots if one wanted (only one usuable at a time for other reasons).

If combined with "using" somewhere, maybe in precompiled.hpp, this solution may not be that invasive.

Are you suggesting something like this?

// precompiled.hpp
namespace hotspot {}
using Bar = hotspot::Bar;    

// bar.h, the "Bar" class is automatically put inside "hotspot" namespace.
class Bar {
  static int a() { return 0; }
};

I tried it but couldn't get it to work. The using must be put after the type of hotspot::Bar has already been declared.

[JornVernee]

using namespace hotspot; could be used I think:

using namespace ns-name ; | (5)

5. using-directive: From the point of view of unqualified name lookup of any name after a using-directive and until the end of the scope in which it appears, every name from ns-name is visible as if it were declared in the nearest enclosing namespace which contains both the using-directive and ns-name.

(From: https://en.cppreference.com/w/cpp/language/namespace)

But, if we put everything in the same hotspot namespace we shouldn't need to add hotspot:: qualifiers since things in the same namespace can already refer to each other:

namespace hotspot {
class Bar {
public:
  static int a() { return 0; }
};

int func(Bar* bar) { // Bar is unqualified here
  return bar->a();
}
}

[jianglizhou]

Hi,
to prevent clashes like this, libraries that support static linking tend to define a global namespace or a common prefix, usually switchable via defines. IIRC sqlite does this, zlib, jemalloc... If we could pull such a thing off with a minimum of invasiveness, it would be a much more robust solution.

Do the libraries that you mention just enclose every .cpp and .hpp files with namespace xxx { and }.

Both zlib and jemalloc achieve name space isolations via #define, although not in C++ namespace definition. I think that's what @tstuefe was referring to.

Something like

// thread.hpp
#include <....>
#include <....>
#include <....>

namespace hotspot { // <<< add
class Thread {....}
} // <<< add
That doesn't look too bad to me, as we just need to do this once, and it can (probably) be done with a script.

Global namespace (as @JornVernee has suggested) does sound attractive as long as we can do it without very invasive changes. It's also a more future-proof solution. I'll do some exploration as well.

[iklam]

Global namespace (as @JornVernee has suggested) does sound attractive as long as we can do it without very invasive changes. It's also a more future-proof solution. I'll do some exploration as well.

Something like this might work. We need to enclose all declarations inside namespace hotspot {...}, so all .hpp files need to be touched (which I think is OK).

We can add the using namespace hotspot in precompiled.hpp so we that can avoid modifying most .cpp files.

using namespace affects only the lookup of existing names. It doesn't affect the declaration of new types (like Bar in my example below).

// precompiled.hpp
namespace hotspot {}
using namespace hotspot;

// foo.hpp
namespace hotspot {
  class Foo {
    public:
    static int x();
  };
}


// bar.hpp
class Bar { // declares ::Bar, not hotspot::Bar
public:
  static int a() { return 2; }
};


// foo.cpp
int Foo::x() { return 1; } // defines Hotspot::Foo::x()

int main() {
  printf("%d %d\n", Foo::x(), hotspot::Foo::x());
  //printf("%d %d\n", Bar::a(), hotspot::Bar::x()); <-- this doesn't work
}  

[dean-long]

I think we already used -fvisibility=hidden to solve problems like this.

[jianglizhou]

I think we already used -fvisibility=hidden to solve problems like this.

Right, the stringTable.o was already compiled with -fvisibility=hidden for example. -fvisibility=hidden didn't resolve the duplicate symbol issue with static linking. We ran into the duplicate duplicate symbol linking failure when user code also defines the symbol and statically linked together with hotspot libjvm.a.

[dean-long]

Does passing -Wl,--exclude-libs,ALL to gcc work for static libs?

[jianglizhou]

Something like this might work. We need to enclose all declarations inside namespace hotspot {...}, so all .hpp files need to be touched (which I think is OK).

I just looked briefly. For .hpp files alone in hotspot dir, we have 2276. Although the number of files is large, I think the actual changes are trivial. Still want to make sure everyone is okay with the changes at this scale.

[iklam]

Something like this might work. We need to enclose all declarations inside namespace hotspot {...}, so all .hpp files need to be touched (which I think is OK).

I just looked briefly. For .hpp files alone in hotspot dir, we have 2276. Although the number of files is large, I think the actual changes are trivial. Still want to make sure everyone is okay with the changes at this scale.

If adding using hotspot in precompiled.hpp works, we can roll this out in stages. At the beginning, we can just put the three problematic classes (Thread, StringTable and ProfileData) in the hotspot namespace, so only a few files need to be changed.

[jianglizhou]

Does passing -Wl,--exclude-libs,ALL to gcc work for static libs?

I just did a quick test using gcc with testing code added to define StringTable related symbols outside libjvm.a. With -Wl,--exclude-libs,ALL, statically linking the launcher executable with libjvm.a and the testing code fails due to multiple definition of the related symbols. I guess that's expected as --exclude-libs is for controlling symbol exporting.

From https://linux.die.net/man/1/ld#:~:text=Specifying%20%22%2D%2Dexclude%2Dlibs%20ALL,and%20for%20ELF%20targeted%20ports.

--exclude-libs lib,lib,...
Specifies a list of archive libraries from which symbols should not be automatically exported. The library names may be delimited by commas or colons. Specifying "--exclude-libs ALL" excludes symbols in all archive libraries from automatic export. This option is available only for the i386 PE targeted port of the linker and for ELF targeted ports. For i386 PE , symbols explicitly listed in a .def file are still exported, regardless of this option. For ELF targeted ports, symbols affected by this option will be treated as hidden.

[iklam]

I found a way to hide the unwanted symbols in libjvm.a. This requires ld --relocatable and objcopy --keep-global-symbols=.... See the prototype here:

• https://github.com/iklam/tools/tree/main/misc/staticlib

So potentially we can do this completely in the makefiles, without adding namespaces to HotSpot.

[jianglizhou]

I found a way to hide the unwanted symbols in libjvm.a. This requires ld --relocatable and objcopy --keep-global-symbols=.... See the prototype here:

• https://github.com/iklam/tools/tree/main/misc/staticlib

So potentially we can do this completely in the makefiles, without adding namespaces to HotSpot.

Yeah, objcopy can be used to localize symbols. One of my colleague @cjmoon1 implemented symbol localizing for libfreetype.a and libharfbuzz.a for static linking issue. In some cases, user might want to link with a different version of the harfbuzz library than the version linked with the JDK code. Then multiple versions of the libraries could be linked together into the executable. That was a solution suggested by C++ experts and it worked. Doing partial linking that produces a single .o file simplifies the work of objcopy. This is not a very portable solution though.

[jianglizhou]

Something like this might work. We need to enclose all declarations inside namespace hotspot {...}, so all .hpp files need to be touched (which I think is OK).

I just looked briefly. For .hpp files alone in hotspot dir, we have 2276. Although the number of files is large, I think the actual changes are trivial. Still want to make sure everyone is okay with the changes at this scale.

If adding using hotspot in precompiled.hpp works, we can roll this out in stages. At the beginning, we can just put the three problematic classes (Thread, StringTable and ProfileData) in the hotspot namespace, so only a few files need to be changed.

Moving to the namespace incrementally seems to be reasonable to me. Will let others to chime in on this for their thoughts.

Add using <namespace> to precompiled.hpp does help avoid adding <namespace>:: in many places. We still need to put the implementation code inside namespace <namespace> { ...}, or use <namespace>::. I experimented with StringTable and only needed to edit stringTable.* and precompiled.hpp. I tested with and without --disable-precompiled-headers and both built ok.

C++ references that I read discourages using using directive. https://stackoverflow.com/questions/1452721/why-is-using-namespace-std-considered-bad-practice has some helpful information. For the namespace usages within hotspot, there is probably no concern with pollution. It seems eventually all code would be moved to the namespace then using would not be needed.

Thread change would be bigger. There are references like class Thread;. It seems to be a good idea to be handled with https://bugs.openjdk.org/browse/JDK-8311846 separately.

[jianglizhou]

Add using <namespace> to precompiled.hpp does help avoid adding <namespace>:: in many places. We still need to put the implementation code inside namespace <namespace> { ...}, or use <namespace>::. I experimented with StringTable and only needed to edit stringTable.* and precompiled.hpp. I tested with and without --disable-precompiled-headers and both built ok.

src/hotspot/share/classfile/javaClasses.hpp also needs change, otherwise it fails to build on windows.

I updated the PR with suggestions incorporated. Please take a look of the updated version, thanks.

[iklam]

Hi Jiangli,

Putting the HotSpot code in a namespace is a fundamental change that should be done in a JEP. The idea of using namespaces for HotSpot has been around for a while, but so far there hasn't been a strong motivation for doing it. Perhaps static linking would finally give us a go-ahead reason.

The JEP should discuss the goals, design choices, risks and alternatives. For example:

• Other than static linking, what other problems can we solve with namespaces? Knowing the other goals may help us in choosing a design.
• Do we want a single namespace, or multiple namespaces (one for GC, one for JIT, etc)
• Do we want to change over incrementally (as in this PR), or in a single step
• Do we want to enable namespaces optionally (e.g., only for static linking)?
• With namespaces, the debug symbols will be much bigger, and we could also run into issues with debuggers and other tools.
• There may be alternatives for static linking that may have less impact than namespaces.

The hotspot-dev mailing list would be the best place to have such discussions. Also, at this time, many OpenJDK developers who are interested on this topic are on vacation, so we probably need to wait till the end of the summer so everyone has a chance to chime in.

JEP may feel like a lot of process, but for this change, I think it's the best avenue for exploration.

Thanks
Ioi

[tstuefe]

Moving to the namespace incrementally seems to be reasonable to me. Will let others to chime in on this for their thoughts.

I also like the incremental namespace solution.

@iklam wrote:

The hotspot-dev mailing list would be the best place to have such discussions. Also, at this time, many OpenJDK developers who are interested on this topic are on vacation, so we probably need to wait till the end of the summer so everyone has a chance to chime in.

JEP may feel like a lot of process, but for this change, I think it's the best avenue for exploration.

I agree with @iklam that we should discuss this on hotspot-dev. It will affect the whole JDK and constitute a change that is visible from outside, so we should get this right.

I also think that making the hotspot linkable in a static way could be very useful.

[jianglizhou]

Hi Jiangli,

Putting the HotSpot code in a namespace is a fundamental change that should be done in a JEP. The idea of using namespaces for HotSpot has been around for a while, but so far there hasn't been a strong motivation for doing it. Perhaps static linking would finally give us a go-ahead reason.

The JEP should discuss the goals, design choices, risks and alternatives. For example:

• Other than static linking, what other problems can we solve with namespaces? Knowing the other goals may help us in choosing a design.
• Do we want a single namespace, or multiple namespaces (one for GC, one for JIT, etc)
• Do we want to change over incrementally (as in this PR), or in a single step
• Do we want to enable namespaces optionally (e.g., only for static linking)?
• With namespaces, the debug symbols will be much bigger, and we could also run into issues with debuggers and other tools.
• There may be alternatives for static linking that may have less impact than namespaces.

The hotspot-dev mailing list would be the best place to have such discussions. Also, at this time, many OpenJDK developers who are interested on this topic are on vacation, so we probably need to wait till the end of the summer so everyone has a chance to chime in.

JEP may feel like a lot of process, but for this change, I think it's the best avenue for exploration.

Thanks Ioi

Thanks for the response, @iklam. Going through the JEP for the hotspot namespace makes a lot of sense. Waiting until the end of summer also sounds good to me.

Maybe it's still early to ask the question, would you be willing to drive the JEP for the namespace changes? I'm happy to write up the JEP otherwise.

[iklam]

Maybe it's still early to ask the question, would you be willing to drive the JEP for the namespace changes? I'm happy to write up the JEP otherwise.

I'll be happy to facilitate the discussion for this topic, but I am really not an expert on C++ namespaces. Also, I think such a JEP should be led by someone who has an interest in putting HotSpot in namespaces.

[dholmes-ora]

I am concerned that something that JDK-8303796 claimed to be a small enhancement is in fact requiring a lot of changes in different areas of the codebase. There are already a number of "duplicate symbol definition" issues that have been filed and fixed in an ad-hoc manner in the JDK, which is the kind of whack-a-mole approach that we should not be taking. If static linking requires symbol isolation then a generally applicable approach to doing that should be investigated (in a separate project? Leyden?) and then brought to mainline via a JEP (not just for the hotspot aspect as currently suggested).

[jianglizhou]

I am concerned that something that JDK-8303796 claimed to be a small enhancement is in fact requiring a lot of changes in different areas of the codebase.

@dholmes-ora, sorry for the slow response, I was OOO. The fixes and enhancements for JDK-8303796 include:

• duplicate symbol fixes
• Make file changes to create the full set of static libraries
• Runtime fixes open sourced at https://github.com/jianglizhou/jdk/tree/static-java
  • Fix runtime crashes
  • Enhance builtin libraries lookup
  • Use runtime checks to detect JDK static build. Allow removing STATIC_BUILD macro and building static and dynamic libraries without rebuild .o files

The overall changes are not small. Individually each fix or enhancement specifically the symbol fix is relatively small and non-complicated (except the built-in libraries lookup changes). Handling them as separate bugs and enhancements seems to be reasonable.

The built-in library lookup and runtime changes may be suitable for a new JEP or putting in a Leyden project repo and propose later. It's built on top of the existing JEP 178: Statically-Linked JNI Libraries. To me, handling it with a new JEP as an enhancement on top of JEP 178 is reasonable. Any thoughts on that?

There are already a number of "duplicate symbol definition" issues that have been filed and fixed in an ad-hoc manner in the JDK, which is the kind of whack-a-mole approach that we should not be taking. If static linking requires symbol isolation then a generally applicable approach to doing that should be investigated (in a separate project? Leyden?) and then brought to mainline via a JEP (not just for the hotspot aspect as currently suggested).

The duplicated symbol issues consist three parts:

1. Symbol conflicts among the JDK native libraries and hotspot
   Each issue is different and has been fixed in an ad-hoc manner. All issues have fixed by bugs linked with JDK-8303796. With enabling static JDK build (not yet done currently), we will be able to detect and prevent introducing new symbol issues in the future.

2. Symbol conflicts between hotspot and user code
   This is discussed by the thread in this PR. Moving hotspot code to hotspot unique namespace as suggested by @JornVernee is a good general solution.

3. Symbol conflicts between JDK library and user code
   With our prototype we found about 5 of those. I think these are outliers. Either fixing them as individually bugs or putting them in a Leyden repo and proposing together seems ok. Since the number is small enough, fixing individually might be slightly cleaner. Please let me know your thoughts. The JNI code defines APIs with naming convention consisting Java package and class names. There is no symbol issue with those. To prevent future issues, following the same naming style for helper methods and variables in JDK native code would work.

[bridgekeeper]

@jianglizhou This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!

[bridgekeeper]

@jianglizhou This pull request has been inactive for more than 8 weeks and will now be automatically closed. If you would like to continue working on this pull request in the future, feel free to reopen it! This can be done using the /open pull request command.

[theRealAph]

I found a way to hide the unwanted symbols in libjvm.a. This requires ld --relocatable and objcopy --keep-global-symbols=.... See the prototype here:

• https://github.com/iklam/tools/tree/main/misc/staticlib

So potentially we can do this completely in the makefiles, without adding namespaces to HotSpot.

Yeah, objcopy can be used to localize symbols. One of my colleague @cjmoon1 implemented symbol localizing for libfreetype.a and libharfbuzz.a for static linking issue. In some cases, user might want to link with a different version of the harfbuzz library than the version linked with the JDK code. Then multiple versions of the libraries could be linked together into the executable. That was a solution suggested by C++ experts and it worked. Doing partial linking that produces a single .o file simplifies the work of objcopy. This is not a very portable solution though.

OK, but it is the right thing to do on Linux. If some other operating systems don't provide useful tools, that's on them.
I haven't checked, but I strongly suspect that LLVM can do it too, so all that remains is Windows, and maybe they can't have static linking (or maybe they have to use something like this PR) until the right tooling is provided.

If Windows really can't do it, that's no reason to burden systems that can. Namespaces are not a low-cost solution for developers.

[jianglizhou]

I found a way to hide the unwanted symbols in libjvm.a. This requires ld --relocatable and objcopy --keep-global-symbols=.... See the prototype here:

• https://github.com/iklam/tools/tree/main/misc/staticlib

So potentially we can do this completely in the makefiles, without adding namespaces to HotSpot.

Yeah, objcopy can be used to localize symbols. One of my colleague @cjmoon1 implemented symbol localizing for libfreetype.a and libharfbuzz.a for static linking issue. In some cases, user might want to link with a different version of the harfbuzz library than the version linked with the JDK code. Then multiple versions of the libraries could be linked together into the executable. That was a solution suggested by C++ experts and it worked. Doing partial linking that produces a single .o file simplifies the work of objcopy. This is not a very portable solution though.

Additional discussions in #17456 (comment) thread