8326587: Separate out Microsoft toolchain linking

[magicus]

There is not much overlap on how linking is done on Windows on one hand, and on all Unix platforms on the other. This makes Link.gmk basically consists of two parts, each in it own half of if statements, and the few common parts are artificially shoehorned in to fit both sides.

The code will be much clearer if we just split this into two different files.

Note that this PR slightly collides with JDK-8326583 (#17986). Whichever of this goes in first will mean that the other one needs to make some adaptations.

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8326587: Separate out Microsoft toolchain linking (Bug - P3)

Reviewers

• Erik Joelsson (@erikj79 - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/17987/head:pull/17987
$ git checkout pull/17987

Update a local copy of the PR:
$ git checkout pull/17987
$ git pull https://git.openjdk.org/jdk.git pull/17987/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 17987

View PR using the GUI difftool:
$ git pr show -t 17987

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/17987.diff

Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back ihse! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@magicus The following label will be automatically applied to this pull request:

• build

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 03: Full - Incremental (1bdc1350)
• 02: Full - Incremental (afd1edd7)
• 01: Full (27500808)
• 00: Full (370a05b0)

[erikj79]

Is it really a good idea to split makefiles based on target platform? It creates a less flexible separation that makes experimentation with different combinations of toolchains and platforms much harder. We currently have a pretty well established mapping between target platform and toolchain type in practice, while trying to keep the concepts somewhat separated, but this kind of change really cements that mapping. Given Julian's ambition for making it possible to compile for Windows using a different toolchain, I'm not sure this is the right way to go.

[magicus]

I have verified that there is no differences in the resulting output using COMPARE_BUILD, for the platforms in Oracle's CI: windows-x64,linux-x64,linux-aarch64,macosx-x64,macosx-aarch64, confirming that this is a pure build system refactoring.

[magicus]

Ok, fair enough, I should have phrase this split as "Microsoft toolchain" and "everything else". The point is that linking is vastly different on Windows than on other platforms. Our efforts to force "lib" to behave as "ar" etc was far-fetched to begin with. And to fully support static libraries, we need to separate the static linking into several steps, running ld, objcopy and then finally ar There is no correspondence to of this at all with the Microsoft toolchain, and trying to shoehorn this in is just going to make matters worse.

I'll rename the split according to toolchain and not platform. (And then I think this will actually help Julian, since we'll push the microsoft strangeness away in a separate file.) Sounds OK to you then?

[erikj79]

Ok, fair enough, I should have phrase this split as "Microsoft toolchain" and "everything else". The point is that linking is vastly different on Windows than on other platforms. Our efforts to force "lib" to behave as "ar" etc was far-fetched to begin with. And to fully support static libraries, we need to separate the static linking into several steps, running ld, objcopy and then finally ar There is no correspondence to of this at all with the Microsoft toolchain, and trying to shoehorn this in is just going to make matters worse.

I'll rename the split according to toolchain and not platform. (And then I think this will actually help Julian, since we'll push the microsoft strangeness away in a separate file.) Sounds OK to you then?

That sounds good to me, thanks!

[erikj79]

I think this looks good. There looks to be a conflict with #17986, but I assume you are aware.

[TheShermanTanker]

And then I think this will actually help Julian, since we'll push the microsoft strangeness away in a separate file

Thanks! Though I do first need to rebase on top of it and fix all the merge conflicts first...
(Thinking about it, this likely means I have to excise some of the Microsoft linking logic out into the "regular" linking file since some of the Microsoft linking process is used by Windows linking in general, even with the gcc toolchain)

Just a comment: Why not reuse the AR variable for lib rather than introduce an entirely new LIB variable? The logic treating lib.exe as ar is gone with this change, but that doesn't mean they have to be split into entirely different variables. LinkMicrosoft could just run $($1_AR) without treating it as ar. This saves one autoconf and Makefile variable

[magicus]

Why not reuse the AR variable for lib rather than introduce an entirely new LIB variable?

The reason will become more apparent as I progress towards proper handling of static libs. In short, AR is not by far a 1-to-1 mapping with LIB. On non-Microsoft toolchains, we are basically going to do static linking by ld + objcopy + ar; on Windows, it's just lib. So there is nothing to be gained by pretending that lib is ar.

[magicus]

/integrate

[openjdk]

@magicus This pull request has not yet been marked as ready for integration.

[TheShermanTanker]

@magicus This pull request has not yet been marked as ready for integration.

That last second save is nothing short of impressive haha

[openjdk]

@magicus This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8326587: Separate out Microsoft toolchain linking

Reviewed-by: erikj

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 7 new commits pushed to the master branch:

• bb6b048: 8326099: GZIPOutputStream should use Deflater.getBytesRead() instead of Deflater.getTotalIn()
• 490825f: 8325589: C2 SuperWord refactoring: create VLoopAnalyzer with Submodules
• 3a00fc7: 8323698: Class use page does not include extends/implements type arguments
• f32f574: 8326575: Remove unused ContiguousSpace::set_top_for_allocations
• 09f755a: 8325857: G1 Full GC flushes mark stats cache too early
• 1799ffe: 8310351: Typo in ImmutableCollections
• d10f277: 8326006: Allow TEST_VM_FLAGLESS to set flagless mode

Please see this link for an up-to-date comparison between the source branch of this pull request and the master branch.
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

[magicus]

/integrate

[openjdk]

Going to push as commit 3780ad3.
Since your change was applied there have been 10 commits pushed to the master branch:

• 2d3c9c5: 8325553: Parallel: Use per-marker cache for marking stats during Full GC
• 20c71ce: 8321156: Improve the handling of invalid UTF-8 byte sequences for ZipInputStream::getNextEntry and ZipFile::getComment
• c042f08: 8326653: Remove jdk.internal.reflect.UTF8
• bb6b048: 8326099: GZIPOutputStream should use Deflater.getBytesRead() instead of Deflater.getTotalIn()
• 490825f: 8325589: C2 SuperWord refactoring: create VLoopAnalyzer with Submodules
• 3a00fc7: 8323698: Class use page does not include extends/implements type arguments
• f32f574: 8326575: Remove unused ContiguousSpace::set_top_for_allocations
• 09f755a: 8325857: G1 Full GC flushes mark stats cache too early
• 1799ffe: 8310351: Typo in ImmutableCollections
• d10f277: 8326006: Allow TEST_VM_FLAGLESS to set flagless mode

Your commit was automatically rebased without conflicts.

[openjdk]

@magicus Pushed as commit 3780ad3.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.