8375359: Improve GC serviceability init staging

[shipilev]

As per docs, CollectedHeap::post_initialize is for doing inits that should be done before general allocations are accepted. Currently, serviceability layer initialized there via the call to CollectedHeap::initialize_serviceability. This bug shows that after CollectedHeap::post_initialize returns, the rest of the serviceability layer had not been fully initialized yet. But GC code might assume general allocation path works at that point! So if that allocation path reports anything to the serviceability layer, it could encounter not yet fully initialized serviceability support and fail.

This readily manifests in improved Epsilon stress test. Improved test fails pretty consistently on my machines, and serves as a good regression test.

So the fix is to peel off serviceability initialization from post-init stage, and do it separately, before going (and finishing) post_initialize. Conveniently, we are nearly there, and "just" need to touch up a few things in universe_post_init().

Additional testing:

• Linux x86_64 server fastdebug, gc/epsilon/TestInitAllocs.java, 10x
• Linux x86_64 server fastdebug, hotspot_gc

---

Progress

• Change must be properly reviewed (1 review required, with at least 1 Reviewer)
• Change must not contain extraneous whitespace
• Commit message must refer to an issue

Issue

• JDK-8375359: Improve GC serviceability init staging (Bug - P4)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/29254/head:pull/29254
$ git checkout pull/29254

Update a local copy of the PR:
$ git checkout pull/29254
$ git pull https://git.openjdk.org/jdk.git pull/29254/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 29254

View PR using the GUI difftool:
$ git pr show -t 29254

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/29254.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back shade! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

❗ This change is not yet ready to be integrated.
See the Progress checklist in the description for automated requirements.

[openjdk]

@shipilev The following label will be automatically applied to this pull request:

• hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

[mlbridge]

Webrevs

• 00: Full (9418a170)