8253332: ZGC: Make heap views reservation platform independent #236

stefank · 2020-09-18T07:51:28Z

ZVirtualMemoryManager::reserve_contiguous_platform tries to reserve three views of a given address range. The posix and windows versions are more or less duplicates, with calls to platform dependent versions of reserve/unreserve functions.

I'd like to clean this up in preparation of an alternative implementation for heap memory allocation on Windows.

I choose to prefix the OS dependent functions with os_. For consistency, initialize_os should have been renamed as well, but the plan is to change that in a separate patch that splits that function into two, so I skipped it for now.

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed

Issue

JDK-8253332: ZGC: Make heap views reservation platform independent

Reviewers

Aleksey Shipilev (@shipilev - Reviewer) ⚠️ Review applies to bb10b9c
Per Lidén (@pliden - Reviewer)

Download

$ git fetch https://git.openjdk.java.net/jdk pull/236/head:pull/236
$ git checkout pull/236

stefank · 2020-09-18T07:53:04Z

/label add hotspot-gc

bridgekeeper · 2020-09-18T07:53:41Z

👋 Welcome back stefank! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2020-09-18T07:54:25Z

@stefank
The hotspot-gc label was successfully added.

openjdk · 2020-09-18T07:55:10Z

@stefank The following label will be automatically applied to this pull request: hotspot.

When this pull request is ready to be reviewed, an RFR email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label (add|remove) "label" command.

mlbridge · 2020-09-18T07:59:44Z

Webrevs

shipilev

Drive-by review below.

shipilev · 2020-09-18T08:20:22Z

src/hotspot/os/posix/gc/z/zVirtualMemory_posix.cpp

    // Failed to reserve memory at the requested address
-    unmap((uintptr_t)res, size);
-    return false;
+    os_unreserve(res, size);


A bit odd to see the assymetry here: mmap is used above, so munmap would be expected here? I see os_unreserve does the same thing, but does it guaranteed to do so?

Yes. I agree. Will fix.

shipilev · 2020-09-18T08:20:22Z

src/hotspot/share/gc/z/zVirtualMemory.cpp

+  }
+
+  // Register address views with native memory tracker
+  nmt_reserve(ZAddress::marked0(start), size);


The first arguments are the same as locals marked0, marked1, remapped above, right? Can be reused then?

Yes. At one point I had this part extracted into a separate function and had to re-resolve those values.

stefank · 2020-09-18T08:37:04Z

This was meant to only be reviewed on hotspot-gc-dev. Removing hotspot-dev.

stefank · 2020-09-18T08:37:17Z

/label remove hotspot

openjdk · 2020-09-18T08:38:16Z

@stefank
The hotspot label was successfully removed.

shipilev

Looks fine to me.

shipilev · 2020-09-18T08:40:47Z

src/hotspot/os/posix/gc/z/zVirtualMemory_posix.cpp

    // Failed to reserve memory at the requested address
-    unmap((uintptr_t)res, size);
-    return false;
+    munmap((void*)res, size);


No assert for munmap result? I don't care either way, but it would probably be nice to capture this.

Also, hold on a sec. Shouldn't this path return 0 too? Otherwise callers get the non-zero address that is effectively unusable.

For this particular function, the calling code is responsible for dealing with that. That is, I adopted the style used by the windows reserve_contiguous_platform, instead of using the posix implementation. But it's probably nicer to not leak out that address unnecessarily. I'll change this to the posix version instead.

openjdk · 2020-09-18T08:41:36Z

@stefank This change now passes all automated pre-integration checks. In addition to the automated checks, the change must also fulfill all project specific requirements

After integration, the commit message will be:

8253332: ZGC: Make heap views reservation platform independent

Reviewed-by: shade, pliden

If you would like to add a summary, use the /summary command.
To credit additional contributors, use the /contributor command.
To add additional solved issues, use the /issue command.

Since the source branch of this PR was last updated there have been 26 commits pushed to the master branch:

dad6edb: 8253321: java.util.Locale.LanguageRange#equals is inconsistent after calling hashCode
edc14f9: 8253286: Use expand_exact() instead of expand_at() for fixed requests in G1
bba948f: 8253411: [BACKOUT] [REDO] G1 incorrectly limiting young gen size when using the reserve can result in repeated full gcs
955c2e6: 8253303: G1: Move static initialization of G1FromCardCache to a proper location
34ec1be: 8252104: parallel heap inspection for ShenandoahHeap
fdce055: 8253253: Binutils tar ball extension update to gz
388c8f2: 8253348: Remove unimplemented JNIHandles::initialize
bca9e55: 8253167: ARM32 builds fail after JDK-8247910
cc7521c: 8252199: Reimplement support of Type 1 fonts without MappedByteBuffer
3d88d38: 8252070: Some platform-specific BLIT optimizations are not effective
... and 16 more: https://git.openjdk.java.net/jdk/compare/73c9088b81775c579789d96069c2f59364841e0b...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid automatic rebasing, please merge master into your branch, and then specify the current head hash when integrating, like this: /integrate dad6edbf83ca38cf9534e519629eee9847a54645.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

stefank · 2020-09-18T09:22:30Z

Thanks @shipilev. I went ahead and updated the code to not leak out the bad address, fixed a parameter name, and added a windows assert.

shipilev

Still look fine.

pliden · 2020-09-18T15:14:25Z

I choose to prefix the OS dependent functions with os_. For consistency, initialize_os should have been renamed as well, but the plan is to change that in a separate patch that splits that function into two, so I skipped it for now.

May I suggest that we instead just keep to the convention we have (here and in other classes) and use an _os suffix on these new functions?

pliden · 2020-09-18T15:21:28Z

src/hotspot/share/gc/z/zVirtualMemory.cpp

+    // Make the address range free
+    _manager.free(start, size);


I think we can just move these two lines to the end of reserve_contiguous_inner(), and remove this function.

Good point.

stefank · 2020-09-21T06:17:10Z

I choose to prefix the OS dependent functions with os_. For consistency, initialize_os should have been renamed as well, but the plan is to change that in a separate patch that splits that function into two, so I skipped it for now.

May I suggest that we instead just keep to the convention we have (here and in other classes) and use an _os suffix on these new functions?

There's no strong convention that we postfix with os. During the Windows port I introduced two usages, both named initialize_os. Now when more OS specific code is added, and I need to split initialize_os into two function, the postfix makes it less clear what the OS-specific API extension points are.

Some of my motivations for going with a prefix instead:

HotSpot already does that in other places. See the pd_ functions. For example, pd_commit_memory vs commit_memory.
Splitting initialize_os wouldn't create an infix:
os_initialize_before_reserve
os_initialize_after_reserve
vs
initialize_os_before_reserve
initialize_os_after_reserve
Easily recognized group of os specific funtions
os_initialize_before_reserve
os_initialize_after_reserve
os_reserve
os_unreserve
reserve_os sounded worse than os_reserve (to me)

I was considering splitting it out into a sub-interface, just to not have to deal with the *fix discussion/problem, but decided against it since the implementations needed member variables from ZVirtualMemoryManager.

pliden · 2020-09-21T09:58:49Z

@stefank That convention actually goes back to when ZGC was first integrated, with *_platform. However some of those *_platform functions has been removed over time as they were no longer needed. When the Windows port came in we said we should move away from *_platform to *_os, *_cpu and *_os_cpu to be more explicit and cater for potential future needs. To me, the suffix-style reads a bit better, the relationship between e.g. initialize() and initialize_os() becomes a bit more obvious, and it maps well to the platform specific file name convention we have in HotSpot (e.g. *_linux_x86.hpp). Having said that, I'm ok with changing the convention to prefix-style.

stefank · 2020-09-21T10:20:28Z

@stefank That convention actually goes back to when ZGC was first integrated, with *_platform. However some of those *_platform functions has been removed over time as they were no longer needed. When the Windows port came in we said we should move away from *_platform to *_os, *_cpu and *_os_cpu to be more explicit and cater for potential future needs. To me, the suffix-style reads a bit better, the relationship between e.g. initialize() and initialize_os() becomes a bit more obvious, and it maps well to the platform specific file name convention we have in HotSpot (e.g. *_linux_x86.hpp). Having said that, I'm ok with changing the convention to prefix-style.

About the last point: HotSpot code uses suffix for the platform file names, but the pd part is mostly a prefix.

Would you mind if I changed the few x_platform functions we have to pd_x functions? That way we don't invent our own way to mark platform-dependent code.

pliden · 2020-09-21T10:55:24Z

@stefank That would work for me.

stefank · 2020-09-21T12:46:19Z

/integrate

openjdk · 2020-09-21T12:48:42Z

@stefank Since your change was applied there have been 26 commits pushed to the master branch:

dad6edb: 8253321: java.util.Locale.LanguageRange#equals is inconsistent after calling hashCode
edc14f9: 8253286: Use expand_exact() instead of expand_at() for fixed requests in G1
bba948f: 8253411: [BACKOUT] [REDO] G1 incorrectly limiting young gen size when using the reserve can result in repeated full gcs
955c2e6: 8253303: G1: Move static initialization of G1FromCardCache to a proper location
34ec1be: 8252104: parallel heap inspection for ShenandoahHeap
fdce055: 8253253: Binutils tar ball extension update to gz
388c8f2: 8253348: Remove unimplemented JNIHandles::initialize
bca9e55: 8253167: ARM32 builds fail after JDK-8247910
cc7521c: 8252199: Reimplement support of Type 1 fonts without MappedByteBuffer
3d88d38: 8252070: Some platform-specific BLIT optimizations are not effective
... and 16 more: https://git.openjdk.java.net/jdk/compare/73c9088b81775c579789d96069c2f59364841e0b...master

Your commit was automatically rebased without conflicts.

Pushed as commit fbfb62d.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8253332: ZGC: Make heap views reservation platform independent

1a071cb

openjdk bot added the hotspot-gc label Sep 18, 2020

stefank marked this pull request as ready for review Sep 18, 2020

openjdk bot added hotspot rfr labels Sep 18, 2020

shipilev reviewed Sep 18, 2020

View changes

openjdk bot removed the hotspot label Sep 18, 2020

Review 1

2509713

shipilev approved these changes Sep 18, 2020

View changes

openjdk bot added the ready label Sep 18, 2020

Review 2

bb10b9c

shipilev approved these changes Sep 18, 2020

View changes

pliden reviewed Sep 18, 2020

View changes

Review 3

4830837

pliden approved these changes Sep 21, 2020

View changes

openjdk bot closed this Sep 21, 2020

openjdk bot added integrated and removed ready rfr labels Sep 21, 2020

stefank deleted the 8253332_unify_heap_view_reservation branch Sep 28, 2020

8253332: ZGC: Make heap views reservation platform independent #236

8253332: ZGC: Make heap views reservation platform independent #236

stefank commented Sep 18, 2020 •

edited by openjdk[bot] bot

stefank commented Sep 18, 2020

bridgekeeper bot commented Sep 18, 2020

openjdk bot commented Sep 18, 2020

openjdk bot commented Sep 18, 2020

mlbridge bot commented Sep 18, 2020 •

edited

shipilev left a comment

shipilev Sep 18, 2020

stefank Sep 18, 2020

shipilev Sep 18, 2020

stefank Sep 18, 2020

stefank commented Sep 18, 2020

stefank commented Sep 18, 2020

openjdk bot commented Sep 18, 2020

shipilev left a comment

shipilev Sep 18, 2020

shipilev Sep 18, 2020

stefank Sep 18, 2020

openjdk bot commented Sep 18, 2020 •

edited

stefank commented Sep 18, 2020

shipilev left a comment

pliden commented Sep 18, 2020

pliden Sep 18, 2020

stefank Sep 21, 2020

stefank commented Sep 21, 2020

pliden commented Sep 21, 2020

stefank commented Sep 21, 2020

pliden commented Sep 21, 2020

stefank commented Sep 21, 2020

openjdk bot commented Sep 21, 2020

8253332: ZGC: Make heap views reservation platform independent #236

8253332: ZGC: Make heap views reservation platform independent #236

Conversation

stefank commented Sep 18, 2020 • edited by openjdk[bot] bot

Progress

Issue

Reviewers

Download

stefank commented Sep 18, 2020

bridgekeeper bot commented Sep 18, 2020

openjdk bot commented Sep 18, 2020

openjdk bot commented Sep 18, 2020

mlbridge bot commented Sep 18, 2020 • edited

Webrevs

shipilev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stefank commented Sep 18, 2020

stefank commented Sep 18, 2020

openjdk bot commented Sep 18, 2020

shipilev left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openjdk bot commented Sep 18, 2020 • edited

stefank commented Sep 18, 2020

shipilev left a comment

pliden commented Sep 18, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stefank commented Sep 21, 2020

pliden commented Sep 21, 2020

stefank commented Sep 21, 2020

pliden commented Sep 21, 2020

stefank commented Sep 21, 2020

openjdk bot commented Sep 21, 2020

stefank commented Sep 18, 2020 •

edited by openjdk[bot] bot

mlbridge bot commented Sep 18, 2020 •

edited

openjdk bot commented Sep 18, 2020 •

edited