Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8253332: ZGC: Make heap views reservation platform independent #236

Closed

Conversation

stefank
Copy link
Member

@stefank stefank commented Sep 18, 2020

ZVirtualMemoryManager::reserve_contiguous_platform tries to reserve three views of a given address range. The posix and windows versions are more or less duplicates, with calls to platform dependent versions of reserve/unreserve functions.

I'd like to clean this up in preparation of an alternative implementation for heap memory allocation on Windows.

I choose to prefix the OS dependent functions with os_. For consistency, initialize_os should have been renamed as well, but the plan is to change that in a separate patch that splits that function into two, so I skipped it for now.


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8253332: ZGC: Make heap views reservation platform independent

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/236/head:pull/236
$ git checkout pull/236

@stefank
Copy link
Member Author

@stefank stefank commented Sep 18, 2020

/label add hotspot-gc

@bridgekeeper
Copy link

@bridgekeeper bridgekeeper bot commented Sep 18, 2020

👋 Welcome back stefank! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the hotspot-gc label Sep 18, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Sep 18, 2020

@stefank
The hotspot-gc label was successfully added.

@stefank stefank marked this pull request as ready for review Sep 18, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Sep 18, 2020

@stefank The following label will be automatically applied to this pull request: hotspot.

When this pull request is ready to be reviewed, an RFR email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label (add|remove) "label" command.

@openjdk openjdk bot added hotspot rfr labels Sep 18, 2020
@mlbridge
Copy link

@mlbridge mlbridge bot commented Sep 18, 2020

Webrevs

Copy link
Contributor

@shipilev shipilev left a comment

Drive-by review below.

// Failed to reserve memory at the requested address
unmap((uintptr_t)res, size);
return false;
os_unreserve(res, size);
Copy link
Contributor

@shipilev shipilev Sep 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A bit odd to see the assymetry here: mmap is used above, so munmap would be expected here? I see os_unreserve does the same thing, but does it guaranteed to do so?

Copy link
Member Author

@stefank stefank Sep 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I agree. Will fix.

}

// Register address views with native memory tracker
nmt_reserve(ZAddress::marked0(start), size);
Copy link
Contributor

@shipilev shipilev Sep 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first arguments are the same as locals marked0, marked1, remapped above, right? Can be reused then?

Copy link
Member Author

@stefank stefank Sep 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. At one point I had this part extracted into a separate function and had to re-resolve those values.

@stefank
Copy link
Member Author

@stefank stefank commented Sep 18, 2020

This was meant to only be reviewed on hotspot-gc-dev. Removing hotspot-dev.

@stefank
Copy link
Member Author

@stefank stefank commented Sep 18, 2020

/label remove hotspot

@openjdk openjdk bot removed the hotspot label Sep 18, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Sep 18, 2020

@stefank
The hotspot label was successfully removed.

Copy link
Contributor

@shipilev shipilev left a comment

Looks fine to me.

// Failed to reserve memory at the requested address
unmap((uintptr_t)res, size);
return false;
munmap((void*)res, size);
Copy link
Contributor

@shipilev shipilev Sep 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No assert for munmap result? I don't care either way, but it would probably be nice to capture this.

Copy link
Contributor

@shipilev shipilev Sep 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, hold on a sec. Shouldn't this path return 0 too? Otherwise callers get the non-zero address that is effectively unusable.

Copy link
Member Author

@stefank stefank Sep 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For this particular function, the calling code is responsible for dealing with that. That is, I adopted the style used by the windows reserve_contiguous_platform, instead of using the posix implementation. But it's probably nicer to not leak out that address unnecessarily. I'll change this to the posix version instead.

@openjdk
Copy link

@openjdk openjdk bot commented Sep 18, 2020

@stefank This change now passes all automated pre-integration checks. In addition to the automated checks, the change must also fulfill all project specific requirements

After integration, the commit message will be:

8253332: ZGC: Make heap views reservation platform independent

Reviewed-by: shade, pliden
  • If you would like to add a summary, use the /summary command.
  • To credit additional contributors, use the /contributor command.
  • To add additional solved issues, use the /issue command.

Since the source branch of this PR was last updated there have been 26 commits pushed to the master branch:

  • dad6edb: 8253321: java.util.Locale.LanguageRange#equals is inconsistent after calling hashCode
  • edc14f9: 8253286: Use expand_exact() instead of expand_at() for fixed requests in G1
  • bba948f: 8253411: [BACKOUT] [REDO] G1 incorrectly limiting young gen size when using the reserve can result in repeated full gcs
  • 955c2e6: 8253303: G1: Move static initialization of G1FromCardCache to a proper location
  • 34ec1be: 8252104: parallel heap inspection for ShenandoahHeap
  • fdce055: 8253253: Binutils tar ball extension update to gz
  • 388c8f2: 8253348: Remove unimplemented JNIHandles::initialize
  • bca9e55: 8253167: ARM32 builds fail after JDK-8247910
  • cc7521c: 8252199: Reimplement support of Type 1 fonts without MappedByteBuffer
  • 3d88d38: 8252070: Some platform-specific BLIT optimizations are not effective
  • ... and 16 more: https://git.openjdk.java.net/jdk/compare/73c9088b81775c579789d96069c2f59364841e0b...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid automatic rebasing, please merge master into your branch, and then specify the current head hash when integrating, like this: /integrate dad6edbf83ca38cf9534e519629eee9847a54645.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

@openjdk openjdk bot added the ready label Sep 18, 2020
@stefank
Copy link
Member Author

@stefank stefank commented Sep 18, 2020

Thanks @shipilev. I went ahead and updated the code to not leak out the bad address, fixed a parameter name, and added a windows assert.

Copy link
Contributor

@shipilev shipilev left a comment

Still look fine.

@pliden
Copy link
Contributor

@pliden pliden commented Sep 18, 2020

I choose to prefix the OS dependent functions with os_. For consistency, initialize_os should have been renamed as well, but the plan is to change that in a separate patch that splits that function into two, so I skipped it for now.

May I suggest that we instead just keep to the convention we have (here and in other classes) and use an _os suffix on these new functions?

// Make the address range free
_manager.free(start, size);
Copy link
Contributor

@pliden pliden Sep 18, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can just move these two lines to the end of reserve_contiguous_inner(), and remove this function.

Copy link
Member Author

@stefank stefank Sep 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point.

@stefank
Copy link
Member Author

@stefank stefank commented Sep 21, 2020

I choose to prefix the OS dependent functions with os_. For consistency, initialize_os should have been renamed as well, but the plan is to change that in a separate patch that splits that function into two, so I skipped it for now.

May I suggest that we instead just keep to the convention we have (here and in other classes) and use an _os suffix on these new functions?

There's no strong convention that we postfix with os. During the Windows port I introduced two usages, both named initialize_os. Now when more OS specific code is added, and I need to split initialize_os into two function, the postfix makes it less clear what the OS-specific API extension points are.

Some of my motivations for going with a prefix instead:

  1. HotSpot already does that in other places. See the pd_ functions. For example, pd_commit_memory vs commit_memory.

  2. Splitting initialize_os wouldn't create an infix:
    os_initialize_before_reserve
    os_initialize_after_reserve
    vs
    initialize_os_before_reserve
    initialize_os_after_reserve

  3. Easily recognized group of os specific funtions
    os_initialize_before_reserve
    os_initialize_after_reserve
    os_reserve
    os_unreserve

  4. reserve_os sounded worse than os_reserve (to me)

I was considering splitting it out into a sub-interface, just to not have to deal with the *fix discussion/problem, but decided against it since the implementations needed member variables from ZVirtualMemoryManager.

@pliden
Copy link
Contributor

@pliden pliden commented Sep 21, 2020

@stefank That convention actually goes back to when ZGC was first integrated, with *_platform. However some of those *_platform functions has been removed over time as they were no longer needed. When the Windows port came in we said we should move away from *_platform to *_os, *_cpu and *_os_cpu to be more explicit and cater for potential future needs. To me, the suffix-style reads a bit better, the relationship between e.g. initialize() and initialize_os() becomes a bit more obvious, and it maps well to the platform specific file name convention we have in HotSpot (e.g. *_linux_x86.hpp). Having said that, I'm ok with changing the convention to prefix-style.

pliden
pliden approved these changes Sep 21, 2020
@stefank
Copy link
Member Author

@stefank stefank commented Sep 21, 2020

@stefank That convention actually goes back to when ZGC was first integrated, with *_platform. However some of those *_platform functions has been removed over time as they were no longer needed. When the Windows port came in we said we should move away from *_platform to *_os, *_cpu and *_os_cpu to be more explicit and cater for potential future needs. To me, the suffix-style reads a bit better, the relationship between e.g. initialize() and initialize_os() becomes a bit more obvious, and it maps well to the platform specific file name convention we have in HotSpot (e.g. *_linux_x86.hpp). Having said that, I'm ok with changing the convention to prefix-style.

About the last point: HotSpot code uses suffix for the platform file names, but the pd part is mostly a prefix.

Would you mind if I changed the few x_platform functions we have to pd_x functions? That way we don't invent our own way to mark platform-dependent code.

@pliden
Copy link
Contributor

@pliden pliden commented Sep 21, 2020

@stefank That would work for me.

@stefank
Copy link
Member Author

@stefank stefank commented Sep 21, 2020

/integrate

@openjdk openjdk bot closed this Sep 21, 2020
@openjdk openjdk bot added integrated and removed ready rfr labels Sep 21, 2020
@openjdk
Copy link

@openjdk openjdk bot commented Sep 21, 2020

@stefank Since your change was applied there have been 26 commits pushed to the master branch:

  • dad6edb: 8253321: java.util.Locale.LanguageRange#equals is inconsistent after calling hashCode
  • edc14f9: 8253286: Use expand_exact() instead of expand_at() for fixed requests in G1
  • bba948f: 8253411: [BACKOUT] [REDO] G1 incorrectly limiting young gen size when using the reserve can result in repeated full gcs
  • 955c2e6: 8253303: G1: Move static initialization of G1FromCardCache to a proper location
  • 34ec1be: 8252104: parallel heap inspection for ShenandoahHeap
  • fdce055: 8253253: Binutils tar ball extension update to gz
  • 388c8f2: 8253348: Remove unimplemented JNIHandles::initialize
  • bca9e55: 8253167: ARM32 builds fail after JDK-8247910
  • cc7521c: 8252199: Reimplement support of Type 1 fonts without MappedByteBuffer
  • 3d88d38: 8252070: Some platform-specific BLIT optimizations are not effective
  • ... and 16 more: https://git.openjdk.java.net/jdk/compare/73c9088b81775c579789d96069c2f59364841e0b...master

Your commit was automatically rebased without conflicts.

Pushed as commit fbfb62d.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@stefank stefank deleted the 8253332_unify_heap_view_reservation branch Sep 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot-gc integrated
3 participants