New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
8237363: Remove automatic is in heap verification in OopIterateClosure #797
8237363: Remove automatic is in heap verification in OopIterateClosure #797
Conversation
👋 Welcome back stefank! A progress list of the required criteria for merging this PR into |
This is mostly of concerns for the hotspot-gc, but touches compressed oops so I'll move this to hotspot instead. /label add hotspot |
@stefank |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This has two minor drawbacks for GC implementations that verify oops with their own asserts (like Shenandoah): they would call into CollectedHeap::is_in
twice (once from shared code assert, and once from their own), and then also fail with non-rich assert (in the shared code) when something goes wrong. Of course, that can be mitigated by calling into _raw
versions.
@@ -1737,7 +1737,7 @@ address FileMapInfo::decode_start_address(FileMapRegion* spc, bool with_current_ | |||
size_t offset = spc->mapping_offset(); | |||
narrowOop n = CompressedOops::narrow_oop_cast(offset); | |||
if (with_current_oop_encoding_mode) { | |||
return cast_from_oop<address>(CompressedOops::decode_not_null(n)); | |||
return cast_from_oop<address>(CompressedOops::decode_raw_not_null(n)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why does this line skip verification now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This code is used to setup the CDS archive. At that point in time, the heap isn't mapped yet, and there's no heap region at the suggested offset, so the new assert fails.
HeapRegion::is_in_reserved (this=0x0, p=0x7bfe00000)
HeapRegion::is_in (this=0x0, p=0x7bfe00000)
G1CollectedHeap::is_in (this=0x7ffff003a100, p=0x7bfe00000)
CompressedOops::decode_not_null (v=(unknown: -134479872))
FileMapInfo::decode_start_address (this=0x7ffff05d3e60, spc=0x7ffff05d4000, with_current_oop_encoding_mode=true)
FileMapInfo::start_address_as_decoded_with_current_oop_encoding_mode
FileMapInfo::get_heap_regions_range_with_current_oop_encoding_mode
FileMapInfo::map_heap_regions_impl
FileMapInfo::map_heap_regions
MetaspaceShared::map_archives
MetaspaceShared::initialize_runtime_shared_and_meta_spaces ()
Metaspace::global_initialize
universe_init ()
init_globals ()
Threads::create_vm
Note that the heap region is null (this=0x0). This also mean that the returned oop is not actually a valid oop at all, and this can sort of be seen by the immediate cast to address.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ew. That seems to imply that coops decoding is now tied to heap initialization for these asserts to work. I am pleasantly surprised it only fails in one place! I guess it is fine.
Yes, those are the trade-offs. Do you consider this a blocker? I personally wouldn't mind completely removing this verification from the shared code, and let all the GCs do their own oop verification. But others have had the opposite opinion. |
Not really a blocker, just mentioning the follow-up work that might need to be done. On one hand, we almost never omit asserts on performance grounds, but on the other hand, this adds assert on a rather frequent path. I don't mind having an extra safety there, and then let callers (e.g. GCs) to use |
OK. Thanks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good. Great work!
@stefank This change now passes all automated pre-integration checks. ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details. After integration, the commit message for the final commit will be:
You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed. At the time when this comment was updated there had been 46 new commits pushed to the
As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details. ➡️ To integrate this PR with the above commit message to the |
/integrate |
@stefank Since your change was applied there have been 48 commits pushed to the
Your commit was automatically rebased without conflicts. Pushed as commit 6666dcb. 💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored. |
There's verification code in the "oop iterate" framework that asserts that a pointer is "is in the heap". This works for most GCs, but ZGC can eagerly decommit the old relocation set pages, which means that pointers to the old / from copy of the object could point to memory that is currently not a part of the current heap.
To combat this in the past I've added a way for oop iterate closures to turn off this verification. However, every single time we add a new closure we have to consider if we can allow this verification check or if we have to remove it. Personally, I think this is a false abstraction and also widens the oop iterate closure interface. I previously proposed a patch that moved the verification code down into the oop iterate closures. It wasn't a huge patch, but I got push-back that it was convenient for other GCs to get this automatic verification, and the review stalled.
In this new patch I propose a different way to retain the verification. The realization is that most oop iterate closures have to deal with both compressed and non-compressed oops, so the code typically looks like this:
Therefore the suggest new place to put the is_in verification is in the CompressedOops::decode*. This injects the assert into almost all non-ZGC closures, and also to places that don't use the oop iterate closure framework. I think this is a neat workaround, and hope this patch is accepted this time.
I've tested this patch a few weeks ago, but will rerun the relevant tiers.
Progress
Testing
Failed test task
Issue
Reviewers
Download
$ git fetch https://git.openjdk.java.net/jdk pull/797/head:pull/797
$ git checkout pull/797