Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

8074101: Add verification that all tasks are actually claimed during roots processing #2046

Closed
wants to merge 5 commits into from

Conversation

albertnetymk
Copy link
Member

@albertnetymk albertnetymk commented Jan 12, 2021

The first commit removes some obsolete enum items, while the second commit adds the verification logic. Commit 2 introduces some "empty" task claims for the verification logic, explicitly marked in the comments.

Test: hotspot_gc


Progress

  • Change must not contain extraneous whitespace
  • Commit message must refer to an issue
  • Change must be properly reviewed

Issue

  • JDK-8074101: Add verification that all tasks are actually claimed during roots processing

Reviewers

Download

$ git fetch https://git.openjdk.java.net/jdk pull/2046/head:pull/2046
$ git checkout pull/2046

@bridgekeeper
Copy link

bridgekeeper bot commented Jan 12, 2021

👋 Welcome back ayang! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

@openjdk openjdk bot added the rfr Pull request is ready for review label Jan 12, 2021
@openjdk
Copy link

openjdk bot commented Jan 12, 2021

@albertnetymk The following label will be automatically applied to this pull request:

  • hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

@openjdk openjdk bot added the hotspot hotspot-dev@openjdk.org label Jan 12, 2021
@albertnetymk
Copy link
Member Author

/label hotspot-gc

@openjdk openjdk bot added the hotspot-gc hotspot-gc-dev@openjdk.org label Jan 12, 2021
@openjdk
Copy link

openjdk bot commented Jan 12, 2021

@albertnetymk
The hotspot-gc label was successfully added.

@mlbridge
Copy link

mlbridge bot commented Jan 12, 2021

Webrevs

// only for verification purpose
// already processed in java roots.
_process_strong_tasks.try_claim_task(G1RP_PS_CodeCache_oops_do);
#endif

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than these fake claims, consider something like this:

template<typename... Ts>
void all_tasks_completed(uint nworkers, Ts... tags) {
  // Type-check more_skipped are all of the same type as first_skipped.
  T0 typed_skipped[] = { first_skipped, more_skipped... };
  uint skipped[] = { static_cast<uint>(tags)... };
  all_tasks_completed_impl(nworkers, skipped, ARRAY_SIZE(skipped));
}

void all_tasks_completed(uint nworkers) {
  all_tasks_completed_impl(nworkers, nullptr, 0);
}

Usage:

all_tasks_completed(n_workers(),
                    G1RP_PS_CodeCache_oops_do,
                    G1RP_PS_refProcessor_oops_do)

all_tasks_completed_impl can check that all tasks have been claimed except
the skipped ones, which have not been claimed.

There might be better ways to write the variadic all_tasks_completed. It's
been a while since I've done anything with variadic templates.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's a better version of the variadic all_tasks_completed

template<typename T0, typename... Ts,
         ENABLE_IF(Conjunction<std::is_same<T0, Ts>...>::value)>
void all_tasks_completed(uint n_threads, T0 first_skipped, Ts... more_skipped) {
  static_assert(std::is_convertible<T0, uint>::value, "not convertible");
  uint skipped[] = { static_cast<uint>(first_skipped), static_cast<uint>(more_skipped)... };
  all_tasks_completed_impl(n_threads, skipped, ARRAY_SIZE(skipped));
}

Conjunction is in metaprogramming/logical.hpp.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you; updated as suggested.

}
}
assert(is_skipped, "%d not claimed.", i);
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can also do a separate loop over skipped to verify _tasks[skipped[i]] == 0 (with appropriate bounds checking).

@@ -329,7 +333,18 @@ class SubTasksDone: public CHeapObj<mtInternal> {
// cleared.)
//
// n_threads - Number of threads executing the sub-tasks.
void all_tasks_completed(uint n_threads);
// followed by vararg skipped tasks

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The description comment should be augmented to describe the optional skipped values.

// CodeCache is already processed in java roots
// refProcessor is not needed since we are inside a safe point
_process_strong_tasks.all_tasks_completed(n_workers(),
G1RP_PS_CodeCache_oops_do, G1RP_PS_refProcessor_oops_do);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When parameters or arguments are on multiple lines, we usually align all the arguments, and usually one per line. There are some other similar cases elsewhere in this change that I didn't redundantly comment.

@@ -26,6 +26,7 @@
#define SHARE_GC_SHARED_WORKGROUP_HPP

#include "memory/allocation.hpp"
#include "metaprogramming/logical.hpp"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should also add metaprogramming/enableIf.hpp, to avoid implicit include dependency.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I include both enableIf.hpp and logical.hpp, or only enableIf.hpp since it includes logical.hpp already?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have any tooling support, but there has been a trend away from relying on implicit includes, because they lead to breakages far away from refactorings that change includes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it; including both then.

@albertnetymk
Copy link
Member Author

Addressed all suggestions in the revision.

// must execute this. (When the last thread does so, the task array is
// cleared.)
// The calling thread asserts that it has attempted to claim all the tasks
// that it will try to claim. Tasks that is meant to be skipped must be

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/is meant/are meant/

// cleared.)
// The calling thread asserts that it has attempted to claim all the tasks
// that it will try to claim. Tasks that is meant to be skipped must be
// explicitly passed as extra arguments using the variadic version below.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would drop "using the variadic version below". I think of this as a function that takes some optional "these are expected to be skipped" task designators, and the non-variadic overload is just an implementation detail to handle the base case of there being none of those.

@openjdk
Copy link

openjdk bot commented Jan 13, 2021

@albertnetymk This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8074101: Add verification that all tasks are actually claimed during roots processing

Reviewed-by: kbarrett, tschatzl

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 101 new commits pushed to the master branch:

  • 917f7e9: 8259650: javax/swing/JComponent/7154030/bug7154030.java still fails with "Exception: Failed to hide opaque button"
  • 3f19ef6: 8202880: Test javax/swing/JPopupMenu/8075063/ContextMenuScrollTest.java fails
  • 68cf65d: 8023980: JCE doesn't provide any class to handle RSA private key in PKCS#1
  • 5dc5d94: 8256110: Create implementation for NSAccessibilityStepper protocol
  • 5f2e280: 8259865: (fs) test/jdk/java/nio/file/attribute/UserDefinedFileAttributeView/Basic.java failing on macOS 10.13
  • da4cf05: 8258755: jpackage: Invalid 32-bit exe when building app-image
  • c3bdbf9: 8259238: Clean up Log.java and remove usage of non-final static variables.
  • 6d6a23e: 8259062: Remove MacAppStoreBundler
  • afd3f78: 8030048: (fs) Support UserDefinedFileAttributeView/extended attributes on OS X / HFS+
  • bbb93ca: 8256126: Create implementation for NSAccessibilityImage protocol peer
  • ... and 91 more: https://git.openjdk.java.net/jdk/compare/a6ab9e4740024302fa74c3ede286c3fb6776b438...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change. Possible candidates are the reviewers of this PR (@kimbarrett, @tschatzl) but any other Committer may sponsor as well.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

@openjdk openjdk bot added the ready Pull request is ready to be integrated label Jan 13, 2021
@albertnetymk
Copy link
Member Author

Addressed suggestions on the comments.

Copy link
Contributor

@tschatzl tschatzl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgmt.

Atomic::inc(&_claimed);
// all non-skipped tasks are claimed
for (uint i = 0; i < _n_tasks; ++i) {
if (_tasks[i] == 0) {
Copy link
Contributor

@tschatzl tschatzl Jan 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pre-existing: This could be fixed in a separate CR: _tasks could be an array of bool instead of (u)int. Using an int is a historic artifact of not having a good Atomics library.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created a ticket (JDK-8259851) for it; will start working on that after this is merged. Thanks for the suggestion.

@albertnetymk
Copy link
Member Author

Thanks for the review.

/integrate

@albertnetymk
Copy link
Member Author

/integrate

@openjdk openjdk bot added the sponsor Pull request is ready to be sponsored label Jan 18, 2021
@openjdk
Copy link

openjdk bot commented Jan 18, 2021

@albertnetymk
Your change (at version 15892bd) is now ready to be sponsored by a Committer.

@openjdk
Copy link

openjdk bot commented Jan 18, 2021

@albertnetymk
Your change (at version 15892bd) is now ready to be sponsored by a Committer.

@tschatzl
Copy link
Contributor

/sponsor

@openjdk openjdk bot closed this Jan 18, 2021
@openjdk openjdk bot added the integrated Pull request has been integrated label Jan 18, 2021
@openjdk openjdk bot removed sponsor Pull request is ready to be sponsored ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jan 18, 2021
@openjdk
Copy link

openjdk bot commented Jan 18, 2021

@tschatzl @albertnetymk Since your change was applied there have been 101 commits pushed to the master branch:

  • 917f7e9: 8259650: javax/swing/JComponent/7154030/bug7154030.java still fails with "Exception: Failed to hide opaque button"
  • 3f19ef6: 8202880: Test javax/swing/JPopupMenu/8075063/ContextMenuScrollTest.java fails
  • 68cf65d: 8023980: JCE doesn't provide any class to handle RSA private key in PKCS#1
  • 5dc5d94: 8256110: Create implementation for NSAccessibilityStepper protocol
  • 5f2e280: 8259865: (fs) test/jdk/java/nio/file/attribute/UserDefinedFileAttributeView/Basic.java failing on macOS 10.13
  • da4cf05: 8258755: jpackage: Invalid 32-bit exe when building app-image
  • c3bdbf9: 8259238: Clean up Log.java and remove usage of non-final static variables.
  • 6d6a23e: 8259062: Remove MacAppStoreBundler
  • afd3f78: 8030048: (fs) Support UserDefinedFileAttributeView/extended attributes on OS X / HFS+
  • bbb93ca: 8256126: Create implementation for NSAccessibilityImage protocol peer
  • ... and 91 more: https://git.openjdk.java.net/jdk/compare/a6ab9e4740024302fa74c3ede286c3fb6776b438...master

Your commit was automatically rebased without conflicts.

Pushed as commit e93f08e.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

@albertnetymk albertnetymk deleted the claim branch January 18, 2021 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hotspot hotspot-dev@openjdk.org hotspot-gc hotspot-gc-dev@openjdk.org integrated Pull request has been integrated
3 participants