8276789: Support C++ lambda in ResourceHashtable::iterate #8761

iklam · 2022-05-17T23:37:52Z

I added two new template functions to reduce the boilerplate code when walking the entries in a ResourceHashtable

template<typename F> void ResourceHashtable::iterate(F f)
template<typename F> void ResourceHashtable::iterate_all(F f)

I also modified a couple of places in systemDictionaryShared.cpp to use the new functionality.

Testing with tiers 1-4.

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8276789: Support C++ lambda in ResourceHashtable::iterate

Reviewers

Stefan Karlsson (@stefank - Reviewer) ⚠️ Review applies to 8c53c418
Coleen Phillimore (@coleenp - Reviewer) ⚠️ Review applies to 083fb72d

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.java.net/jdk pull/8761/head:pull/8761
$ git checkout pull/8761

Update a local copy of the PR:
$ git checkout pull/8761
$ git pull https://git.openjdk.java.net/jdk pull/8761/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 8761

View PR using the GUI difftool:
$ git pr show -t 8761

Using diff file

Download this PR as a diff file:
https://git.openjdk.java.net/jdk/pull/8761.diff

bridgekeeper · 2022-05-17T23:38:57Z

👋 Welcome back iklam! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

iklam · 2022-05-17T23:41:02Z

GCC 10.3.0 generates identical code for the following before and after this PR. So there should be no impact with existing code that uses the old style iterator.

class MyIter : StackObj {
public:
  int _count;
  MyIter() : _count(0) {}
  bool do_entry(InstanceKlass* k, DumpTimeClassInfo& info) {
    _count ++;
    return true; // keep on iterating
  }
};

int doit(DumpTimeSharedClassTable* dumptime_table) {
  MyIter myiter;
  dumptime_table->iterate(&myiter);
  return myiter._count;
}

openjdk · 2022-05-17T23:41:14Z

@iklam The following label will be automatically applied to this pull request:

hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2022-05-17T23:47:15Z

Webrevs

dholmes-ora · 2022-05-18T02:06:21Z

src/hotspot/share/cds/dumpTimeClassInfo.inline.hpp

-  bool do_entry(InstanceKlass* k, DumpTimeClassInfo& info) {
+template<typename F>
+void DumpTimeSharedClassTable::iterate(F f) const {
+  auto g = [=] (InstanceKlass* k, DumpTimeClassInfo& info) {


The style guide states we should try to only use [&] for capture by reference.

dougxc · 2022-05-18T07:01:22Z

src/hotspot/share/cds/dumpTimeClassInfo.inline.hpp

@@ -36,21 +36,15 @@
 #if INCLUDE_CDS

 // For safety, only iterate over a class if it loader is alive.


"if it loader" -> "if its loader is"

@stefank

…ggested by @@stefank

stefank · 2022-05-24T12:29:32Z

src/hotspot/share/classfile/systemDictionaryShared.cpp

+  auto check_for_exclusion = [&] (InstanceKlass* k, DumpTimeClassInfo& info) {
+    SystemDictionaryShared::check_for_exclusion(k, &info);
+  };
+  _dumptime_table->iterate_all(check_for_exclusion);


Can't this be changed to just:
_dumptime_table->iterate_all(SystemDictionaryShared::check_for_exclusion);

or maybe even:
_dumptime_table->iterate_all(check_for_exclusion);

The second parameter to SystemDictionaryShared::check_for_exclusion() is a DumpTimeClassInfo* because NULL is allowed. However, the iterator requires a DumpTimeClassInfo&.

stefank · 2022-05-24T12:33:59Z

src/hotspot/share/utilities/resourceHash.hpp

  template<class ITER>
  void iterate(ITER* iter) const {
+    auto function = [&] (K& k, V& v) {
+      return iter->do_entry(k, v);
+    };
+    iterate(function);
+  }


Do you still have any usages of this? I'm a bit surprised that the compiler don't complain about not being able to disambiguate calls to this vs the new lambda enabled function.

This is still needed as there are many places that use the old style iteration. Also, when the body of the iteration is complex, it might be more readable to do it as a separate class. E.g., here in systemDictionaryShared.cpp:

UnregisteredClassesDuplicationChecker dup_checker; _dumptime_table->iterate(&dup_checker);

I think there's no ambiguity because iter->do_entry() cannot be substituted with the other template, and function(node->_key, node->_value) in the other template cannot be substituted here. So by SFINAE the C++ compiler can pick the correct template.

Interesting. I got problems when I converted BitMap to support both closures and lambdas. The problem might have been that the iterate function took an specific closure class, and didn't use a template.

[Not a review, just a drive-by comment]
SFINAE operates on arguments and signatures; it doesn't reach into the bodies
of (potential) callees like that. The disambiguation is happening because one
argument (the Closure object) is passed as a pointer, while the other is a
lambda object (not a pointer). In the pointer to closure object case the ITER*
overload is a better match, so is selected. In the lambda case, it's not a
pointer so only the non-pointer overload is applicable.

(And SFINAE can't reach into the body like that, since the body might not be available.)

@kimbarrett Thanks for the explanation. So we're lucky because we use ITER* in the old API. If it were ITER& then the C++ compiler would complain about ambiguous overloading.

stefank · 2022-05-25T06:28:34Z

src/hotspot/share/classfile/systemDictionaryShared.cpp

+  auto check_for_exclusion = [&] (InstanceKlass* k, DumpTimeClassInfo& info) {
+    SystemDictionaryShared::check_for_exclusion(k, &info);
+  };
+  _dumptime_table->iterate_all(check_for_exclusion);


stefank · 2022-05-25T06:28:34Z

src/hotspot/share/utilities/resourceHash.hpp

  template<class ITER>
  void iterate(ITER* iter) const {
+    auto function = [&] (K& k, V& v) {
+      return iter->do_entry(k, v);
+    };
+    iterate(function);
+  }


Interesting. I got problems when I converted BitMap to support both closures and lambdas. The problem might have been that the iterate function took an specific closure class, and didn't use a template.

openjdk · 2022-05-25T06:29:30Z

@iklam This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8276789: Support C++ lambda in ResourceHashtable::iterate

Reviewed-by: stefank, coleenp

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

coleenp

I'm a bit baffled by the layers of lambda expressions.

coleenp · 2022-06-01T20:23:01Z

src/hotspot/share/cds/dumpTimeClassInfo.hpp

@@ -209,8 +209,15 @@ class DumpTimeSharedClassTable: public DumpTimeSharedClassTableBaseType
    }


This file has a blank first line, can you fix this?

And no new line between these here:
#define SHARED_CDS_DUMPTIMESHAREDCLASSINFO_HPP
#include "cds/archiveBuilder.hpp"

coleenp · 2022-06-01T20:23:01Z

src/hotspot/share/cds/dumpTimeClassInfo.inline.hpp

+  auto function = [&] (InstanceKlass* k, DumpTimeClassInfo& v) {
+    return iter->do_entry(k, v);
+  };
+  iterate(function);


I don't understand this wrapper level. Why doesn't this call into the resourceHashtable::iterate function directly? Same with the iterate_all function?

I renamed the functions to DumpTimeSharedClassTable::iterate_all_live_classes() to disambiguate them from the ResourceHashtable::iterate/_all functions.

coleenp · 2022-06-01T20:23:01Z

src/hotspot/share/classfile/systemDictionaryShared.cpp

 void SystemDictionaryShared::check_excluded_classes() {
  assert(no_class_loading_should_happen(), "sanity");
  assert_lock_strong(DumpTimeTable_lock);

  if (DynamicDumpSharedSpaces) {
    // Do this first -- if a base class is excluded due to duplication,
-    // all of its subclasses will also be excluded by ExcludeDumpTimeSharedClasses
+    // all of its subclasses will also be excluded.
    ResourceMark rm;
    UnregisteredClassesDuplicationChecker dup_checker;
    _dumptime_table->iterate(&dup_checker);


Which iterate function does this call?

I changed it to _dumptime_table->iterate_all_live_classes()

… confusion with overloading

coleenp

Looks good. Thanks for the help and discussion offline. Now it makes sense.

coleenp · 2022-06-03T00:47:01Z

src/hotspot/share/cds/dumpTimeClassInfo.hpp

+  // instead.
+  template<class ITER> void iterate(ITER* iter) const;
+  template<typename Function> void iterate(Function function) const;
+  template<typename Function> void iterate_all(Function function) const;


This helps a lot and thanks for the comment, which describes why there's a wrapper for this class.

…cpp-lambda-for-resourceHash-iterate

iklam · 2022-06-03T05:58:44Z

Thanks @stefank and @coleenp for the review.
/integrate

openjdk · 2022-06-03T05:59:39Z

Going to push as commit b544b8b.

openjdk · 2022-06-03T05:59:57Z

@iklam Pushed as commit b544b8b.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

8276789: Support C++ lambda in ResourceHashtable::iterate

e482828

openjdk bot added the hotspot label May 17, 2022

iklam marked this pull request as ready for review May 17, 2022

openjdk bot added the rfr label May 17, 2022

dholmes-ora reviewed May 18, 2022

View changes

changed to use lambda capture by reference [&]

97cf613

dougxc reviewed May 18, 2022

View changes

ioilam added 3 commits May 18, 2022

Better naming of variables. E.g., (F f) -> (Function function), as su…

6533a75

…ggested by @@stefank

Fixed typo in comment (it loader -> its loader)

f3ca12a

some clean up suggested by @stefank

8c53c41

stefank reviewed May 24, 2022

View changes

stefank approved these changes May 25, 2022

View changes

openjdk bot added the ready label May 25, 2022

coleenp reviewed Jun 1, 2022

View changes

ioilam added 2 commits Jun 2, 2022

Renamed DumpTimeSharedClassTable::iterate_all_live_classes() to avoid…

e47fe20

… confusion with overloading

removed empty first line

083fb72

coleenp approved these changes Jun 3, 2022

View changes

Merge branch 'master' of https://github.com/openjdk/jdk into 8276789-…

811beca

…cpp-lambda-for-resourceHash-iterate

openjdk bot added the integrated label Jun 3, 2022

openjdk bot closed this Jun 3, 2022

openjdk bot removed ready rfr labels Jun 3, 2022

8276789: Support C++ lambda in ResourceHashtable::iterate #8761

8276789: Support C++ lambda in ResourceHashtable::iterate #8761

iklam commented May 17, 2022 •

edited by openjdk[bot] bot

bridgekeeper bot commented May 17, 2022

iklam commented May 17, 2022

openjdk bot commented May 17, 2022

mlbridge bot commented May 17, 2022 •

edited

dholmes-ora May 18, 2022

iklam May 18, 2022

dougxc May 18, 2022

stefank May 24, 2022 •

edited

iklam May 24, 2022

stefank May 25, 2022

stefank May 24, 2022

iklam May 24, 2022

stefank May 25, 2022

kimbarrett Jun 2, 2022

kimbarrett Jun 2, 2022

iklam Jun 3, 2022

stefank May 25, 2022

stefank May 25, 2022

openjdk bot commented May 25, 2022 •

edited

coleenp left a comment

coleenp Jun 1, 2022

coleenp Jun 1, 2022

iklam Jun 2, 2022

coleenp Jun 1, 2022

iklam Jun 2, 2022

coleenp Jun 1, 2022

iklam Jun 2, 2022

coleenp left a comment

coleenp Jun 3, 2022

iklam commented Jun 3, 2022

openjdk bot commented Jun 3, 2022

openjdk bot commented Jun 3, 2022

		@@ -36,21 +36,15 @@
		#if INCLUDE_CDS

		// For safety, only iterate over a class if it loader is alive.

		@@ -209,8 +209,15 @@ class DumpTimeSharedClassTable: public DumpTimeSharedClassTableBaseType
		}

8276789: Support C++ lambda in ResourceHashtable::iterate #8761

8276789: Support C++ lambda in ResourceHashtable::iterate #8761

Conversation

iklam commented May 17, 2022 • edited by openjdk[bot] bot

Progress

Issue

Reviewers

Reviewing

bridgekeeper bot commented May 17, 2022

iklam commented May 17, 2022

openjdk bot commented May 17, 2022

mlbridge bot commented May 17, 2022 • edited

Webrevs

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stefank May 24, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

openjdk bot commented May 25, 2022 • edited

coleenp left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

coleenp left a comment

Choose a reason for hiding this comment

iklam commented Jun 3, 2022

openjdk bot commented Jun 3, 2022

openjdk bot commented Jun 3, 2022

iklam commented May 17, 2022 •

edited by openjdk[bot] bot

mlbridge bot commented May 17, 2022 •

edited

stefank May 24, 2022 •

edited

openjdk bot commented May 25, 2022 •

edited