Make UserComparatorWrapper not Customizable #10837

siying · 2022-10-19T21:14:57Z

Summary:
Right now UserComparatorWrapper is a Customizable object, although it is not, which introduces some intialization overhead for the object. In some benchmarks, it shows up in CPU profiling. Make it not configurable by defining most functions needed by UserComparatorWrapper to an interface and implement the interface.

Test Plan: Make sure existing tests pass

facebook-github-bot · 2022-10-19T21:15:16Z

@siying has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

facebook-github-bot · 2022-10-19T21:21:04Z

@siying has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-10-19T21:21:19Z

@siying has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

mrambacher · 2022-10-20T18:09:03Z

include/rocksdb/comparator.h

+// The general interface for comparing two Slices that are related to
+// timestamps.
+// They are defined for both Comparator and some internal data structures.
+class CompareWithTimestampInterface {


Would we ever have a Compare that was a CompareWithTimestamp that is not also a CompareInterface? If not, is there a reason why to not have CompareWithTimestamp extend CompareInterface?

CompareInterface is used for Internal comparators, and internal operators don't seem to do those timestamp comparisons.

Mixing the two adds to confusion and bugs. AFAIK the only reason they are mixed in Comparator is API compatibility. CompareInterface has internal applications where only one kind of compare is available.

mrambacher · 2022-10-20T18:09:21Z

include/rocksdb/comparator.h

+class CompareWithTimestampInterface {
+ public:
+  virtual ~CompareWithTimestampInterface() {}
+  int CompareWithoutTimestamp(const Slice& a, const Slice& b) const {


Should this function be virtual?

It is not virtual in the first place. I don't think we should create virtual functions when nobody use them.

It's apparently a convenience function (and should be documented accordingly)

mrambacher · 2022-10-20T18:09:41Z

include/rocksdb/comparator.h

+  virtual int CompareTimestamp(const Slice& /*ts1*/,
+                               const Slice& /*ts2*/) const = 0;
+
+  virtual int CompareWithoutTimestamp(const Slice& a, bool /*a_has_ts*/,


Please add a function comment

mrambacher · 2022-10-20T18:10:02Z

include/rocksdb/comparator.h

+                                      const Slice& b,
+                                      bool /*b_has_ts*/) const = 0;
+
+  virtual bool EqualWithoutTimestamp(const Slice& a, const Slice& b) const = 0;


Please add a function comment

mrambacher · 2022-10-20T18:11:05Z

include/rocksdb/comparator.h

  virtual int CompareTimestamp(const Slice& /*ts1*/,
-                               const Slice& /*ts2*/) const {
+                               const Slice& /*ts2*/) const override {


Just override, not virtual+override.

This same comment applies to many of the functions below.

mrambacher · 2022-10-20T18:17:24Z

include/rocksdb/comparator.h

+// The general interface for comparing two Slices that are related to
+// timestamps.
+// They are defined for both Comparator and some internal data structures.
+class CompareWithTimestampInterface {


timestamp_size should be moved into this class (and the Comparator constructor) should call the appropriate related constructor.

This interface may be used a wrapper. If we move timestamp_size there, there will be potential consistency issues.

I can see the argument for keeping this an "interface" with no data. Multiple inheritance with data fields should be minimized.

mrambacher · 2022-10-20T18:18:37Z

util/user_comparator_wrapper.h

  explicit UserComparatorWrapper(const Comparator* const user_cmp)
-      : Comparator(user_cmp->timestamp_size()), user_comparator_(user_cmp) {}
+      : user_comparator_(user_cmp) {}


Is there a reason that user_comparator_ should not be the interface rather than the Comparator? Comparably why this constructor does not take the interface?

Users sometimes call other functions of the inner comparator, and we will need to support it.

Seems fine to me. Wrapper shares a common interface for essential functions while also providing lower level access to those who know they're using the wrapper.

Though I don't see anywhere where the wrapper is strictly used as a CompareWithTimestampInterface. Does it really need virtual functions and a virtual destructor if we are counting on its uses being de-virtualized anyway?

I think it's a good point. Let me see whether I can just make them non-virtual and avoid this new interface.

pdillinger · 2022-10-20T19:02:26Z

I figured out why this seems so familiar. This is basically part 2 of #10342 .

siying · 2022-10-20T19:06:09Z

I figured out why this seems so familiar. This is basically part 2 of #10342 .

Yes, it is very similar.

pdillinger · 2022-10-20T19:08:29Z

db/db_iter.cc

@@ -522,7 +523,8 @@ bool DBIter::MergeValuesNewToOld() {
      return false;
    }

-    if (!user_comparator_.Equal(ikey.user_key, saved_key_.GetUserKey())) {
+    if (!user_comparator_.user_comparator()->Equal(ikey.user_key,


Why bypassing the perf counter here?

pdillinger · 2022-10-20T19:08:40Z

db/db_iter.cc

@@ -1158,7 +1160,8 @@ bool DBIter::FindValueForCurrentKeyUsingSeek() {
    if (!ParseKey(&ikey)) {
      return false;
    }
-    if (!user_comparator_.Equal(ikey.user_key, saved_key_.GetUserKey())) {
+    if (!user_comparator_.user_comparator()->Equal(ikey.user_key,


Why bypassing the perf counter here?

pdillinger · 2022-10-20T19:14:36Z

include/rocksdb/comparator.h

+// The general interface for comparing two Slices that are related to
+// timestamps.
+// They are defined for both Comparator and some internal data structures.
+class CompareWithTimestampInterface {


Mixing the two adds to confusion and bugs. AFAIK the only reason they are mixed in Comparator is API compatibility. CompareInterface has internal applications where only one kind of compare is available.

pdillinger · 2022-10-20T19:16:05Z

include/rocksdb/comparator.h

+// The general interface for comparing two Slices that are related to
+// timestamps.
+// They are defined for both Comparator and some internal data structures.
+class CompareWithTimestampInterface {


I can see the argument for keeping this an "interface" with no data. Multiple inheritance with data fields should be minimized.

pdillinger · 2022-10-20T19:17:49Z

include/rocksdb/comparator.h

+class CompareWithTimestampInterface {
+ public:
+  virtual ~CompareWithTimestampInterface() {}
+  int CompareWithoutTimestamp(const Slice& a, const Slice& b) const {


It's apparently a convenience function (and should be documented accordingly)

pdillinger · 2022-10-20T19:25:33Z

util/user_comparator_wrapper.h

  explicit UserComparatorWrapper(const Comparator* const user_cmp)
-      : Comparator(user_cmp->timestamp_size()), user_comparator_(user_cmp) {}
+      : user_comparator_(user_cmp) {}


Seems fine to me. Wrapper shares a common interface for essential functions while also providing lower level access to those who know they're using the wrapper.

Though I don't see anywhere where the wrapper is strictly used as a CompareWithTimestampInterface. Does it really need virtual functions and a virtual destructor if we are counting on its uses being de-virtualized anyway?

Summary: Right now UserComparatorWrapper is a Customizable object, although it is not, which introduces some intialization overhead for the object. In some benchmarks, it shows up in CPU profiling. Make it not configurable by defining most functions needed by UserComparatorWrapper to an interface and implement the interface. Test Plan: Make sure existing tests pass

siying

@pdillinger @mrambacher I remove the new interface and hopefully it looks better to you now.

facebook-github-bot · 2022-10-20T23:22:54Z

@siying has updated the pull request. You must reimport the pull request before landing.

facebook-github-bot · 2022-10-20T23:23:14Z

@siying has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

pdillinger

LGTM, thanks!

siying requested a review from pdillinger October 19, 2022 21:14

facebook-github-bot added the CLA Signed label Oct 19, 2022

siying force-pushed the cmp_wrap_no_customizable branch from 5952cee to ed1fecb Compare October 19, 2022 21:21

mrambacher reviewed Oct 20, 2022

View reviewed changes

pdillinger reviewed Oct 20, 2022

View reviewed changes

siying added 2 commits October 20, 2022 16:20

Make wrapper not implement anything

1166c98

siying force-pushed the cmp_wrap_no_customizable branch from ed1fecb to 1166c98 Compare October 20, 2022 23:20

siying commented Oct 20, 2022

View reviewed changes

pdillinger approved these changes Oct 21, 2022

View reviewed changes

facebook-github-bot closed this in 5d17297 Oct 21, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make UserComparatorWrapper not Customizable #10837

Make UserComparatorWrapper not Customizable #10837

siying commented Oct 19, 2022

facebook-github-bot commented Oct 19, 2022

facebook-github-bot commented Oct 19, 2022

facebook-github-bot commented Oct 19, 2022

mrambacher Oct 20, 2022

siying Oct 20, 2022

pdillinger Oct 20, 2022

mrambacher Oct 20, 2022

siying Oct 20, 2022

pdillinger Oct 20, 2022

mrambacher Oct 20, 2022

mrambacher Oct 20, 2022

mrambacher Oct 20, 2022

mrambacher Oct 20, 2022

siying Oct 20, 2022

pdillinger Oct 20, 2022

mrambacher Oct 20, 2022

siying Oct 20, 2022

pdillinger Oct 20, 2022

siying Oct 20, 2022

pdillinger commented Oct 20, 2022

siying commented Oct 20, 2022

pdillinger Oct 20, 2022

pdillinger Oct 20, 2022

pdillinger Oct 20, 2022

pdillinger Oct 20, 2022

pdillinger Oct 20, 2022

pdillinger Oct 20, 2022

siying left a comment

facebook-github-bot commented Oct 20, 2022

facebook-github-bot commented Oct 20, 2022

pdillinger left a comment

Make UserComparatorWrapper not Customizable #10837

Make UserComparatorWrapper not Customizable #10837

Conversation

siying commented Oct 19, 2022

facebook-github-bot commented Oct 19, 2022

facebook-github-bot commented Oct 19, 2022

facebook-github-bot commented Oct 19, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pdillinger commented Oct 20, 2022

siying commented Oct 20, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

siying left a comment

Choose a reason for hiding this comment

facebook-github-bot commented Oct 20, 2022

facebook-github-bot commented Oct 20, 2022

pdillinger left a comment

Choose a reason for hiding this comment