Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support merge operators in java #4805

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

gfouquier
Copy link

@gfouquier gfouquier commented Dec 20, 2018

Support for merge operator in java.

Add AbstractAssociativeMergeOperator and AbstractMergeOperator which allow to implement merge operators in java. The MergeOperator class in java now inherit from RocksCallBacksObject and existing merge operators have been rewritten accordingly. In c++, AssociativeMergeOperatorJniCallback and MergeOperatorJniCallback inherit from JniCallback.

This PR is based on #3432 but without thread JNI management neither loader management for static variables. Performances are slow since jni thread must be attached/detached at each call.

resolve #2282

@vjnadimpalli
Copy link
Contributor

@gfouquier thanks for contributing this change, its amazing. Would you be able to rebase on top of the latest? Also it would be nice to write some comments explaining the interaction between the various java and c++ code files.

abstract public String name();

@Override
protected long initializeNative(final long... nativeParameterHandles) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is this method called?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I must read that part again, but it's probably from the portal class AssociativeMergeOperatorJni which call this : RocksDBNativeClass::getJClass(env, "org/rocksdb/AbstractAssociativeMergeOperator");

@gfouquier
Copy link
Author

@gfouquier thanks for contributing this change, its amazing. Would you be able to rebase on top of the latest? Also it would be nice to write some comments explaining the interaction between the various java and c++ code files.

I am currently trying to rebase the PR, but I need some time since I didn't read this code since last year.

Geoffroy Fouquier added 3 commits September 17, 2019 12:50
rename NotAssociativeMergeOperator as MergeOperator
Cosmetic changes: sort includes, add description, remove commented parts, add missing licenses
@gfouquier gfouquier force-pushed the v2_mergeOperator_as_rocksCallBackObject branch from eaf3baf to 5747d69 Compare September 17, 2019 10:54
@gfouquier gfouquier force-pushed the v2_mergeOperator_as_rocksCallBackObject branch from 5747d69 to 731e3a4 Compare September 17, 2019 11:44
@gfouquier
Copy link
Author

The PR is now up to date.

Also it would be nice to write some comments explaining the interaction between the various java and c++ code files.

I mimic existing behaviour between java and c++ files, so I don't think this should appear as comment in this PR files specifically.
In a nutshell:

  • there is two java important classes : AbstractMergeOperator and AbstractAssociativeMergeOperator.
  • methods of these classes are implemented in merge_operator.cc and associative_merge_operator.cc
  • jnicallback classes and files allows to communicate from c++ to java instances

void JniCallback::catchAndLog(JNIEnv* env) const {
jboolean hasException = env->ExceptionCheck();
if (hasException == JNI_TRUE) {
env->ExceptionDescribe();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this just write to StdOut/StdErr?

It would be better to use the Logger here.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure how to do this. I saw that ExceptionDescribe was used in other places too. Maybe it should be part of a more global ticket for jni env exception handling

Copy link
Collaborator

@adamretter adamretter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general this looks good, thank you.
It just needs a small bit of cleanup around error handling.

Copy link
Author

@gfouquier gfouquier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated version with a lot more checking around jni calls and some minor fixes.

@amitsharma10
Copy link

When will this PR merge ?

@pmeister
Copy link

It looks like this PR has been untouched for close to a year. Is there any possibility of moving this forward soon?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support MergeOperator in Java
6 participants