Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Plumb WriteBufferManager through JNI #4492

Closed
wants to merge 2 commits into from
Closed

Plumb WriteBufferManager through JNI #4492

wants to merge 2 commits into from

Conversation

jbhati
Copy link
Contributor

@jbhati jbhati commented Oct 15, 2018

Allow rocks java to explicitly create WriteBufferManager by plumbing it to the native code through JNI.

@facebook-github-bot
Copy link
Contributor

Thank you for your pull request and welcome to our community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. In order for us to review and merge your code, please sign up at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need the corporate CLA signed.

If you have received this in error or have any questions, please contact us at cla@fb.com. Thanks!

@jbhati
Copy link
Contributor Author

jbhati commented Oct 15, 2018

@sagar0 can you please review this pull request? Thanks in advance! :)

@sagar0
Copy link
Contributor

sagar0 commented Oct 15, 2018

Thanks! Can you please sign the CLA?

@jbhati
Copy link
Contributor Author

jbhati commented Oct 15, 2018

@sagar0 Signed the CLA. Thanks

@facebook-github-bot
Copy link
Contributor

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Facebook open source project. Thanks!

@jbhati
Copy link
Contributor Author

jbhati commented Oct 16, 2018

@sagar0 ping :)

@sagar0 sagar0 self-assigned this Oct 16, 2018
sagar0
sagar0 reviewed Oct 17, 2018
View reviewed changes
Copy link
Contributor

@sagar0 sagar0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me with a few minor suggestions.
Once they are handled I believe this will be in a good shape to be accepeted.

java/rocksjni/write_buffer_manager.cc Outdated
@@ -0,0 +1,38 @@
// Copyright (c) 2014, Vlad Balan (vlad.gm@gmail.com). All rights reserved.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Copyright (c) 2014, Vlad Balan (vlad.gm@gmail.com). All rights reserved.
// Copyright (c) 2011-present, Facebook, Inc. All rights reserved.

java/src/test/java/org/rocksdb/DBOptionsTest.java Outdated
try (final Options opt = new Options()) {
final Cache cache = new LRUCache(1 * 1024 * 1024);
final long bufferSize = 2000l;
final WriteBufferManager writeBufferManager = new WriteBufferManager(bufferSize, cache);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets move the instantiation of Cache and writeBufferManager (lines 430 - 432) also into the try block.
Same comment for the code change in OptionsTest as well.

java/src/test/java/org/rocksdb/DBOptionsTest.java Outdated
try (final Options opt = new Options()) {
final Cache cache = new LRUCache(1 * 1024 * 1024);
final long bufferSize = 0l;
final WriteBufferManager writeBufferManager = new WriteBufferManager(bufferSize, cache);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets move the instantiation of Cache and writeBufferManager also into the try block.

java/src/main/java/org/rocksdb/WriteBufferManager.java
/**
* Java wrapper over native write_buffer_manager class
*/
public class WriteBufferManager extends RocksObject {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

May be for later: others can contribute the remaining methods in native WriteBufferManager (like memroy_usage, FreeMem, etc).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, maybe we will end up implementing it :)

@sagar0
Copy link
Contributor

sagar0 commented Oct 17, 2018

I handled the travis and appveyor failures for you; those are green now.

@siying
Copy link
Contributor

siying commented Oct 17, 2018

We don't do it a lot but users should be able to add themselves to https://github.com/facebook/rocksdb/blob/master/AUTHORS if they want.

@siying
Copy link
Contributor

siying commented Oct 17, 2018

Feel free to update AUTHORS, but we need to ask lawyers about non-standard headers.

@sagar0
Copy link
Contributor

sagar0 commented Oct 17, 2018

In this particular case the copyright (and the surrounding author name) was a copy-paste error from merge_operator.cc/MergeOperator.java.

@jbhati
Copy link
Contributor Author

jbhati commented Oct 17, 2018

Thanks for the quick review and taking care of the builds. Appreciate it @sagar0. Made the requested changes. It is reviewable again.

facebook-github-bot
facebook-github-bot reviewed Oct 17, 2018
View reviewed changes
Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sagar0 has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@jbhati
Copy link
Contributor Author

jbhati commented Oct 22, 2018 •
edited
Loading

@sagar0 quick question. I applied this patch to our Manhattan database and I am getting the below runtime error. Was wondering if you know what might be wrong here?
java.lang.UnsatisfiedLinkError: org.rocksdb.DBOptions.setWriteBufferManager(JJ)V

I have Block cache enabled and WriteBufferManager() creation went through. However, when using DBOptions. java -> setWriteBufferManager(writeBufferManager), it throws the above exception. Running the test (DBOptionsTest.java and OptionsTest.java) from facebook fork of rocksdb works fine. Can you think of anything on top of your head which might have caused this?

@jbhati
Copy link
Contributor Author

jbhati commented Oct 22, 2018

Never mind. Figured it out. Thanks. Will send a PR soon for the fix.

@jbhati jbhati mentioned this pull request Oct 22, 2018
Closed
jbhati added a commit to jbhati/rocksdb that referenced this pull request Oct 23, 2018
@jbhati
RTSG-3808: Fix DBOptions WriteBufferManager jni settings and set Writ…
7fe6714 
…eBufferManager life as lifetime of Options

Summary: Had bugs in PR: facebook#4492 which is fixed in this

Test Plan: test cases

Reviewers: bclay, sowmyalakshmip

JIRA Issues: RTSG-3808

Differential Revision: https://phabricator.twitter.biz/D231875
mikekap pushed a commit to ladderlife/frocksdb that referenced this pull request Jun 27, 2019
@jbhati @mikekap
Plumb WriteBufferManager through JNI (#4492)
2a78717 
Summary:
Allow rocks java to explicitly create WriteBufferManager by plumbing it to the native code through JNI.
Pull Request resolved: facebook/rocksdb#4492

Differential Revision: D10428506

Pulled By: sagar0

fbshipit-source-id: cd9dd8c2ef745a0303416b44e2080547bdcca1fd
mikekap pushed a commit to ladderlife/frocksdb that referenced this pull request Jun 27, 2019
@jbhati @mikekap
Plumb WriteBufferManager through JNI (#4492)
85d4362 
Summary:
Allow rocks java to explicitly create WriteBufferManager by plumbing it to the native code through JNI.
Pull Request resolved: facebook/rocksdb#4492

Differential Revision: D10428506

Pulled By: sagar0

fbshipit-source-id: cd9dd8c2ef745a0303416b44e2080547bdcca1fd
Myasuka pushed a commit to Myasuka/frocksdb that referenced this pull request Oct 31, 2019
@jbhati @Myasuka
Plumb WriteBufferManager through JNI (#4492)
0fc1a17 
Summary:
Allow rocks java to explicitly create WriteBufferManager by plumbing it to the native code through JNI.
Pull Request resolved: facebook/rocksdb#4492

Differential Revision: D10428506

Pulled By: sagar0

fbshipit-source-id: cd9dd8c2ef745a0303416b44e2080547bdcca1fd
StephanEwen pushed a commit to ververica/frocksdb that referenced this pull request Nov 21, 2019
@jbhati @StephanEwen
Plumb WriteBufferManager through JNI (#4492)
fa60a7c 
Summary:
Allow rocks java to explicitly create WriteBufferManager by plumbing it to the native code through JNI.
Pull Request resolved: facebook/rocksdb#4492

Differential Revision: D10428506

Pulled By: sagar0

fbshipit-source-id: cd9dd8c2ef745a0303416b44e2080547bdcca1fd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@facebook-github-bot facebook-github-bot facebook-github-bot left review comments

@sagar0 sagar0 sagar0 approved these changes

Assignees

@sagar0 sagar0

Labels
CLA Signed java-api
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jbhati @facebook-github-bot @sagar0 @siying