Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LevelDB native support #1839 #1902

Merged
merged 1 commit into from Oct 15, 2020

Conversation

JiriOndrusek
Copy link
Contributor

@JiriOndrusek JiriOndrusek commented Oct 12, 2020

fixes #1839

There are several "topics" where I'd appreciate opinion of others.

  1. Preferred implementation of LevelDb is in C++, implementation in Java is much slower (see https://github.com/dain/leveldb#leveldb-in-java). Current PR uses only Java implementation in native. I'd like to investigate possible usage of C++ library in native, but I see it more like an enhancement, because current PR works.

  2. Component levelDb uses serialization of DefaultExchangeHolder. I've used workaround with Jackson serialization instead. (see https://github.com/apache/camel-quarkus/compare/master...JiriOndrusek:1839-leveldb-native?expand=1#diff-5fbe79b4f16469a82d30051dc3cf564aR46)

  3. Java implementation of LevelDB uses methodHandle which is not supported by GraalVM. I used previous version of ByteBufferSupport and applied it via substitutions until issue Implement method handle support for GraalVM Native Image oracle/graal#2761 is fixed. See more information here: https://github.com/apache/camel-quarkus/compare/master...JiriOndrusek:1839-leveldb-native?expand=1#diff-799b2f59808725c0c8b95feacb5b4c8aR25

From my POV this PR could be merged. I'm currently starting to investigate support of C++ levelDB. If this approach will work, I'd enhance this extension (which is working as is)

What do you think, @ppalaga ?

[ ] An issue should be filed for the change unless this is a trivial change (fixing a typo or similar). One issue should ideally be fixed by not more than one commit and the other way round, each commit should fix just one issue, without pulling in other changes.
[ ] Each commit in the pull request should have a meaningful and properly spelled subject line and body. Copying the title of the associated issue is typically enough. Please include the issue number in the commit message prefixed by #.
[ ] The pull request description should explain what the pull request does, how, and why. If the info is available in the associated issue or some other external document, a link is enough.
[ ] Phrases like Fix # or Fixes # will auto-close the named issue upon merging the pull request. Using them is typically a good idea.
[ ] Please run mvn process-resources -Pformat (and amend the changes if necessary) before sending the pull request.
[ ] Contributor guide is your good friend: https://camel.apache.org/camel-quarkus/latest/contributor-guide.html

@JiriOndrusek JiriOndrusek force-pushed the 1839-leveldb-native branch 2 times, most recently from b9b7044 to 1a91fbb Compare October 12, 2020 12:23
Copy link
Contributor

@ppalaga ppalaga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Preferred implementation of LevelDb is in C++, implementation in Java is much slower (see https://github.com/dain/leveldb#leveldb-in-java). Current PR uses only Java implementation in native. I'd like to investigate possible usage of C++ library in native, but I see it more like an enhancement, because current PR works.

+1. If the LevelDB Java port works well in the native, it is a good state to start with. We can file an issue to track how much interest is there to support also the C++ implementation and consider revisiting it if enough people ask for it.

  1. Component levelDb uses serialization of DefaultExchangeHolder. I've used workaround with Jackson serialization instead. (see https://github.com/apache/camel-quarkus/compare/master...JiriOndrusek:1839-leveldb-native?expand=1#diff-5fbe79b4f16469a82d30051dc3cf564aR46)

Commented inline

  1. Java implementation of LevelDB uses methodHandle which is not supported by GraalVM. I used previous version of ByteBufferSupport and applied it via substitutions until issue oracle/graal#2761 is fixed. See more information here: https://github.com/apache/camel-quarkus/compare/master...JiriOndrusek:1839-leveldb-native?expand=1#diff-799b2f59808725c0c8b95feacb5b4c8aR25

Commented inline

From my POV this PR could be merged. I'm currently starting to investigate support of C++ levelDB. If this approach will work, I'd enhance this extension (which is working as is)

Let me review properly. so far I looked only at the areas you asked about.

Copy link
Contributor

@ppalaga ppalaga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, thanks @JiriOndrusek ! Some minor suggestions and comments inline.

@JiriOndrusek JiriOndrusek marked this pull request as ready for review October 13, 2020 10:57
@JiriOndrusek
Copy link
Contributor Author

JiriOndrusek commented Oct 13, 2020

Do not merge, please!

It seems, that current implementation has another limoitation (see https://issues.apache.org/jira/browse/CAMEL-15679?focusedCommentId=17213151&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17213151) This implementation will work only for text based payloads.

I'll try to formulate limitations.adoc and try to add some validations.

@JiriOndrusek
Copy link
Contributor Author

I've added into limitation.adoc new limitation. Binary payloads are not supported in native mode. This will be possible once camel-leveldb component is refactored (https://issues.apache.org/jira/browse/CAMEL-15679)
New test for binary payload is added, test in native mode expects failure.

Can be merged.

@ppalaga
Copy link
Contributor

ppalaga commented Oct 14, 2020

I rebased on top of the current master. Let's merge if the CI passes.

@jamesnetherton jamesnetherton merged commit 4c7e016 into apache:master Oct 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

LevelDB native support
3 participants