Provide C++ level access to LevelDB objects #44

Merged
merged 1 commit into from Aug 2, 2012

Projects

None yet

1 participant

@justmoon
Collaborator

I expect this change might be controversial, so I'm going to explain my use case first. I'm maintaining a rather large app called BitcoinJS using LevelDB as it's backend. To initialize it has to load/index a 2.4 GB dataset called the block chain. Right now I'm parsing the block chain in JS and inserting it with individual Batch.put operations. Unfortunately, this results in a huge number of calls across the JS/C++ boundary.

So what I'd like to do instead is pass a whole block to my own C++ code and have it do the parsing and inserting in straight C++.

To do that though, I need to be able to get access to the leveldb::DB and leveldb::WriteBatch handles in my C++ code.

The cleanest way I've found is to pass node-leveldb's internal handles (self) to my native object's constructor and unwrap them:

node_leveldb::JHandle* hnd = Unwrap<node_leveldb::JHandle>(args[0]->ToObject());
leveldb::DB* ldb = hnd->GetDB();
MyObj* self = new MyObj(ldb);

I then also make sure that I keep a persistent handle to self so it can't be garbage collected until MyObj is destroyed.

Unfortunately this depends on being able to get the actual LevelDB handle out of node_leveldb::JHandle, which is why I had to add some accessors.

Feel free to tell me if there is a better way of doing this, but this seems reasonably clean and it's obviously for advanced uses only.

@justmoon
Collaborator

If I don't hear any objections I'll go ahead and merge this. :)

I now have everything implemented and working. I can share the LevelDB handle between node-leveldb and my own C++ code. So all of my heavy lifting database code is now C++ resulting in a massive performance increase.

In case anyone is interested doing the same thing in their own code, here's the relevant excerpt from the .gyp on our end:

"link_settings": {
 "libraries": [
   "../node_modules/leveldb/build/Release/leveldb.node"
 ],
  "ldflags": [
     "-Wl,-rpath='$$ORIGIN/../../node_modules/leveldb/build/Release/'"
   ]
}

libraries let's it find the .node at build time and rpath lets it find it at runtime. Other than that you just include the headers and that's it. :)

@justmoon justmoon merged commit 2a96b69 into my8bird:master Aug 2, 2012
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment