New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Concurrency support for multiple processes (1 exclusive initializer / n readers) #182

Closed
cmumford opened this Issue Sep 9, 2014 · 15 comments

Comments

Projects
None yet
@cmumford
Copy link
Contributor

cmumford commented Sep 9, 2014

Original issue 176 created by shri314 on 2013-06-10T20:03:38.000Z:

Can the designers of leveldb explain the rational behind the design decision of not supporting multiple processes in leveldb implementation?

The documentation clearly says, under Concurrency section that: "A database may only be opened by one process at a time. The leveldb implementation acquires a lock from the operating system to prevent misuse."

Currently I can see that when one process opens level db, it uses fcntl with RW lock (exclusive lock). However this is a severely limiting, as no other process can ever open the same database even if it wants to just inspect the database contents for RDONLY purposes.

The use case for example is - one process exclusively opens leveldb database and fills up the database, then closes it. Then n different processes start reading that database.

@cmumford

This comment has been minimized.

Copy link
Contributor

cmumford commented Sep 9, 2014

Comment #1 originally posted by cbsmith on 2013-10-04T22:36:44.000Z:

On most platforms, fcntl locks are advisory. Mandatory locks require specific additional steps are taken (on Linux, you have to have enabled it for both the file system and the file). So, if you want to read the file directly, you can totally do that.

In the case of multiple read-only readers without altering the code base, you could simply copy the file for each reader. Yes, it will be inefficient (though not on file systems that dedupe data), but then again, so would having multiple leveldb processes running as they wouldn't be able to share their memory/buffer/etc.

So, this doesn't seem like a good feature for LevelDB to implement. If you really feel you need it, you can always subclass the PosixEnv and provide your own LockFile and UnlockFile methods whose logic suits your unique use case.

Out of curiosity... why not just use multiple threads instead of multiple processes?

@cmumford

This comment has been minimized.

Copy link
Contributor

cmumford commented Sep 9, 2014

Comment #2 originally posted by johdro@mpi-inf.mpg.de on 2013-12-19T15:48:19.000Z:

Like in our case, where the key-value object is constructed once and remains static, having something like a constant access object which opens the file RDONLY and that does not alter the backend files, can be used concurrently in threads without locking and concurrently across processes does indeed seem appealing to me.

@sguada

This comment has been minimized.

Copy link

sguada commented Dec 30, 2014

We use leveldb within Caffe but the limitation of only allowing one process to read the DB is very limiting. Since we create each DB only once, and use for different experiments it will be great if it could be open multiple times with RDONLY.

@fatso83

This comment has been minimized.

Copy link

fatso83 commented Jan 5, 2015

@sguada couldn't this be fixed by using Unix domain sockets? One process doing the bidding of the others through sockets.

@sguada

This comment has been minimized.

Copy link

sguada commented Jan 5, 2015

@fatso83 that will make things quite complicated, I think just not blocking the DB when having multiple readers will be far easier.

@juliangruber

This comment has been minimized.

Copy link

juliangruber commented Jan 6, 2015

for node.js, we solved this through https://github.com/juliangruber/multilevel, a library that exposes the local leveldb api over the network, so you don't need to change any code. maybe that works for you as well?

@sguada

This comment has been minimized.

Copy link

sguada commented Jan 6, 2015

@juliangruber thanks for the pointer, but we need a really fast access to leveldb, since typically have 200Gb DB with sequential readings speeds over 10Mb/s. Therefore a native access to leveldb will be the best.

@samritchie

This comment has been minimized.

Copy link

samritchie commented Apr 20, 2015

I’d also be interested in this - I’d like to use leveldb in an iOS App Group container. This is a sandboxed directory that can be accessed by different apps (or an app & its extensions), but generally the processes can’t communicate with each other at all. An 'open read-only' option like SQLite would be ideal.

@rightgenius

This comment has been minimized.

Copy link

rightgenius commented Aug 27, 2015

Multiple processes scenario for Service and Activity is very common in Android. The lock strategy limits the app's background service accessing the database. It would be great if leveldb supports this.

@cmumford cmumford removed their assignment Jan 16, 2016

@davidblurton

This comment has been minimized.

Copy link

davidblurton commented Feb 2, 2016

I'd really like the option to open a leveldb read only. Even if it wasn't enforced, it would be nice to get a connection without creating the lock file, or override checking if the lock file exists.

@tamsky

This comment has been minimized.

Copy link

tamsky commented Apr 19, 2016

I'm interested in using leveldb as a static lookup table. Guaranteed to be read-only at all times. Can I get a connection without a lock file?

@emschwartz

This comment has been minimized.

Copy link

emschwartz commented Nov 22, 2017

Any word on this issue? I also have a use case like those above where it would be really useful to be able to read from a leveldb instance from a separate process.

@pwnall

This comment has been minimized.

Copy link
Member

pwnall commented Nov 22, 2017

One writer / multiple readers: seems like it'd require serious changes to the system.

No writer / multiple readers: you might be able to do this with a read-only env and manifest reuse turned on. If you have a lot of reads and no writes, a different storage system might be more suitable.

In either case, this issue has been open for a while, and it seems like none of the maintainers is interested in implementing this. Sorry!

@pwnall pwnall closed this Nov 22, 2017

@emschwartz

This comment has been minimized.

Copy link

emschwartz commented Nov 22, 2017

Ok, guess we have to switch to using rocksdb instead :/

@forgetso

This comment has been minimized.

Copy link

forgetso commented Nov 30, 2017

This is pretty frustrating. I've just learned enough Go to build a sharded LevelDB reader to find out that I can't create the DB instances in the partitions. Should've been testing with more than 1 partition, I guess...

@adambabik adambabik referenced this issue May 14, 2018

Closed

Mailserver to discard old messages #948

1 of 1 task complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment