Database Locking

[birkelbach]

We've never implemented any locking on the database. Most of the plugins operate in their own thread and all access the database at some level. It seems to be working well enough. I know in Python a lot of things are atomic that are not in other languages. I can't really convince myself that we need it and it's seems to be working really well, but it goes against the grain on multithreaded programing to not lock common resources. Comments?

[neil-d95]

I was talking with a coworker about this if we are talking about a multiple read, single write for a value. We might seen unusual thing in the data if it's reading while writing where as if we use a lock for the write process we can pull the most current data as well as mitigate odd data.

[birkelbach]

We are actually doing multiple writes and multiple reads (or at least it's possible to do so). What I understand is that any read or write of a Python builtin type is guaranteed to be atomic. Also the Python interpreter has the GIL so there is some locking going on inside the interpreter itself. I don't know to what extent that helps. My worry on locking is performance, but we may not have a performance problem. Once we start throwing a bunch more data at this thing we'll see. I should probably write some kind of test and see if I can make it fail.

[neil-d95]

Yeah that sounds like a this might need to stay out there till we get a
handle on how the GIL locking holds up.

On 06/15/16 07:30, Phil Birkelbach wrote:

We are actually doing multiple writes and multiple reads (or at least
it's possible to do so). What I understand is that any read or write
of a Python builtin type is guaranteed to be atomic. Also the Python
interpreter has the GIL so there is some locking going on inside the
interpreter itself. I don't know to what extent that helps. My worry
on locking is performance, but we may not have a performance problem.
Once we start throwing a bunch more data at this thing we'll see. I
should probably write some kind of test and see if I can make it fail.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#9 (comment),
or mute the thread
https://github.com/notifications/unsubscribe/AEjkWW6jLEMVB3o0QTbpLvnnUEOX7pWvks5qL-JQgaJpZM4I1Z6A.

[birkelbach]

If I was seeing any problems or weird data I'd throw some locking at it in a heartbeat, but it all seems to be working fine. My embedded and C programming experience says that this should not be... but Python is a different beast.

[birkelbach]

I've done a bit more research on this issue. It seems that dictionary accesses are indeed atomic in CPython but it's not specified behavior, rather it's a side effect of the implementation. It would be best if database locking was added.