Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
reprocess.py generates huge sqlite3 temp files #1
Options may be:
added a commit
Oct 13, 2018
I didn't like the idea of option 1 or 3 so didn't bother.
I experimented with option 2 and it had no difference. Transpires that using shared cached mode is effectively the same thing as consolidating all connections into one since sqlite3 serializes everything internally anyway in this mode (but will always do it faster a the C leave than at Python level). However, this is a little slower than normal "one cache per connection" mode in true multithreaded environments, but since CPython's GIL doesn't give us that (even with
In the end I went with option 4 and refactored
So I believe this issue is resolved by commit 215e5f3.