Allow single process access to multiple repos. #20
Conversation
These changes modify the behavior of the Environment and hdf5 FileHandler singleton classes to instantiate singleton instances based on the path of the checked out repository. This seems to fix all outstanding bugs related to these odd behaviors, but is untested as of yet. The TransactionRegister singleton did not need to be updated because it the reference counting scheme is isolated for each lmdb environment which is passed to it. As different repository instances in the same process will now have unique lmdb environment handles, it does not require any modifications to it's current behavior.
There was a problem hiding this comment.
It works for me. Just make sure we return an error in case we don't pass the repo_path in.
The only gotcha to consider is that the singleton(s) will now live indefinitely. There's no way to get rid of them - which is what a singleton does anyway, but now we have more then one, so we should have a way to get rid of some of them if they're not used. We can't rely on GC, maybe we can do it on close.
|
So this is an interesting point. Ive been purposefully avoiding putting a close method on the repo because i think it would be annoying for users to have to remember. Only checkouts need to be closed, which should not close the entire repo context when they are called. Not sure there's a solution if im stuck on not putting in an explicit |
If this is not a pathlike object, a directory on disk, or a directory which the user agent does not have write permissions for then an exception will be thrown and the no repository object will be instantiated.
|
I think we should at least give the option of cleaning up the singletons (either a specific one or all of them). In general I'm not 100% sure why we need a singleton (or something like it) design-wise, can you explain in a one-liner? |
Fix behavior which prevented a single python process from working on multiple repositories.
Description
These changes modify the behavior of the Environment and hdf5 FileHandler singleton classes to instantiate singleton instances based on the path of the checked out repository. This seems to fix all outstanding bugs related to these odd behaviors, but is untested as of yet.
The TransactionRegister singleton did not need to be updated because it the reference counting scheme is isolated for each lmdb environment which is passed to it. As different repository instances in the same process will now have unique lmdb environment handles, it does not require any modifications to it's current behavior.
Motivation and Context
Addresses:
How Has This Been Tested?
Types of changes
Checklist:
ping @hhsecond for sanity check.