Conversation
This will allow for easier instantiation of test cache managers.
This lets us make lines shorter!
Really there are only two supported stores: - Sleepycat/BDB - Sqlite
The sqlite-based MetadataCache should work on all systems (given that sqlite is in the Python standard library). Thus we can use this cache as a fall-back in case we don't manage to use Sleepycat (e.g. on Py3/Win).
Also: - remove explicit dependency on bsddb3 under py2 There are some issues installing BSD-DB on Windows (see #26, #31) so this fix should make gutenberg work on Windows Python 2 again. On Windows Python 3, we can't use the standard-library bsddb module and we can't fall-back to bsddb3 so we use the slow SQLite cache instead.
gutenberg/acquire/metadata.py
Outdated
There was a problem hiding this comment.
The set_metadata_cache() function is not just for testing, but rather an integral part of my branch. Without a way to explicitly set a cache, you can't control where the cache is stored on the filesystem, or in the new code, control which of the two backend types you want to use. This level of control is vital when using this as part of a larger service.
For an example of what I'm trying to achieve with this module, see:
https://github.com/cpeel/openlibrary/blob/095df4b3be522eef218813a0106ee2ce7934f120/openlibrary/plugins/gutenberg/pgebook.py
and
https://github.com/cpeel/openlibrary/blob/73533a3e011074e2ffbfe416f241a6aa14fb5b6e/scripts/dev-instance/populate_gutenberg_cache.py
|
Overall I like your refactor -- very clean. My one big concern is the removal of |
There was a problem hiding this comment.
Built-in python errors (almost) all end in Exception, Error, or Warning (https://docs.python.org/2/library/exceptions.html). See also PEP8: https://www.python.org/dev/peps/pep-0008/#exception-names
(I should have raised this comment against the commit when the exceptions.py file went in, as they're all this way.)
This reverts commit 7c8f7c1 and adds some more documentation on how to use the method.
|
Ship it! :) |
|
@cpeel: I did a bit of refactoring on your metadata cache manager code in order to gain an understanding of it. I'd like to get some feedback on whether I understood everything correctly before I merge.
The gist of this refactor: