Allow to pickle db models by removing the current connection. #5641
Conversation
|
Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry. |
There was a problem hiding this comment.
Copilot reviewed 2 out of 3 changed files in this pull request and generated no comments.
Files not reviewed (1)
- docs/changelog.rst: Language not supported
Comments suppressed due to low confidence (1)
beets/dbcore/db.py:725
- [nitpick] Consider removing the '_db' key using 'del state["_db"]' instead of setting it to None, to ensure it is fully excluded from the pickled state.
state["_db"] = None
|
Not sure how I feel about this, given that beets doesn't require it directly. Of how much help would it be for a project that uses beets as a library to have this upstream? Note that a related issue led to the implementation of |
wisp3rwind
added
the
needinfo
|
We need this for serialization at the moment and patch it manually. I thought this is an easy enhancement without too many tradeoffs and maybe some other people might find it useful. This fix allows us to do a fully statefull import where we can continue at any point on a standard beets import. Quite usefully for our project. Currently we do the following: match: BeetsAlbumMatch | BeetsTrackMatch
# Remove db from all items as it can't be pickled
# FIXME: this should go into beets __getstate__ method
# see https://github.com/beetbox/beets/pull/5641
if isinstance(match, BeetsAlbumMatch):
for item in match.mapping.keys():
item._db = None
item._Item__album = None
for item in match.extra_items:
item._db = None
item._Item__album = NoneJust doing |
There was a problem hiding this comment.
Fair enough, I'd say this can be merged. Whether or not pickling (rather than actually consistently persisting import state to the db, maybe with a flexible attribute saying that it's incomplete) is the best solution to your original problem might be a different issue. Having pickle support seems generally in line with the vision that the dbcore module should be as generic and reusable as possible, rather than being specific to beets data model.
I think the test should go to test/dbcore.py, however. Would you mind moving it there?
wisp3rwind
added
feature
|
We are actually storing the pickled value in a database 😁 It is quite hacky but any other solution would come with a pile of other issues which tbh wasn't worth the time for us right now. |
…x#5641) ## Description This might be a quick one, depending on how you feel about it... It allows you to pickle DB model objects. I don't think this is used directly in Beets, but it might be useful in general. For instance, we encountered an issue where we wanted to quickly pickle an Item or Album. This sometimes worked and other times failed, which seemed quite inconsistent. Some DB model methods and properties have the side effect of attaching an SQLite connection to self (._db), which prevents serialization. The fix is quite straightforward, so I thought we might want to integrate this into beets directly. ## To Do - [x] Changelog - [x] Tests
…x#5641) ## Description This might be a quick one, depending on how you feel about it... It allows you to pickle DB model objects. I don't think this is used directly in Beets, but it might be useful in general. For instance, we encountered an issue where we wanted to quickly pickle an Item or Album. This sometimes worked and other times failed, which seemed quite inconsistent. Some DB model methods and properties have the side effect of attaching an SQLite connection to self (._db), which prevents serialization. The fix is quite straightforward, so I thought we might want to integrate this into beets directly. ## To Do - [x] Changelog - [x] Tests
Description
This might be a quick one, depending on how you feel about it... It allows you to pickle DB model objects. I don't think this is used directly in Beets, but it might be useful in general. For instance, we encountered an issue where we wanted to quickly pickle an Item or Album. This sometimes worked and other times failed, which seemed quite inconsistent.
Some DB model methods and properties have the side effect of attaching an SQLite connection to self (._db), which prevents serialization. The fix is quite straightforward, so I thought we might want to integrate this into beets directly.
To Do