-
Notifications
You must be signed in to change notification settings - Fork 58
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Proposal implementation for handling model attachments
The idea is that we sometimes want to attach files to models, such as HTML reports or the like, and in the backend, these files should be stored separately, to allow easy access. Here we're implemeting this idea for the 'FileLike' model persister and testing it for the 'File' subclass. This should work for 'Rest' and 'S3' as well, but I thought it's best to add tests when we all agreed on the idea. Usage is demonstrated in 'TestFileAttachments'. The contract is as follows: Use 'palladium.util.annotate' to add an arbitrary number of attachments to the model, like so: ``` annotate(model1, {'attachments/myatt.txt': 'aGV5', 'attachments/my2ndatt.txt': 'aG8='}) ``` Note that the keys of such attachments must start with 'attachments/', with the rest indicating a filename. The values must be base64 encoded but converted from bytes to strings. This is arguably a bit awkward, but we do this because the attachments dictionary must in general be JSON serializable, and using bytes would violate this. When 'model1' is persisted, 'FileLike' will create one file for each attachment and call them 'model-1-myatt.txt' and 'model-1-my2ndatt.txt'. The implementation chooses to use flat files rather than a folder to hold all attachments for a given model. This is done so that we do not need to add extra methods to 'FileLikeIO' (such as mkdir), which means we should get support for other 'FileLike' implementations such as 'Rest' and 'S3' for free. Moreover, the attachments will be removed from the model's pickle and from the metadata files, in order not to blow up the size of those. When the model is loaded back through the model persister, the attachments are loaded and put back into the model's metadata dictionary. What's a good time to add the attachments to the model? Use the 'write_model_decorators' pluggable decorator hook to add a decorator that adds your attachment just before it's persisted. A toy example: ``` def my_write_model_decorator(self, model): report = my_make_report(model) # assume returns an HTML string report_encoded = b64encode(report.encode('utf-8')).decode('ascii') annotate(model, {'attachments/report.html': report_encoded}) ``` Let me know what you think. Once we've settled on the right way to do this, we'll put this into proper docs and examples.
- Loading branch information
Showing
2 changed files
with
116 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters