FilesystemStore: Add a size(key) method to return the file size on disk #3

Closed
wants to merge 1 commit into
from

Projects

None yet

2 participants

@rslinckx

It's useful to be able to return the size of the file on disk.

I'm not sure whether this is the right approach or if the size() should be defined on the base Store class and implemented across all backends...

@mbr
Owner
mbr commented Nov 21, 2011

I've given this some thought, and I don't think it's a good idea either.

Basically, there is no guarantee that a store knows the size of its contents without retrieving everything and checking its size (for example, think compression or encryption that needs padding). For that reason, it can never become part of the basic interface supported by all backends.

Furthermore, the parts of simplekv should be as reasonably interchangeable as possible - switching out a filesystem store for another backend is an important feature, especially since should often be up to the user to decide upon on a storage backend on a deployment-basis. Relying on a size() method of the FilesystemStore defeats this endeavor, unfortunately.

To alleviate any need you might have for the size() method though, here are two suggestions: The "best" way is to simply store metadata elsewhere. simplekv is designed to not handle metadata other than the very bare minimum it needs to operate. Tracking the size of objects currently is meant to be stored where all your other metadata resides (maybe it has an owner, a name, a date or anything else?).

If that is not an option, you can hack things together by simply subclassing FilesystemStore with your class and add the size() method there. I'm still debating whether or not to make _build_filename() part of a protected interface and document it to make hacks of this kind stable.

@mbr mbr closed this Nov 21, 2011
@rslinckx

About the _build_filename: I had to override it because I'm migrating from another system which used to store files in several subdirectories based on the file hash, so that a file with hash '123456789' would be stored in '/root/12/123456789'

Since simplekv didn't do that by default i had to subclass and override _build_filename to generate that same structure. I think it should be documented and usable for such purposes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment