ENH: on_full policy #17
Conversation
|
Hey, This feature has been working pretty well for me, Do you think we merge this wehn you get a chance to look it over? |
| self._dump(data, tmp) | ||
| bs = tmp.getvalue() | ||
| while self.disk_usage + len(bs) > self.available_disk: | ||
| self.on_full(self) |
There was a problem hiding this comment.
What is the overhead of this like when there are many keys on disk? This seems like potentially a lot of file system access.
There was a problem hiding this comment.
Should we maintain a total instead?
There was a problem hiding this comment.
The issue is that I couldn't generalize the way to account for the filesystem overhead. I wasn't positive how big all of the files would actually be on disk so I figured it was safest to ask the os.
There was a problem hiding this comment.
Perhaps we could maintain a total, like how we do for memory_usage and add or subtract from it as we add or remove files from disk.
Alternatively can you measure the overhead of how this works when we have one million files? I expect this to be non-negligible at that scale.
|
I apologize for letting this linger for so long. Thanks for the ping. |
|
Nol worries. this isn't blocking me because I can always deploy from my branch. |
Allows a user to set a maximum disk usage and a policy defining what to do when the max disk is used and a new entry must be added.
The two options are currently:
raise_: raise anOSErrorindicating you ran out of space.pop_lru: rotate the least recently used element out of the chest.Users may pass any callable here; however, these two are defined for them.