A quick and easy to use python caching system
Python
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.gitignore
LICENSE Initial commit Apr 18, 2016
MANIFEST.in doc update Sep 15, 2016
README.rst
quick_cache.py
setup.cfg universal wheel Jun 8, 2017
setup.py version bump Jun 20, 2018

README.rst

quick_cache

A quick and easy to use python caching system.

You can install quick_cache via

pip install --user quick_cache

and import it in python using:

from quick_cache import QuickCache

Create the cache object as follows:

def msg(message, *args, **kwargs):
    print(message.format(*args, **kwargs), file=sys.stderr)

cache = QuickCache(base_file, quota=500, ram_quota=100, warnings=msg)

where base_file is an optional file whose content invalidates the cache (ie., when the content of the file changes the cache is invalidated; for large files it might be desirable to use the mtime in the cache object below) and msg is an optional formatting function that prints warnings (by default it's None which doesn't print anything; warnings are emitted when the actual computation is faster than reading the results from the cache or if other exceptional situations occur). quota and ram_quota are optional maximal cache sizes, both in RAM and on disk, in MB.

The caching functionality can then be used via:

with cache.get_hnd({
    # object identifying the task to cache
    # can be any combination of keys and values
    "param_a": 5,
    "input_file_c": os.path.getmtime(input_file_c), # for file change time
    ...
}) as hnd:
    if not hnd.has():
        res = hnd.write(do_compute()) # compute your result here
    else:
        res = hnd.read()
# your result is in res

The cache object used for creating the handle uniquely defines the task. The object should contain all parameters of the task and the task computation itself should be deterministic.