Repository navigation
Releases
v5.8.4
Extends SafeDisk.with_numpy() to numpy data in keys and to more numpy types in values, and fixes the data stored for short-lived array views. The cache.db schema and the file layout are unchanged, and entries 5.8.3 could store are written byte for byte as before.
Changes
SafeDisk.with_numpy()accepts numpy booleans, integers, floats, complex numbers, strings and bytes in keys, also inside tuples, lists, dicts, sets and frozensets, and stores them as the Python valuesitem()gives, soint64(1)and1name the same entry. They raisedTypeError(PR #47).SafeDisk.with_numpy()accepts arrays in keys, alone or inside other keys, and asmemoizearguments. Every array in a key is stored in C order, so an equal array of the same dtype and shape names the same entry; another dtype, shape or time unit names another one. A key without numpy keeps the bytes earlier versions wrote (PR #47).SafeDisk.with_numpy()storesdatetime64andtimedelta64scalars with their unit, arrays of objects element by element (each element checked like any value), record arrays and masked arrays with their type, and memory maps as the plain array of their data. They raisedTypeError. A time unit with a multiple of more than nine digits still raisesTypeError. These entries, and keys holding numpy data, read as unreadable on 5.8.3 and earlier (PR #47).
Fixes
SafeDisk.with_numpy()stores each short-lived array a value hands to pickle with its own data. A value whose pickling made temporary views of one array, such as an object whose__reduce__yields slices, could be stored with one view's data in place of later ones, which then read back wrong. Affected 5.8.2 and 5.8.3 (PR #47).
v5.8.3
Fixes a regression from 5.8.2 and several SafeDisk issues found by testing downstream libraries, and updates the documentation. The cache.db schema and the file layout are unchanged.
Security
SafeDisk.with_typesrefuses the classes of scipy, torch, bitstruct and pyahocorasick, by class, by module name and throughDISKCACHE_SAFE_MODULES, as it refuses numpy, so a crafted scipy sparse matrix, pyahocorasick automaton or bitstruct compiled format cannot crash the reader and torch objects cannot run code through the cache (PR #42).
Fixes
Cache.getworks again in subclasses that override_conto keep their rows in another database file, as subclasses written for 5.6.3 do. Since 5.8.2 its fast path took its cursor from a connection tocache.db, so every read raisedOperationalError: no such table: Cache(PR #42).SafeDiskstores the classes in its allowlist, such asintandstr, somemoize(typed=True)works with it (PR #42).SafeDisk.with_numpy()storesdatetime64andtimedelta64arrays, which raisedValueErrorfrompickle.PickleBuffer(PR #42).- Every
SafeDiskchecks a key or value outside the standard-library types against the names its pickle looks up, as the disks fromwith_typesalready did. A plainSafeDisktherefore stores a dict subclass that pickles as adict, such as DRF'sReturnDict, and reads back adict(PR #42). - Refusal messages say what to do: a refused function no longer points to
with_types(), which allows only classes; numpy objects point towith_numpy(); an array of objects mentions pandas DataFrames and what to store instead; and a class from a packagewith_typesrefuses says so (PR #42). incranddecronCacheandFanoutCachetype-check withdefault=None, which makes a missing key raiseKeyError: the stubs typeddefaultasint, so type checkers rejected the call.tox -e mypynow also checkstests/typecheck_stubs.py, a file of calls that must type-check against the stubs (PR #41).- The
modulesargument ofSafeDisk.with_typestype-checks with a single string or any iterable of strings, as the code accepts (PR #42).
Docs
- The CVE page and the migration guide say up front that
JSONDiskdoes not protect reads, because it reads each row as itsmodecolumn says, and point toSafeJSONDisk. The migration guide explains that Dash'sDiskcacheManagertakes only an upstream cache and keeps its signing secret there, so its folder should be its own (PR #42). - The tutorial no longer says that
memoize(typed=True)needs the defaultDisk(PR #42).
v5.8.2
Adds SafeJSONDisk and SafeDisk.with_numpy(), keeps float NaN values, and makes get and set faster. The cache.db schema and the file layout are unchanged.
Security
-
SafeJSONDiskstores JSON likeJSONDisk, reads and writes the same files, and never reads a pickle.JSONDiskwrites only JSON but inherits the pickle reads ofDisk, so a crafted row still ran code and a crafted path could still delete a file outside the cache.SafeJSONDiskrefuses pickled keys and values without parsing them and confines paths asSafeDiskdoes (PR #33). -
SafeDisk.with_numpy()returns aSafeDisksubclass that stores numpy arrays in the.npyformat instead of pickling them, through pickle's persistent IDs, anywhere in a value including inside classes allowed withwith_types, which combines with it in either order, aswith_optionsdoes. numpy scalars read back as the same type. Reading a large array takes no second copy of it. Headers are checked against the data, so forged arrays are refused rather than raisingMemoryError, and every forged entry raisesUnpicklingErroror reads as a miss withon_error='miss'. Object arrays,ndarraysubclasses and arrays the reader would refuse raiseTypeErrorat write time. Entries holding numpy data can't be read by other disks, including a plainSafeDisk, or by 5.6.3, which raiseUnpicklingError. numpy stays optional, andSafeDiskwithoutwith_numpy()still refuses it; itswith_typeserror now points towith_numpy()(PR #34). -
DISKCACHE_SAFE_TYPESandDISKCACHE_SAFE_MODULESnow apply whenDISKCACHE_DISKnames aSafeDisksubclass, not only withsafe(PR #34).
Fixes
Cache.getis as fast on a hit as in 5.6.3 again; since 5.7.0 it was about 5% slower (PR #35).- A float NaN stored as a value is kept, on every disk. It was written to SQLite, which stores NaN as NULL, so it read back as
None. It is now stored as a pickle, which 5.6.3 reads too (PR #34). memoize_stampedeworks withSafeDisk: the key of its early recomputation marker held the internalENOVALconstant, whichSafeDiskrefuses, so a call raisedTypeErroronce a cached value neared expiry (PR #32).
Performance
Cache.getis about 15% faster on a hit and 12% on a miss, andDjangoCache.getabout 16%: a plain lookup reads only the value columns, on a cursor reused per thread (PR #38).Cache.setis about 15% faster, andDjangoCache.setabout 17%: each write runs fewer SQL statements and less Python (PR #37).
Packaging
- The package installs on Python 3.8 and later (PR #36).
v5.8.1
Ships type stubs and installs on Python 3.9. The code and the files on disk are unchanged from 5.8.0.
Packaging
- The package ships type information:
py.typedand.pyistubs covering the API, includingSafeDisk,SafeDisk.with_types,Disk.with_options,configure,default_disk,ReadOnlyError,read_only,store_time,iterkeys(tag=...)and thediskarguments ofDeque,IndexandFanoutCache.dequeandindex, so mypy and pyright check code that uses the fork without a separate stubs package.mypy.stubtestreports no differences from the runtime (PR #29). requires-pythonis>=3.9, so the fork installs on Python 3.9; the test suite passes there unchanged (PR #29).
v5.8.0
The package is now imported as diskcache_community, so it installs next to the upstream diskcache package instead of replacing it. The cache.db schema, the file layout, the serialization modes, the memoize key format and the persisted settings are unchanged, so a 5.6.3 process opens a directory written by 5.8.0 and the reverse. SafeDisk stays opt-in and gains an allowlist for your own classes, a process-wide default disk selects it for caches that libraries create, and caches can treat unreadable entries as missing.
Security
SafeDisk.with_types(*types, modules=())returns aSafeDisksubclass whose allowlist adds the given classes, matched by identity, and every class defined in the named modules or below them. A module name may only reach a class defined in that module: a dotted name such asos.system, reached through a module that importsos, a class the module merely imports, a module and a dunder attribute are refused, and reading never imports a module, so a pickle cannot run a module's top-level code. numpy classes are refused (UNSAFE_MODULES), because numpy restores a dtype's flags from the pickle without checking them and a crafted array pickle can corrupt memory. Writes stay exact: a value outside the standard-library types is pickled and the names its pickle looks up are checked against the same allowlist, so a value the reader would refuse raisesTypeErrornaming the class before anything is written. Callingwith_typesagain adds to the allowlist, and aCache,FanoutCache,Deque,IndexorDjangoCacheusing the returned class pickles as before. The plainSafeDiskis unchanged (PR #23).
Docs
SECURITY.mddescribes what 5.8.0 adds for CVE-2025-69872 and links the steps (PR #28).- Two new pages: "Fix CVE-2025-69872 in diskcache" gives the steps to read caches with
SafeDisk, including caches that libraries create, your own classes and entries already in the cache, and lists what any restricted disk has to cover; "Migrating from diskcache 5.6.3" covers the dependency, imports, Django settings, running next to upstreamdiskcache, the behaviour changes since 5.6.3 and going back. Both open the documentation and are linked from the README (PR #27). - A new benchmarks page, measured on 2026-10-12, compares diskcache-community with diskcache 5.6.3, memcached, Redis, lmdb, sqlitedict, shelve and the Django cache backends, with four charts. The scripts that produce it are in
tests/benchmarks(driver.pyruns the stores,report.pywrites the page and charts from the JSON results). The 2019 pages from upstream stay as the old benchmarks, labelled as such and linked from the new page, and the README header, features and documentation list link to the new page (PR #26). - Shared links show a project card: a social preview image in
.github/, and Open Graph tags on the documentation throughsphinxext-opengraph(PR #21). - A logo: the README opens with a centered header (logo, name, tagline, links, badges) in the style of the other community forks, and the documentation shows the logo and a favicon (PR #20).
- The README is rewritten in Markdown around install, usage and the fork story; the comparison tables move to a new
comparisonspage in the documentation, which gets its own landing page, and the 2016 testimonials are dropped (PR #19). - The documentation is published on Read the Docs at https://diskcache-community.readthedocs.io/, built from
masteron every push; the README links point there instead of at the.rstsources (PR #18).
Features
Disk.with_options(on_error='miss')returns a disk type, on any disk class includingSafeDiskand the classeswith_typesreturns, whose caches delete an entry they find but cannot read back (a corrupt or stale pickle, a refused type, a path outside the cache, an undecodable text file) and treat it as missing:getreturns its default,cache[key]raisesKeyError,pop,pull,peekandpeekitemmove on, and iteration skips unreadable keys.Index.items()andvalues()skip such entries too. The first such entry per cache is logged as a warning on thediskcache_communitylogger; a read-only cache does not delete it, and itspeekandpeekitemraise. A missing value file,MemoryErrorandRecursionErrorare never treated as unreadable. The default,on_error='raise', keeps raising as before, and the option is a class attribute that is never persisted (PR #25).configure(disk)sets the disk type of everyCache,FanoutCache,Deque,IndexandDjangoCachecreated afterwards without a disk, anddefault_disk()returns the type in effect. Withoutconfigure, the environment decides:DISKCACHE_DISK=safeselectsSafeDisk, withDISKCACHE_SAFE_TYPESandDISKCACHE_SAFE_MODULESpassed toSafeDisk.with_types, andDISKCACHE_DISKalso takesdefaultor the import path of a disk class. A disk passed explicitly always wins, so libraries that choose their own disk keep it; caches that already exist keep theirs, and nothing is persisted. Thediskarguments now default toNone, meaning that type (PR #24).
Packaging
- Breaking: the package is imported as
diskcache_communityinstead ofdiskcache, so it installs next to the upstreamdiskcachedistribution without overwriting it, and a project can move to the fork while other dependencies still requirediskcache. The public API, thecache.dbschema, the file layout and the serialization are unchanged, so cache directories are shared with 5.6.3 in both directions. Django settings namediskcache_community.DjangoCacheanddiskcache_community.SafeDisk. The import-time warning about an upstreamdiskcacheinstalled in the same environment is gone, and a CI job installs both distributions together, runs the golden tests in both directions and checks that uninstalling either leaves the other working (PR #22). LICENSEnow holds the full Apache License 2.0 text instead of the short boilerplate notice, so GitHub and package indexes detect the license; the copyright and fork notice stay inNOTICE(PR #21).
v5.7.0
First release under the diskcache-community name: upstream commit ebfa37c plus the changes below. The import name stays diskcache; the cache.db schema, the file layout, the serialization modes, the memoize key format and the persisted settings are unchanged, so a 5.6.3 process opens a directory written by 5.7.0 and the reverse.
Security
- Add
SafeDisk, an opt-inDiskfor CVE-2025-69872: it unpickles only an allowlist of standard-library types, raisesTypeErrorat write time for anything the allowlist could not read back, and confines the paths read from thefilenamecolumn to the cache directory.DequeandIndextake adiskargument,FanoutCache.cache(),deque()andindex()honour theirs, and the Django backend gains aDISKoption. The defaultDiskis unchanged, so every 5.6.3 cache keeps opening (#11). - Create every directory the cache makes with mode
0700, including missing parents and the value subdirectories, and leave existing directories alone; includes upstream pull request 359 (#7). - Warn at import time when the upstream
diskcachedistribution is installed next todiskcache-community, since both own the same package directory (#9).
Features
- Read-only caches (upstream issue 113): opening an existing cache writes nothing unless a setting differs from the stored one, so a
cache.dbsnapshot stays byte-identical across runs; a cache whose database or directory is not writable opens read-only, as doesCache(directory, read_only=True), with reads, iteration,check()and memoize hits working and every write raisingReadOnlyError, a subclass ofsqlite3.OperationalError(#17). store_time=TrueonCache.get,FanoutCache.getandDjangoCache.getappends the store time to the returned tuple after any other requested metadata (upstream issue 330) (#16).Cache.iterkeys(tag=...)iterates the keys with a given tag, using the tag index when enabled; includes upstream pull request 372 (upstream issue 341) (#16).
Fixes
- Support SQLite builds compiled with
SQLITE_DQS=0, such as conda-forge's sqlite builds of early 2025: single-quoted literals in theSettingstriggers and statistics counters (upstream pull request 311), and caches written by earlier versions get their triggers rewritten on first open when the stored text differs (#3). FanoutCache.__getattr__raisesAttributeError, sohasattr()works (upstream pull request 366), andFanoutCache.getandpopreturn the documented tuple when a shard times out (upstream pull request 367) (#3).- Write bytes values without the
io.BytesIOcopy (upstream pull request 333) and fix a README typo (upstream pull request 365) (#3). - Accept either path separator in stored filenames, so a cache written on Windows opens on other systems and
check()stays clean across platforms (#6). - A parameter name in the
ignoreset ofmemoizematches the argument whether it is passed positionally or by keyword; keys of calls that do not pass an ignored parameter positionally are unchanged (#8). - Close the SQLite connections of threads that have exited, from
close()and when aCacheis garbage collected; keep connections inherited acrossfork()unused in the child, as SQLite requires, and letfork()wait for transactions in flight in other threads (upstream issues 295, 325 and 336; upstream pull request 299 was not taken because it serializes every transaction) (#13).
Packaging
- Move packaging from
setup.pytopyproject.tomlunder the distribution namediskcache-community; the import name staysdiskcache. Includes upstream pull request 326, which removes thesetup.py testcommand (#2).
Docs
- Add the fork notice and migration steps to the README, plus
NOTICE,SECURITY.md,CONTRIBUTING.mdand this changelog; point the documentation and repository links at this fork and remove the upstream analytics, logo and funding links from the docs (#4). - Record the changes in this changelog and extend the README fork notice (#12).
Tests and CI
- Replace the upstream CI with one
ci.ymlworkflow: lint checks, tests on Python 3.10 to 3.14 on Linux, macOS and Windows, Django 4.2 and 6.0 jobs, and an import check without Django (#1). - Add golden compatibility fixtures written by diskcache 5.6.3 for every container, eviction policy, tag index, statistics,
JSONDisk, memoized calls and file-backed values, with tests that read them with the current code and a reverse-direction test that reads a fork-written cache with 5.6.3 (#5). - Run the test suite in CI against a SQLite built from source with
SQLITE_DQS=0(#10). - Rework the release workflow: tag-version check, build and twine check, TestPyPI dry run by dispatch, PyPI publish through trusted publishing behind the
pypienvironment, and a GitHub release with this changelog section (#14). - Make the
incranddecrexpiry tests and the Django tests deterministic under pytest-xdist, so slow runners no longer fail them (#15).