Skip to content

Releases

v5.8.4

Choose a tag to compare

@github-actions github-actions released this 06 Nov 07:12
c5da02f

Extends SafeDisk.with_numpy() to numpy data in keys and to more numpy types in values, and fixes the data stored for short-lived array views. The cache.db schema and the file layout are unchanged, and entries 5.8.3 could store are written byte for byte as before.

Changes

  • SafeDisk.with_numpy() accepts numpy booleans, integers, floats, complex numbers, strings and bytes in keys, also inside tuples, lists, dicts, sets and frozensets, and stores them as the Python values item() gives, so int64(1) and 1 name the same entry. They raised TypeError (PR #47).
  • SafeDisk.with_numpy() accepts arrays in keys, alone or inside other keys, and as memoize arguments. Every array in a key is stored in C order, so an equal array of the same dtype and shape names the same entry; another dtype, shape or time unit names another one. A key without numpy keeps the bytes earlier versions wrote (PR #47).
  • SafeDisk.with_numpy() stores datetime64 and timedelta64 scalars with their unit, arrays of objects element by element (each element checked like any value), record arrays and masked arrays with their type, and memory maps as the plain array of their data. They raised TypeError. A time unit with a multiple of more than nine digits still raises TypeError. These entries, and keys holding numpy data, read as unreadable on 5.8.3 and earlier (PR #47).

Fixes

  • SafeDisk.with_numpy() stores each short-lived array a value hands to pickle with its own data. A value whose pickling made temporary views of one array, such as an object whose __reduce__ yields slices, could be stored with one view's data in place of later ones, which then read back wrong. Affected 5.8.2 and 5.8.3 (PR #47).

v5.8.3

Choose a tag to compare

@github-actions github-actions released this 03 Nov 18:04
ac57330

Fixes a regression from 5.8.2 and several SafeDisk issues found by testing downstream libraries, and updates the documentation. The cache.db schema and the file layout are unchanged.

Security

  • SafeDisk.with_types refuses the classes of scipy, torch, bitstruct and pyahocorasick, by class, by module name and through DISKCACHE_SAFE_MODULES, as it refuses numpy, so a crafted scipy sparse matrix, pyahocorasick automaton or bitstruct compiled format cannot crash the reader and torch objects cannot run code through the cache (PR #42).

Fixes

  • Cache.get works again in subclasses that override _con to keep their rows in another database file, as subclasses written for 5.6.3 do. Since 5.8.2 its fast path took its cursor from a connection to cache.db, so every read raised OperationalError: no such table: Cache (PR #42).
  • SafeDisk stores the classes in its allowlist, such as int and str, so memoize(typed=True) works with it (PR #42).
  • SafeDisk.with_numpy() stores datetime64 and timedelta64 arrays, which raised ValueError from pickle.PickleBuffer (PR #42).
  • Every SafeDisk checks a key or value outside the standard-library types against the names its pickle looks up, as the disks from with_types already did. A plain SafeDisk therefore stores a dict subclass that pickles as a dict, such as DRF's ReturnDict, and reads back a dict (PR #42).
  • Refusal messages say what to do: a refused function no longer points to with_types(), which allows only classes; numpy objects point to with_numpy(); an array of objects mentions pandas DataFrames and what to store instead; and a class from a package with_types refuses says so (PR #42).
  • incr and decr on Cache and FanoutCache type-check with default=None, which makes a missing key raise KeyError: the stubs typed default as int, so type checkers rejected the call. tox -e mypy now also checks tests/typecheck_stubs.py, a file of calls that must type-check against the stubs (PR #41).
  • The modules argument of SafeDisk.with_types type-checks with a single string or any iterable of strings, as the code accepts (PR #42).

Docs

  • The CVE page and the migration guide say up front that JSONDisk does not protect reads, because it reads each row as its mode column says, and point to SafeJSONDisk. The migration guide explains that Dash's DiskcacheManager takes only an upstream cache and keeps its signing secret there, so its folder should be its own (PR #42).
  • The tutorial no longer says that memoize(typed=True) needs the default Disk (PR #42).

v5.8.2

Choose a tag to compare

@github-actions github-actions released this 02 Nov 22:33
897fa65

Adds SafeJSONDisk and SafeDisk.with_numpy(), keeps float NaN values, and makes get and set faster. The cache.db schema and the file layout are unchanged.

Security

  • SafeJSONDisk stores JSON like JSONDisk, reads and writes the same files, and never reads a pickle. JSONDisk writes only JSON but inherits the pickle reads of Disk, so a crafted row still ran code and a crafted path could still delete a file outside the cache. SafeJSONDisk refuses pickled keys and values without parsing them and confines paths as SafeDisk does (PR #33).

  • SafeDisk.with_numpy() returns a SafeDisk subclass that stores numpy arrays in the .npy format instead of pickling them, through pickle's persistent IDs, anywhere in a value including inside classes allowed with with_types, which combines with it in either order, as with_options does. numpy scalars read back as the same type. Reading a large array takes no second copy of it. Headers are checked against the data, so forged arrays are refused rather than raising MemoryError, and every forged entry raises UnpicklingError or reads as a miss with on_error='miss'. Object arrays, ndarray subclasses and arrays the reader would refuse raise TypeError at write time. Entries holding numpy data can't be read by other disks, including a plain SafeDisk, or by 5.6.3, which raise UnpicklingError. numpy stays optional, and SafeDisk without with_numpy() still refuses it; its with_types error now points to with_numpy() (PR #34).

  • DISKCACHE_SAFE_TYPES and DISKCACHE_SAFE_MODULES now apply when DISKCACHE_DISK names a SafeDisk subclass, not only with safe (PR #34).

Fixes

  • Cache.get is as fast on a hit as in 5.6.3 again; since 5.7.0 it was about 5% slower (PR #35).
  • A float NaN stored as a value is kept, on every disk. It was written to SQLite, which stores NaN as NULL, so it read back as None. It is now stored as a pickle, which 5.6.3 reads too (PR #34).
  • memoize_stampede works with SafeDisk: the key of its early recomputation marker held the internal ENOVAL constant, which SafeDisk refuses, so a call raised TypeError once a cached value neared expiry (PR #32).

Performance

  • Cache.get is about 15% faster on a hit and 12% on a miss, and DjangoCache.get about 16%: a plain lookup reads only the value columns, on a cursor reused per thread (PR #38).
  • Cache.set is about 15% faster, and DjangoCache.set about 17%: each write runs fewer SQL statements and less Python (PR #37).

Packaging

  • The package installs on Python 3.8 and later (PR #36).

v5.8.1

Choose a tag to compare

@github-actions github-actions released this 19 Oct 02:56
c326c7b

Ships type stubs and installs on Python 3.9. The code and the files on disk are unchanged from 5.8.0.

Packaging

  • The package ships type information: py.typed and .pyi stubs covering the API, including SafeDisk, SafeDisk.with_types, Disk.with_options, configure, default_disk, ReadOnlyError, read_only, store_time, iterkeys(tag=...) and the disk arguments of Deque, Index and FanoutCache.deque and index, so mypy and pyright check code that uses the fork without a separate stubs package. mypy.stubtest reports no differences from the runtime (PR #29).
  • requires-python is >=3.9, so the fork installs on Python 3.9; the test suite passes there unchanged (PR #29).

v5.8.0

Choose a tag to compare

@github-actions github-actions released this 18 Oct 23:18
22527b6

The package is now imported as diskcache_community, so it installs next to the upstream diskcache package instead of replacing it. The cache.db schema, the file layout, the serialization modes, the memoize key format and the persisted settings are unchanged, so a 5.6.3 process opens a directory written by 5.8.0 and the reverse. SafeDisk stays opt-in and gains an allowlist for your own classes, a process-wide default disk selects it for caches that libraries create, and caches can treat unreadable entries as missing.

Security

  • SafeDisk.with_types(*types, modules=()) returns a SafeDisk subclass whose allowlist adds the given classes, matched by identity, and every class defined in the named modules or below them. A module name may only reach a class defined in that module: a dotted name such as os.system, reached through a module that imports os, a class the module merely imports, a module and a dunder attribute are refused, and reading never imports a module, so a pickle cannot run a module's top-level code. numpy classes are refused (UNSAFE_MODULES), because numpy restores a dtype's flags from the pickle without checking them and a crafted array pickle can corrupt memory. Writes stay exact: a value outside the standard-library types is pickled and the names its pickle looks up are checked against the same allowlist, so a value the reader would refuse raises TypeError naming the class before anything is written. Calling with_types again adds to the allowlist, and a Cache, FanoutCache, Deque, Index or DjangoCache using the returned class pickles as before. The plain SafeDisk is unchanged (PR #23).

Docs

  • SECURITY.md describes what 5.8.0 adds for CVE-2025-69872 and links the steps (PR #28).
  • Two new pages: "Fix CVE-2025-69872 in diskcache" gives the steps to read caches with SafeDisk, including caches that libraries create, your own classes and entries already in the cache, and lists what any restricted disk has to cover; "Migrating from diskcache 5.6.3" covers the dependency, imports, Django settings, running next to upstream diskcache, the behaviour changes since 5.6.3 and going back. Both open the documentation and are linked from the README (PR #27).
  • A new benchmarks page, measured on 2026-10-12, compares diskcache-community with diskcache 5.6.3, memcached, Redis, lmdb, sqlitedict, shelve and the Django cache backends, with four charts. The scripts that produce it are in tests/benchmarks (driver.py runs the stores, report.py writes the page and charts from the JSON results). The 2019 pages from upstream stay as the old benchmarks, labelled as such and linked from the new page, and the README header, features and documentation list link to the new page (PR #26).
  • Shared links show a project card: a social preview image in .github/, and Open Graph tags on the documentation through sphinxext-opengraph (PR #21).
  • A logo: the README opens with a centered header (logo, name, tagline, links, badges) in the style of the other community forks, and the documentation shows the logo and a favicon (PR #20).
  • The README is rewritten in Markdown around install, usage and the fork story; the comparison tables move to a new comparisons page in the documentation, which gets its own landing page, and the 2016 testimonials are dropped (PR #19).
  • The documentation is published on Read the Docs at https://diskcache-community.readthedocs.io/, built from master on every push; the README links point there instead of at the .rst sources (PR #18).

Features

  • Disk.with_options(on_error='miss') returns a disk type, on any disk class including SafeDisk and the classes with_types returns, whose caches delete an entry they find but cannot read back (a corrupt or stale pickle, a refused type, a path outside the cache, an undecodable text file) and treat it as missing: get returns its default, cache[key] raises KeyError, pop, pull, peek and peekitem move on, and iteration skips unreadable keys. Index.items() and values() skip such entries too. The first such entry per cache is logged as a warning on the diskcache_community logger; a read-only cache does not delete it, and its peek and peekitem raise. A missing value file, MemoryError and RecursionError are never treated as unreadable. The default, on_error='raise', keeps raising as before, and the option is a class attribute that is never persisted (PR #25).
  • configure(disk) sets the disk type of every Cache, FanoutCache, Deque, Index and DjangoCache created afterwards without a disk, and default_disk() returns the type in effect. Without configure, the environment decides: DISKCACHE_DISK=safe selects SafeDisk, with DISKCACHE_SAFE_TYPES and DISKCACHE_SAFE_MODULES passed to SafeDisk.with_types, and DISKCACHE_DISK also takes default or the import path of a disk class. A disk passed explicitly always wins, so libraries that choose their own disk keep it; caches that already exist keep theirs, and nothing is persisted. The disk arguments now default to None, meaning that type (PR #24).

Packaging

  • Breaking: the package is imported as diskcache_community instead of diskcache, so it installs next to the upstream diskcache distribution without overwriting it, and a project can move to the fork while other dependencies still require diskcache. The public API, the cache.db schema, the file layout and the serialization are unchanged, so cache directories are shared with 5.6.3 in both directions. Django settings name diskcache_community.DjangoCache and diskcache_community.SafeDisk. The import-time warning about an upstream diskcache installed in the same environment is gone, and a CI job installs both distributions together, runs the golden tests in both directions and checks that uninstalling either leaves the other working (PR #22).
  • LICENSE now holds the full Apache License 2.0 text instead of the short boilerplate notice, so GitHub and package indexes detect the license; the copyright and fork notice stay in NOTICE (PR #21).

v5.7.0

Choose a tag to compare

@github-actions github-actions released this 11 Oct 21:59
b3ef81c

First release under the diskcache-community name: upstream commit ebfa37c plus the changes below. The import name stays diskcache; the cache.db schema, the file layout, the serialization modes, the memoize key format and the persisted settings are unchanged, so a 5.6.3 process opens a directory written by 5.7.0 and the reverse.

Security

  • Add SafeDisk, an opt-in Disk for CVE-2025-69872: it unpickles only an allowlist of standard-library types, raises TypeError at write time for anything the allowlist could not read back, and confines the paths read from the filename column to the cache directory. Deque and Index take a disk argument, FanoutCache.cache(), deque() and index() honour theirs, and the Django backend gains a DISK option. The default Disk is unchanged, so every 5.6.3 cache keeps opening (#11).
  • Create every directory the cache makes with mode 0700, including missing parents and the value subdirectories, and leave existing directories alone; includes upstream pull request 359 (#7).
  • Warn at import time when the upstream diskcache distribution is installed next to diskcache-community, since both own the same package directory (#9).

Features

  • Read-only caches (upstream issue 113): opening an existing cache writes nothing unless a setting differs from the stored one, so a cache.db snapshot stays byte-identical across runs; a cache whose database or directory is not writable opens read-only, as does Cache(directory, read_only=True), with reads, iteration, check() and memoize hits working and every write raising ReadOnlyError, a subclass of sqlite3.OperationalError (#17).
  • store_time=True on Cache.get, FanoutCache.get and DjangoCache.get appends the store time to the returned tuple after any other requested metadata (upstream issue 330) (#16).
  • Cache.iterkeys(tag=...) iterates the keys with a given tag, using the tag index when enabled; includes upstream pull request 372 (upstream issue 341) (#16).

Fixes

  • Support SQLite builds compiled with SQLITE_DQS=0, such as conda-forge's sqlite builds of early 2025: single-quoted literals in the Settings triggers and statistics counters (upstream pull request 311), and caches written by earlier versions get their triggers rewritten on first open when the stored text differs (#3).
  • FanoutCache.__getattr__ raises AttributeError, so hasattr() works (upstream pull request 366), and FanoutCache.get and pop return the documented tuple when a shard times out (upstream pull request 367) (#3).
  • Write bytes values without the io.BytesIO copy (upstream pull request 333) and fix a README typo (upstream pull request 365) (#3).
  • Accept either path separator in stored filenames, so a cache written on Windows opens on other systems and check() stays clean across platforms (#6).
  • A parameter name in the ignore set of memoize matches the argument whether it is passed positionally or by keyword; keys of calls that do not pass an ignored parameter positionally are unchanged (#8).
  • Close the SQLite connections of threads that have exited, from close() and when a Cache is garbage collected; keep connections inherited across fork() unused in the child, as SQLite requires, and let fork() wait for transactions in flight in other threads (upstream issues 295, 325 and 336; upstream pull request 299 was not taken because it serializes every transaction) (#13).

Packaging

  • Move packaging from setup.py to pyproject.toml under the distribution name diskcache-community; the import name stays diskcache. Includes upstream pull request 326, which removes the setup.py test command (#2).

Docs

  • Add the fork notice and migration steps to the README, plus NOTICE, SECURITY.md, CONTRIBUTING.md and this changelog; point the documentation and repository links at this fork and remove the upstream analytics, logo and funding links from the docs (#4).
  • Record the changes in this changelog and extend the README fork notice (#12).

Tests and CI

  • Replace the upstream CI with one ci.yml workflow: lint checks, tests on Python 3.10 to 3.14 on Linux, macOS and Windows, Django 4.2 and 6.0 jobs, and an import check without Django (#1).
  • Add golden compatibility fixtures written by diskcache 5.6.3 for every container, eviction policy, tag index, statistics, JSONDisk, memoized calls and file-backed values, with tests that read them with the current code and a reverse-direction test that reads a fork-written cache with 5.6.3 (#5).
  • Run the test suite in CI against a SQLite built from source with SQLITE_DQS=0 (#10).
  • Rework the release workflow: tag-version check, build and twine check, TestPyPI dry run by dispatch, PyPI publish through trusted publishing behind the pypi environment, and a GitHub release with this changelog section (#14).
  • Make the incr and decr expiry tests and the Django tests deterministic under pytest-xdist, so slow runners no longer fail them (#15).