Skip to content

Commit

Permalink
Decouple serialization logic from CacheBackend classes
Browse files Browse the repository at this point in the history
Reworked the means by which values are serialized and deserialized from
backends, and provided for custom serialization of values.  Added the
:paramref:`.CacheRegion.serializer` and
:paramref:`.CacheRegion.deserializer` parameters which may be set to any
serializer.

Serialization and deserialization now take place within the
:class:`.CacheRegion` so that backends may now assume string values
in all cases.  This simplifies the existing backends and also makes
custom backends easier to write and maintain.

Additionally, the serializer is now applied to the user-defined value
portion of the :class:`.CachedValue` and not to the metadata or other
portions of :class:`.CachedValue` object itself, so the serialized portion
is effectively a "payload" within the larger :class:`.CachedValue`
structure that is passed as part of the larger string format.  The overall
format is a separate JSON of the cached value metadata, followed by the
serialized form.  This allows for end-user serialization schemes that are
hardwired to the values themselves without the need to serialize dogpile's
internal structures as well.

Pull request courtesy Alessio Bogon.

As this is a major refactor and we are Python 3, adds a significant
start to pep-484 annotations.

Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Closes: #191
Pull-request: #191
Pull-request-sha: 06f449c

Change-Id: I71bce715c79b455b88aa9c17e626e929f451e1d9
  • Loading branch information
youtux authored and zzzeek committed Nov 11, 2020
1 parent 3e77d69 commit 21c38da
Show file tree
Hide file tree
Showing 16 changed files with 845 additions and 296 deletions.
10 changes: 10 additions & 0 deletions docs/build/api.rst
Expand Up @@ -8,6 +8,7 @@ Region

.. automodule:: dogpile.cache.region
:members:
:show-inheritance:

Backend API
=============
Expand All @@ -18,40 +19,49 @@ how to alter the behavior of existing backends.

.. automodule:: dogpile.cache.api
:members:
:show-inheritance:


Backends
==========

.. automodule:: dogpile.cache.backends.memory
:members:
:show-inheritance:

.. automodule:: dogpile.cache.backends.memcached
:members:
:show-inheritance:

.. automodule:: dogpile.cache.backends.redis
:members:
:show-inheritance:

.. automodule:: dogpile.cache.backends.file
:members:
:show-inheritance:

.. automodule:: dogpile.cache.proxy
:members:
:show-inheritance:

.. automodule:: dogpile.cache.backends.null
:members:
:show-inheritance:

Exceptions
==========

.. automodule:: dogpile.cache.exception
:members:
:show-inheritance:

Plugins
========

.. automodule:: dogpile.cache.plugins.mako_cache
:members:
:show-inheritance:

Utilities
=========
Expand Down
32 changes: 32 additions & 0 deletions docs/build/unreleased/191.rst
@@ -0,0 +1,32 @@
.. change::
:tags: feature, region
:tickets: 191

Reworked the means by which values are serialized and deserialized from
backends, and provided for custom serialization of values. Added the
:paramref:`.CacheRegion.serializer` and
:paramref:`.CacheRegion.deserializer` parameters which may be set to any
serializer.

Serialization and deserialization now take place within the
:class:`.CacheRegion` so that backends may now assume string values
in all cases. This simplifies the existing backends and also makes
custom backends easier to write and maintain.

Additionally, the serializer is now applied to the user-defined value
portion of the :class:`.CachedValue` and not to the metadata or other
portions of :class:`.CachedValue` object itself, so the serialized portion
is effectively a "payload" within the larger :class:`.CachedValue`
structure that is passed as part of the larger string format. The overall
format is a separate JSON of the cached value metadata, followed by the
serialized form. This allows for end-user serialization schemes that are
hardwired to the values themselves without the need to serialize dogpile's
internal structures as well.

Existing custom backends should continue to work without issue; they
now have the option to forego any separate serialization steps, and
can also subclass a new backend :class:`.BytesBackend` that marks them
as a backend that only deals with bytes coming in and out; all
internal serialization logic from such a backend can be removed.

Pull request courtesy Alessio Bogon.

0 comments on commit 21c38da

Please sign in to comment.