Overview of python-brine
Brine is a Python module that adds support for the "true" pickling
of functions. The default behavior of the
is to reference functions by name alone. This presents a significant
problem when the function you wish to serialize is either a
or not defined at the top level.
brine module provides a way to pickle the actual
underlying code of a function, including any closures, and then
restore them again.
For more advanced features, there is the
module. A barrel is a dictionary-like interface for
brining multiple functions. Barrel's internal brining step is
recursive. This allows anonymous functions to work on closures
referring to other anonymous functions (eg: mutually recursive lambdas
and the like). It also preserves uniqueness, if you happen to add the
same function multiple times.
Before we begin with our examples, let's contrive a function to preform the pickle/unpickle dance. We'll refer to this helper throughout the remainder of the section.
from pickle import Pickler, Unpickler from cStringIO import StringIO def pickle_unpickle(value): buffer = StringIO() Pickler(buffer).dump(value) buffer = StringIO(buffer.getvalue()) return Unpickler(buffer).load()
Anonymous or inner functions
Pickle normally refuses to serialize a function that is not defined in
the top level. The
BrineFunction class wraps a function in a manner
that supports pickling, and will actually put the code and cells into
the serialization stream.
We can use
brine.brine to wrap a
FunctionType instance, and
brine.unbrine to unwrap it again.
from brine import brine, unbrine # create a function that wouldn't normally be supported via pickle myfun = lambda x: ("Why hello there, %s" % str(x)) myfun("Godzilla") # ==> "Why hello there, Godzilla" # if we tried this without the brine/unbrine wrapping, we'd get a # pickle.PicklingError raised all up in our biz myfun_redux = unbrine(pickle_unpickle(brine(myfun))) # this is now a copy of the original myfun_redux("Mothra") # ==> "Why hello there, Mothra"
Here is something more complex -- two functions sharing a captured value (a closure).
def make_actor(line): who = ["nobody special"] def notice(monster): who = str(monster) def alert(): return line % who return notice, alert actor = make_actor("Look out, it's %s!") notice, alert = actor notice("Godzilla") alert() # ==> "Look out, it's Godzilla!" # duplicate our highly trained actor actor_redux = unbrine(pickle_unpickle(brine(actor))) notice_redux, alert_redux = actor_redux # our copy of the actor functions come out sharing their own new # closure cell alert_redux() # ==> "Look out, it's Godzilla!" notice_redux("Mothra") alert_redux() # ==> "Look out, it's Mothra!"
Bound instance methods
Pickle normally refuses to serialize bound instance methods. This is
somewhat odd, because it can be done by name. The
can be used to wrap a bound instance or class method. Note that
because a bound method needs to be associated with an object, that
instance will also need to support pickling (and hence will need its
class definition available at the top level).
BrineMethod is entirely name-based -- it doesn't try to pickle
underlying class code.
# setup a simple class for us to work over class Obj(object): def __init__(self, value=None): self.value = value def get_value(self): return self.value def set_value(self, value): self.value = value inst = Obj("Tacos") getter = inst.get_value setter = inst.set_value setter("Carrots") getter() # ==> "Carrots" # a little dance to brine and unbrine both bound methods tmp = (getter, setter) tmp = unbrine(pickle_unpickle(brine(tmp))) n_getter, n_setter = tmp n_getter() # ==> "Carrots" n_setter("Sandwich") n_getter() # ==> "Sandwich" # the original is unaffected getter() # ==> "Carrots"
- Python 2.6 or later (no support for Python 3, the underlying function fields differ a bit)
In addition, following tools are used in building, testing, or generating documentation from the project sources.
This module uses setuptools, so simply run the following to build the project.
python setup.py build
Tests are written as
unittest test cases. If you'd like to run the
tests, simply invoke:
python setup.py test
You may check code coverage via coverage.py, invoked as:
# generates coverage data in .coverage coverage run --source=brine setup.py test # creates an html report from the above in htmlcov/index.html coverage html
Documentation is built using Sphinx. Invoking the following will
produce HTML documentation in the
cd docs make html
Note that you will need the following installed to successfully build the documentation:
Documentation is also available online.
Some posibile enhancements for future minor versions
- Should we provide a wrapper for exceptions and/or stack traces?
- Should we allow users to extend
BrineObject, in the same manner that pickle can be (somewhat) extended today?
- Perhaps a PKI signing step since we are in-fact sending executable code around? This might be better relegated to a separate project.
Christopher O'Brien email@example.com
If this project interests you, you can read about more of my hacks and ideas on on my blog.
This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 3 of the License, or (at your option) any later version.
This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
You should have received a copy of the GNU Lesser General Public License along with this library; if not, see http://www.gnu.org/licenses/.