This package provides tools to serialize Zope objects and store them in a file system structure as well as to restore Zope objects from this structure.
Additional functions that go beyond the main scope of the package are managing the file system structure in a git repository and dumping schemas and table contents from a PostgreSQL database.
Ján Jockusch firstname.lastname@example.org
Viktor Dick email@example.com
git clone https://github.com/perfact/zodbsync.git
The package should be installed using
pip in the same virt-env as
zope/bin/pip install git+https://github.com/perfact/zodbsync
On PerFact systems, it should automatically be pulled by the
of the package
perfact-dbutils-zope2 and included there. If installing on an
older system, run
sudo -H /opt/zope/zope2.13/bin/pip install git+https://github.com/perfact/zodbsync
config.py as a starting point for your configuration. At the moment,
it is not automatically installed. The canonical path for the configuration is
/etc/perfact/modsync/config.py, so if you do not want to supply the path to
the configuration when calling the scripts, copy the configuration file there
(this will change in a future version).
The most important settings are:
If using ZServer (only Zope 2), set
conf_path to the
zope.conf of your Zope
instance, which must be configured to connect to a ZEO server (no standalone
instance). If using a WSGI server (Zope 2 or Zope 4), set
Inside this folder (actually, in a subfolder named
__root__), the serialized
objects are placed.
The name of a user that must be defined in the top-level UserFolder (
and which has Manager permissions. This user is considered to be the default
owner of objects if no other information is stored in the object and the
transaction that is used to upload objects to the ZODB is done by this user.
The path to the location of the Data.fs file. This is needed for the watcher
The two main tools at this moment are
perfact-zoperecord with no other options, it reads the complete
object tree and stores the objects to the file system. Each object is mapped to
a file named
__meta__ and further subfolders (if it is an object that has
contents, like a
Folder) or a file whose name starts with
__source (if it is
an object with some sort of source, like a
Page Template, a
Script (Python), a
Objects whose type is not yet supported are created with a minimal
containing only the
type and an
At the end of the recording, the last transaction ID is stored in a file called
__last_txn__ and a git commit is performed.
perfact-zoperecord is called with the
--lasttxn option, it reads this
transaction ID, gets a list of all transactions after that and trys to record
only those paths that were affected by these. The paths are extracted from the
transaction note, which works well if editing an object in the management
interface of Zope, but not necessarily if an object is changed from within a
script, if it is transferred by the ZSyncer or if objects are cut and pasted
(in the latter case, only the target of the move operation is recognized).
After running, the largest transaction ID of the list that was obtained at the
beginning is again stored in
If a specific path is required to be recorded, it can also be passed using the
--watch provides the watcher mode, a new mode of operation that
aims to bypass the shortcomings of
--lasttxn. In watcher mode, the recorder
stays alive and builds an object tree of all objects in the ZODB. Each time it
wakes up, it scans for new transactions, opens the Data.FS directly (in
read-only mode) to obtain all affected object IDs, updates its object tree and
uses it to obtain the physical paths of all affected objects. After finishing,
it sleeps for 10 seconds before waking again. This should provide an almost
live recording that does not miss any changes.
Playing objects back
perfact-zopeplayback is the other side of the coin, able to create and modify
objects in the ZODB from a file system structure as it is created by
perfact-zoperecord recurses into the objects below a given
object, removing any superfluous objects and updating existing objects so they
match their file system representation. An exception are objects that are
unsupported, which are ignored if found in the ZODB. If only a
given object itself should be updated (properties, security settings etc.),
--no-recurse can be used.
This package aims to replace similar functionality that was previously found in
python-perfact and perfact-dbutils-zope2. For backwards compatibility, the plan
is to change those packages so the corresponding calls try to import
perfact.zodbsync and use the functionality there, falling back to the
previous implementation if that fails. Corresponding deprecation warnings will
External Methods to be present in the ZODB even if the
corresponding modules are no longer present as extensions. It does not,
however, allow saving such an object. This gives errors if object trees
External Methods are recorded and played back. The same
Z SQL Methods which have
class_file set to a no
longer existing extension.
To Do / Roadmap
At some point, it might become necessary to easily allow playback to read from a different path, to avoid changes that are collected to be played back being overwritten by a running
zoperecordin watcher mode.
These tools can serve as a basis for replacing and improving ZSyncer functionality, offering diffs between development, testing and production systems that include metadata and do not depend on timestamps as well as the possibility to pull objects from a different system.