persistent_doc - An XML-like document with spreadsheet formulas for values and underlying persistent data structures
Clone this repository and then
pip install -r requirements.txt
>>> from persistent_doc.document import Document, FrozenNode as Node, Ex
>>> root = Node("foo", params={"id": "one"})
>>> doc = Document(root)
>>> doc['one'].append(Node("bar", params={"id": "two"}))
>>> doc['one.0']
bar(id=two)[]
>>> doc['one.x'] = 3
>>> doc['one']['x']
3
>>> doc['two.x'] = Ex("`one.x + 3")
>>> doc['two.x']
6
>>> doc['one.x'] += 1
>>> doc['two.x']
7
Examples of relative paths, longer reference chain and undo-redo
>>> doc['two.y'] = Ex("`two.parent.x + `two.x")
>>> doc['two.y']
11
>>> doc['one.x'] += 1
>>> doc['two.y']
13
>>> doc.undo()
>>> doc.undo()
>>> doc.undo()
>>> doc.undo()
>>> doc['one.x'], doc['two.y']
(4, 11)
>>> doc.start_group("one_step")
True
>>> doc['one.x'] += 1
>>> doc.end_group("one_step")
>>> doc['one.x'], doc['two.y']
(5, 13)
>>> doc.undo()
>>> doc['one.x'], doc['two.y']
(4, 11)
>>> doc.redo()
>>> doc['one.x'], doc['two.y']
(5, 13)
Like XML, the document is a tree where each node has a list of children and key-value pairs for parameters. Like SVG, each node has a unique id
property within the document.
Parameter values can be "formulas" treating other nodes' id as variables (and a relative path of parent, children and parameter keys).
The document is persistent to allow easy undo-redo with spending too much memory. Persistent means that mutations to an object always creates a new version but the new version shares memory with the old version. (It could potentially allow erasing history to save more on memory.)
While the underlying data structures are immutable but have the same interface as mutable objects.
Syntax may change in the future. To set a formula as value, create an Ex object with a string (in this syntax) to be evaluated.
- node id:
`foo
evaluates to the node with idfoo
in the document. - parameter keys:
`foo.bar
evaluates to (the value of) parameterbar
of nodefoo
in the document.`foo.bar.baz
works as expected iffoo.bar
is a node. - children and parents:
`foo[1]
and`foo.1
both evaluates to the child of (the node with id)foo
at index 1 (i.e., the second child).`foo.1.0
isfoo
's second child's first child.`foo.parent
is the parent offoo
. - function calls and operations: Function calls
foo(`bar, 3 + `baz)
work as in Python.
If doc
is a persistent_doc.document.Document
, doc['foo.parent.3.2']
gives foo
's parent's fourth child's second child. Function calls and operations cannot be used with Document.__getitem__
(instead use them outside, in Python f(doc['foo.p1'], doc['bar.p1'])
).
There are three reevaluation strategies
- cached (default): The formula is reevaluated when a term it depends on changes. The result is cached for reads.
- reeval: The formula is reevaluated every time it is read.
- on first read: The formula is reevaluated the first time it is read after one of the terms it depends on has changed. The result is cached for reads.
The calc
parameter is passed to Ex
to indicate which one to use. For example Ex(`foo + 3, calc="reeval")
. It is thus possible to have a document with mixed reevaluation strategies.
To get the object for a formula instead of its value, use doc.get_expr('foo.bar')
instead of doc['foo.bar']
.
Expressions that produce an error when evaluated will return an EvalError
object instead of raising an error.
Run python test.py
.
Because the document is persistent, pointers to values in the document can become stale. Use doc.m
or node.L
(instead of doc
and node
) to use the latest version.
Because parent-child relations are encoded as lookups, its not possible to create a new subtree of Nodes "in the void" and then hook it up to existing nodes. It needs at least a doc (memory) to be created. So default_doc
helps with that part.
If there's only one document, persistent_doc.document.default_doc
should be set to that.
Parent sets the child's .parent
.
numpy
isn't really a requirement but since numpy
1.13, equality test behave differently and some of the polymorphism doesn't work otherwise. document.py
contains an alternate definition of equal
that can be used if there are no numpy arrays as values.
Mutable values are expected not to be mutated. They should instead be replaced.
doc['foo.arr'] = numpy.array([1, 2])
doc['foo.arr'] = doc['foo.arr'] + numpy.array([1, 1])
instead of
arr = numpy.array([1, 2])
doc['foo.arr'] = arr
arr += numpy.array([1, 1])
- Find something better than all the explicit type checking with
Ex
andExpr
. - Find a way to automatically decide if a node needs to be replaced by its latest version before an operation. Maybe by comparing timestamps?