Sometimes you want to fetch data from a deeply nested data structure. Dotted notation helps you do that.
Let's say you have a dictionary containing a dictionary containing a list and you wish to fetch the ith value from that nested list.
>>> import dotted
>>> d = {'hi': {'there': [1, 2, 3]}}
>>> dotted.get(d, 'hi.there[1]')
2
Probably the easiest thing to do is pydoc the api layer.
$ pydoc dotted.api
See grammar discussion below about things you can do get data via dotted.
>>> import dotted
>>> dotted.get({'a': {'b': {'c': {'d': 'nested'}}}}, 'a.b.c.d')
'nested'
Update will mutate the object if it can. It always returns the changed object though. If it's not mutable, then get via the return.
>>> import dotted
>>> l = []
>>> t = ()
>>> dotted.update(l, '[0]', 'hello')
['hello']
>>> l
['hello']
>>> dotted.update(t, '[0]', 'hello')
('hello',)
>>> t
()
```
You can update all fields that match pattern given by either a wildcard OR regex.
>>> import dotted
>>> d = {'a': 'hello', 'b': {'bye'}}
>>> dotted.update(d, '*', 'me')
{'a': 'me', 'b': 'me'}
You can remove a field or do so only if it matches value. For example,
>>> import dotted
>>> d = {'a': 'hello', 'b': 'bye'}
>>> dotted.remove(d, 'b')
{'a': 'hello'}
>>> dotted.remove(d, 'a', 'bye')
{'a': 'hello'}
Similar to update, all patterns that match will be removed. If you provide a value as well, only the matched patterns that also match the value will be removed.
Use to match a dotted-style pattern to a field. Partial matching is on by default. You can match via wildcard OR via regex. Here's a regex example:
>>> import dotted
>>> dotted.match('/a.+/', 'abced.b')
'abced.b'
>>> dotted.match('/a.+/', 'abced.b', partial=False)
With the groups=True parameter, you'll see how it was matched:
>>> import dotted
>>> match('hello.*', 'hello.there.bye', groups=True)
('hello.there.bye', ('hello', 'there.bye'))
In the above example, hello matched to hello and * matched to there.bye (partial
matching is enabled by default).
You may wish to expand all fields that match a pattern in an object.
>>> import dotted
>>> d = {'hello': {'there': [1, 2, 3]}, 'bye': 7}
>>> dotted.expand(d, '*')
('hello', 'bye')
>>> dotted.expand(d, '*.*')
('hello.there',)
>>> dptted.expand(d, '*.*[*]')
('hello.there[0]', 'hello.there[1]', 'hello.there[2]')
>>> dotted.expand(d, '*.*[1:]')
('hello.there[1:]',)
Dotted notation shares similarities with python. A dot . field expects to see a
dictionary-like object (using keys and __getitem__ internally. A bracket []
field is biased towards sequences (like lists or strs) but can also act on dicts. A
attr @ field uses getattr/setattr/delattr. Dotted also support slicing notation
as well as transforms discussed below.
A key field is expressed as a or part of a dotted expression, such as a.b. The
grammar parser is permissive for what can be in a key field. Pretty much any non-reserved
char will match. Note that key fields will only work on objects that have a keys
method. Basically, they work with dictionary or dictionary-like objects.
>>> import dotted
>>> dotted.get({'a': {'b': 'hello'}}, 'a.b')
'hello'
If the key field starts with a space or -, you should either quote it OR you may use
a \ as the first char.
You may also use bracket notation, such as a[0] which does a __getitem__ at key 0.
The parser prefers numeric types over string types (if you wish to look up a non-numeric
field using brackets be sure to quote it). Bracketed fields will work with pretty much
any object that can be looked up via __getitem__.
>>> import dotted
>>> dotted.get({'a': ['first', 'second', 'third']}, 'a[0]')
'first'
>>> dotted.get({'a': {'b': 'hello'}}, 'a["b"]')
'first'
An attr field is expressed by prefixing with @. This will fetch data at that attribute.
You may wonder why have this when you can just as easily use standard python to access.
Two important reasons: nested expressions and patterns.
>>> import dotted, types
>>> ns = types.SimpleNamespace
>>> ns.hello = {'me': 'goodbye'}
>>> dotted.get(ns, '@hello.me')
'goodbye'
The parser will attempt to interpret a field numerically if it can, such as field.1
will interpret the 1 part numerically.
>>> import dotted
>>> dotted.get({'7': 'me', 7: 'you'}, '7')
'you'
Sometimes you need to quote a field which you can do by just putting the field in quotes.
>>> import dotted
>>> dotted.get({'has . in it': 7}, '"has . in it"')
7
Non-integer numeric fields may be interpreted incorrectly if they have decimal point. To
solve, use the numerize operator # at the front of a quoted field, such as #'123.45'.
This will coerce to a numeric type (e.g. float).
>>> import dotted
>>> d = {'a': {1.2: 'hello', 1: {2: 'fooled you'}}}
>>> dotted.get(d, 'a.1.2')
'fooled you'
>>> dotted.get(d, 'a.#"1.2"')
'hello'
Dotted slicing works like python slicing and all that entails.
>>> import dotted
>>> d = {'hi': {'there': [1, 2, 3]}, 'bye': {'there': [4, 5, 6]}}
>>> dotted.get(d, 'hi.there[::2]')
[1, 3]
>>> dotted.get(d, '*.there[1:]')
([2, 3], [5, 6])
Both bracketed fileds and slices support the '+' operator which refers to the end of sequence. You may append an item or slice to the end a sequence.
>>> import dotted
>>> d = {'hi': {'there': [1, 2, 3]}, 'bye': {'there': [4, 5, 6]}}
>>> dotted.update(d, '*.there[+]', 8)
{'hi': {'there': [1, 2, 3, 8]}, 'bye': {'there': [4, 5, 6, 8]}}
>>> dotted.update(d, '*.there[+:]', [999])
{'hi': {'there': [1, 2, 3, 8, 999]}, 'bye': {'there': [4, 5, 6, 8, 999]}}
If you want to update only unique items to a list, you can use the ?
postfix. This will ensure that it's only added once (see match-first below).
>>> import dotted
>>> items = [1, 2]
>>> dotted.update(items, '[+?]', 3)
[1, 2, 3]
>>> dotted.update(items, '[+?]', 3)
[1, 2, 3]
You can invert the meaning of the notation by prefixing a -. For example,
to remove an item using update:
>>> import dotted
>>> d = {'a': 'hello', 'b': 'bye'}
>>> dotted.update(d, '-b', dotted.ANY)
{'a': 'hello'}
>>> dotted.remove(d, '-b', 'bye again')
{'a': 'hello', 'b': 'bye again'}
You may use dotted for pattern matching. You can match to wildcards or regular expressions. You'll note that patterns always return a tuple of matches.
>>> import dotted
>>> d = {'hi': {'there': [1, 2, 3]}, 'bye': {'there': [4, 5, 6]}}
>>> dotted.get(d, '*.there[2]')
(3, 6)
>>> dotted.get(d, '/h.*/.*')
([1, 2, 3],)
Dotted will return all values that match the pattern(s).
The wildcard pattern is *. It will match anything.
The regex pattern is enclosed in slashes: /regex/. Note that if the field is a non-str,
the regex pattern will internally match to its str representation.
You can also postfix any pattern with a ?. This will return only
the first match.
>>> import dotted
>>> d = {'hi': {'there': [1, 2, 3]}, 'bye': {'there': [4, 5, 6]}}
>>> dotted.get(d, '*?.there[2]')
(3,)
You can optionally add transforms to the end of dotted notation. These will
be applied on get and update. Transforms are separated by the | operator
and multiple may be chained together. Transforms may be parameterized using
the : operator.
>>> import dotted
>>> d = [1, '2', 3]
>>> dotted.get(d, '[1]')
'2'
>>> dotted.get(d, '[1]|int')
2
>>> dotted.get(d, '[0]|str:number=%d')
'number=1'
You may register new transforms via either register or the @transform
decorator. Look at transforms.py for preregistered.
You may filter by key-value to narrow your result set. You may use with key or
bracketed fields. Key-value fields may be disjunctively (OR) specified via the ,
delimiter.
A key-value field on key field looks like: keyfield.key1=value1,key2=value2....
This will return all key-value matches on a subordinate dict-like object. For example,
>>> d = {
... 'a': {
... 'id': 1,
... 'hello': 'there',
... },
... 'b': {
... 'id': 2,
... 'hello': 'there',
... },
... }
>>> dotted.get(d, '*.id=1')
({'id': 1, 'hello': 'there'},)
>>> dotted.get(d, '*.id=*')
({'id': 1, 'hello': 'there'}, {'id': 2, 'hello': 'there'})
A key-value field on a bracketed field looks like: [key1=value1,key2=value2...].
This will return all items in a list that match key-value filter. For example,
>>> d = {
... 'a': [{'id': 1, 'hello': 'there'}, {'id': 2, 'hello': 'there'}],
... 'b': [{'id': 3, 'hello': 'there'}, {'id': 4, 'hello': 'bye'}],
... }
>>> dotted.get(d, 'a[hello="there"][*].id')
(1, 2)
>>> dotted.get(d, '*[hello="there"][*].id')
r == (1, 2, 3)
You can have it match first by appending a ? to the end of the filter.
>>> d = {
... 'a': [{'id': 1, 'hello': 'there'}, {'id': 2, 'hello': 'there'}],
... 'b': [{'id': 3, 'hello': 'there'}, {'id': 4, 'hello': 'bye'}],
... }
>>> dotted.get(d, 'a[hello="there"?]')
return [{'id': 1, 'hello': 'there'}]
To conjunctively connect filters use the . operator. Filters offer the ability to act
disjunctively as well by using the , operator.
For example, given
*.key1=value1,key2=value2.key3=value3. This will filter
(key1=value1 OR key2=value2) AND key3=value3.
Note that this gives you the abilty to have a key filter multiple values, such as:
*.key1=value1,key2=value2.