dictdiffer: pragmatic merge

[mvesper]

As discussed with @jirikuncar a while ago, here the pull request of a more pragmatic dictdiffer.merge implementation.

Dictdiffer v0.5.0: merging

• Alters the dictdiffer.diff function to allow expanding patches and to utilize a path_limit parameter, which stops the extraction process at a given point
• Adds the following files to handle the merging process:
  • merge.py
  • conflict.py
  • resolve.py
  • unify.py

Which is executed in the following steps:

• The differences between a latest common ancestor (lca) and two other data structures are extracted
• The two occuring differences lists are checked for conflicts
• Those conficts are resolved
• The patches are unified

Signed-off-by: Martin Vesper martin.vesper@cern.ch

[coveralls]

Coverage remained the same at 100.0% when pulling 08db2ee on mvesper:m/wip/pragmatic_merge into 1210935 on inveniosoftware:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 08db2ee on mvesper:m/wip/pragmatic_merge into 1210935 on inveniosoftware:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 08db2ee on mvesper:m/wip/pragmatic_merge into 1210935 on inveniosoftware:master.

[jirikuncar]

string_types?

[coveralls]

Coverage remained the same at 100.0% when pulling 062dfaa on mvesper:m/wip/pragmatic_merge into 1210935 on inveniosoftware:master.

[jirikuncar]

can you re-format the docstring?

    """
    Path Limit:

        >>> list(...)
        [...]
    """"

PS: check how does it look like if you compile them locally.

[coveralls]

Coverage remained the same at 100.0% when pulling 2e721f2 on mvesper:m/wip/pragmatic_merge into 1210935 on inveniosoftware:master.

[jirikuncar]

• s/addtion/addition/ see 8a8d5bd
• please re-format the bullet points (one empty line between, fill blocks, etc.)

[coveralls]

Coverage remained the same at 100.0% when pulling 3a2affb on mvesper:m/wip/pragmatic_merge into 1210935 on inveniosoftware:master.

[coveralls]

Coverage remained the same at 100.0% when pulling 3a2affb on mvesper:m/wip/pragmatic_merge into 1210935 on inveniosoftware:master.

[jirikuncar]

@mvesper I've run some simple test on two random files from CDS.

Data

$ wget http://cds.cern.ch/record/2014647?of=recjson -O a.json
$ wget http://cds.cern.ch/record/2014586?of=recjson -O b.json

Code

import json, dictdiffer
a, b = json.loads(open('a.json').read()), json.loads(open('b.json').read())
%timeit d = list(dictdiffer.diff(a, b))

Results master

1000 loops, best of 3: 1.09 ms per loop

%prun d = list(dictdiffer.diff(a, b))
         2330 function calls (1967 primitive calls) in 0.004 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   427/64    0.002    0.000    0.004    0.000 __init__.py:32(diff)
      917    0.001    0.000    0.001    0.000 {isinstance}
      134    0.000    0.000    0.001    0.000 {map}
      431    0.000    0.000    0.001    0.000 __init__.py:53(<lambda>)
      110    0.000    0.000    0.000    0.000 {method 'encode' of 'unicode' objects}
      110    0.000    0.000    0.001    0.000 __init__.py:62(check)
      134    0.000    0.000    0.000    0.000 {all}
        1    0.000    0.000    0.004    0.004 <string>:1(<module>)
       24    0.000    0.000    0.000    0.000 {range}
       24    0.000    0.000    0.000    0.000 {min}
       16    0.000    0.000    0.000    0.000 {len}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}

Results #53

1000 loops, best of 3: 1.15 ms per loop

%prun d = list(dictdiffer.diff(a, b))
         2330 function calls (1967 primitive calls) in 0.004 seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   427/64    0.002    0.000    0.004    0.000 __init__.py:26(diff)
      917    0.000    0.000    0.000    0.000 {isinstance}
      134    0.000    0.000    0.001    0.000 {map}
      431    0.000    0.000    0.001    0.000 __init__.py:83(<lambda>)
      110    0.000    0.000    0.000    0.000 __init__.py:92(check)
      110    0.000    0.000    0.000    0.000 {method 'encode' of 'unicode' objects}
      134    0.000    0.000    0.000    0.000 {all}
        1    0.000    0.000    0.004    0.004 <string>:1(<module>)
       24    0.000    0.000    0.000    0.000 {range}
       24    0.000    0.000    0.000    0.000 {min}
       16    0.000    0.000    0.000    0.000 {len}
        1    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

Conclusion

No major performance regression spotted.

[jirikuncar]

@mvesper

• add yourself to AUTHORS file
• for new files use only current year in copyright header

[coveralls]

Coverage remained the same at 100.0% when pulling eb5dca0 on mvesper:m/wip/pragmatic_merge into 1210935 on inveniosoftware:master.