Two features: allowing custom operators and dynaimc ignoring order #252

eggachecat · 2021-06-16T14:24:23Z

Add ignore_order_func to make ignore-order operation dynamic with level

I think it is an useful feature since it happens a lot when some of the fields concerns about order while some does not.

Allow custom operators to do/report some custom operations

Allow user to use custom operators to record some info with/without early stopping.
User just need to implement a class which contans two method:
- match(level: DiffLevel) -> boolean to determine whether current operator should be applied on current level
- diff(level: DiffLevel, instance: DeepDiff) -> boolean
  - to do custom diff logic with full access to DeepDiff instance
    - you can use instance.custom_report_result to record info
  - to return a boolean value to determine whether the process should quit or continue with default behavior

5.5.0

Minor cleanup to documentation wording regarding iterable_compare_func

Add ignore_order_func to make ignore-order operation dynamic with level

allow custom operators to do/report some custom operations allow

eggachecat · 2021-06-16T14:26:45Z

hi @seperman would you like to do a CR for there two features?~

seperman

Hi @eggachecat
Thanks for this PR. These are great new features! 👍
I had a couple of minor comments. Please take a look at my comments when you have a chance.

Also do you mind adding the documentation for the 2 features if you have time? It will be more authentic if the docs are written by you. If I write the docs, it will be my own interpretation of what these features mean. Looking at the code seems like you actually had a use case for these features.

The ignore_order_func docs can go to ignore_order.rst and the custom_operators docs can go to other.rst.

seperman · 2021-06-21T14:31:04Z

deepdiff/diff.py

+            # compatibility
+            if "ignore_order_func" not in _parameters:
+                _parameters["ignore_order_func"] = lambda *_args, **_kwargs: _parameters["ignore_order_func"]
+
+            if "custom_operators" not in _parameters:
+                _parameters["custom_operators"] = []
+


You don't need these for the compatibility. The _parameters is a self.__dict__ copy that is passed. If you remove these lines, the only test that fails is one delta test since it has _parameters hard-coded. Pleased update that test instead: https://github.com/seperman/deepdiff/blob/master/tests/test_delta.py#L1055

seperman · 2021-06-21T14:32:32Z

deepdiff/diff.py

            self.ignore_order = ignore_order
+
+            if ignore_order_func is not None:
+                self.ignore_order_func = ignore_order_func


You could use the same simple line like the one for the custom_operators: self.ignore_order_func = ignore_order_func or lambda *_args, **_kwargs: ignore_order

seperman · 2021-06-21T14:33:34Z

deepdiff/diff.py

+
+        if not self._skip_this(level):
+            level.report_type = report_type
+            level.additional[CUSTOM_FILED] = extra_info


Typo: CUSTOM_FIELD

seperman · 2021-06-21T14:38:18Z

docs/diff_doc.rst

 iterable_compare_func:
    :ref:`iterable_compare_func_label`:
-    There are times that we want to guide DeepDiff as to what items to compare with other items. In such cases we can pass a iterable_compare_func that takes a function pointer to compare two items. It function takes two parameters and should return True if it is a match, False if it is not a match or raise CannotCompare if it is unable to compare the two.
+    There are times that we want to guide DeepDiff as to what items to compare with other items. In such cases we can pass a iterable_compare_func that takes a function pointer to compare two items. The function takes three parameters (x, y, level) and should return True if it is a match, False if it is not a match or raise CannotCompare if it is unable to compare the two.


Thanks for fixing this. I thought I had found all the instances of this incorrect sentence already but obviously not!

seperman · 2021-06-21T14:43:27Z

tests/test_operators.py

@@ -0,0 +1,141 @@
+import math


Thanks for adding the thorough tests!

seperman · 2021-06-21T14:53:16Z

deepdiff/model.py

                path = change.path(force=FORCE_DEFAULT)
                self['repetition_change'][path] = RemapDict(change.additional[
-                    'repetition'])
+                                                                'repetition'])


This line is giving pep8 error E126.

eggachecat · 2021-06-24T02:20:31Z

okay! I will make fix these soon

eggachecat · 2021-06-24T03:36:54Z

@seperman just updated the PR~
Would you like to do a CR ~

codecov-commenter · 2021-07-02T22:02:46Z

Codecov Report

Merging #252 (7e778fd) into dev (4ae6170) will decrease coverage by 0.01%.
The diff coverage is 99.39%.

@@            Coverage Diff             @@
##              dev     #252      +/-   ##
==========================================
- Coverage   99.63%   99.62%   -0.02%     
==========================================
  Files          13       14       +1     
  Lines        2768     2900     +132     
==========================================
+ Hits         2758     2889     +131     
- Misses         10       11       +1

Impacted Files	Coverage Δ
deepdiff/operator.py	`90.90% <90.90%> (ø)`
deepdiff/delta.py	`100.00% <100.00%> (ø)`
deepdiff/diff.py	`100.00% <100.00%> (ø)`
deepdiff/distance.py	`100.00% <100.00%> (ø)`
deepdiff/helper.py	`100.00% <100.00%> (ø)`
deepdiff/model.py	`100.00% <100.00%> (ø)`
deepdiff/search.py	`100.00% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4ae6170...7e778fd. Read the comment docs.

seperman · 2021-07-02T22:10:35Z

Hi @eggachecat
Thanks again for the great PR. I will keep you posted once the new DeepDiff with your changes is released.

seperman · 2021-10-13T06:49:50Z

Hi @eggachecat
DeepDiff 5.6.0 is released and it includes your PR.
Thank you for your great contribution!

seperman and others added 7 commits April 28, 2021 23:20

Merge pull request seperman#245 from seperman/dev

bc1ab6a

5.5.0

bye bye readthedocs

1dd0aec

Minor cleanup to documentation wording regarding iterable_compare_func

772220c

Merge pull request seperman#247 from surefyresystems/master

d82d4fc

Minor cleanup to documentation wording regarding iterable_compare_func

add ignore_order_func

cb5b480

Add ignore_order_func to make ignore-order operation dynamic with level

allow custom operators

e55efa0

allow custom operators to do/report some custom operations allow

merge two features

b0fc606

eggachecat changed the title ~~Two features~~ Two features: allowing custom operators and dynaimc ignoring order Jun 16, 2021

eggachecat added 2 commits June 17, 2021 14:53

give custom operators more flexibiliy

ae66cab

fix assertion

e439470

seperman changed the base branch from master to dev June 21, 2021 12:10

seperman reviewed Jun 21, 2021

View reviewed changes

eggachecat added 2 commits June 24, 2021 10:43

fix some code-issue

d49c4ce

docs: add docs for custom operators and ignore_order_func

7e778fd

seperman merged commit a127ebd into seperman:dev Jul 2, 2021

Uh oh!

Two features: allowing custom operators and dynaimc ignoring order #252

Two features: allowing custom operators and dynaimc ignoring order #252

Uh oh!

Conversation

eggachecat commented Jun 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eggachecat commented Jun 16, 2021

Uh oh!

seperman left a comment

Choose a reason for hiding this comment

Uh oh!

seperman Jun 21, 2021

Choose a reason for hiding this comment

Uh oh!

seperman Jun 21, 2021

Choose a reason for hiding this comment

Uh oh!

seperman Jun 21, 2021

Choose a reason for hiding this comment

Uh oh!

seperman Jun 21, 2021

Choose a reason for hiding this comment

Uh oh!

seperman Jun 21, 2021

Choose a reason for hiding this comment

Uh oh!

seperman Jun 21, 2021

Choose a reason for hiding this comment

Uh oh!

eggachecat commented Jun 24, 2021

Uh oh!

eggachecat commented Jun 24, 2021

Uh oh!

codecov-commenter commented Jul 2, 2021

Codecov Report

Uh oh!

seperman commented Jul 2, 2021

Uh oh!

seperman commented Oct 13, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

eggachecat commented Jun 16, 2021 •

edited

Loading