Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DeepDiff raises decimal exception when using significant digits #426

Closed
yawhide opened this issue Oct 17, 2023 · 3 comments
Closed

DeepDiff raises decimal exception when using significant digits #426

yawhide opened this issue Oct 17, 2023 · 3 comments

Comments

@yawhide
Copy link

yawhide commented Oct 17, 2023

Describe the bug
Comparing certain decimals with the significant_digits argument leads to an invalidDecimal exception from being raised within deep diff

To Reproduce

from decimal import Decimal

from deepdiff import DeepDiff


diff = DeepDiff([Decimal("999.99999999")], [Decimal("999.9999999999")], significant_digits=6)
# exception will raise

Expected behavior
There should be no exception and no values_changed

OS, DeepDiff version and Python version (please complete the following information):

  • OS: mac os
  • Version: 13.6
  • Python Version: 3.11.4
  • DeepDiff Version: 6.6.0

Additional context
Stack trace:

Traceback (most recent call last):
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/testdeepdiff.py", line 6, in <module>
    diff = DeepDiff([Decimal("999.99999999")], [Decimal("999.9999999999")], significant_digits=6)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 311, in __init__
    self._diff(root, parents_ids=frozenset({id(t1)}), _original_type=_original_type)
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 1569, in _diff
    self._diff_iterable(level, parents_ids, _original_type=_original_type, local_tree=local_tree)
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 655, in _diff_iterable
    self._diff_iterable_in_order(level, parents_ids, _original_type=_original_type, local_tree=local_tree)
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 761, in _diff_iterable_in_order
    self._diff_ordered_iterable_by_difflib(
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 902, in _diff_ordered_iterable_by_difflib
    self._diff_by_forming_pairs_and_comparing_one_by_one(
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 887, in _diff_by_forming_pairs_and_comparing_one_by_one
    self._diff(next_level, parents_ids_added, local_tree=local_tree)
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 1551, in _diff
    self._diff_numbers(level, local_tree=local_tree, report_type_change=report_type_change)
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/diff.py", line 1360, in _diff_numbers
    t1_s = self.number_to_string(level.t1,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/stefan/code/coin-tracker/coin-tracker-server/.venv/lib/python3.11/site-packages/deepdiff/helper.py", line 397, in number_to_string
    number = number.quantize(Decimal('0.' + '0' * significant_digits))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]

I dont quite understand why your code doesnt work. When i try to reproduce outside of your library, i dont have issues:

from decimal import Decimal
n = Decimal('999.999999999')
n.quantize(Decimal('0.' + '0' * 6))
# => Decimal('1000.000000')
@seperman
Copy link
Owner

Hi @yawhide
It is because the count of the significant digits of the rounded value for 999.9999999 is one digit above precision of the context.

>>> with localcontext() as ctx:
...     ctx.prec = 9
...     Decimal('999.99999999').quantize(Decimal('0.000000'))
...
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
>>>
>>> with localcontext() as ctx:
...     ctx.prec = 10
...     Decimal('999.99999999').quantize(Decimal('0.000000'))
...
Decimal('1000.000000')

@seperman seperman mentioned this issue Oct 17, 2023
@yawhide
Copy link
Author

yawhide commented Oct 18, 2023

oh that makes sense. thanks for the quick fix!

@seperman
Copy link
Owner

Sure!

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Jan 28, 2024
v6-7-1

-  Support for subtracting delta objects when iterable_compare_func
   is used.
-  Better handling of force adding a delta to an object.
-  Fix for
   ```Can't compare dicts with both single and double quotes in keys`` <https://github.com/seperman/deepdiff/issues/430>`__
-  Updated docs for Inconsistent Behavior with math_epsilon and
   ignore_order = True

v6-7-0

-  Delta can be subtracted from other objects now.
-  verify_symmetry is deprecated. Use bidirectional instead.
-  always_include_values flag in Delta can be enabled to include
   values in the delta for every change.
-  Fix for Delta.\__add\_\_ breaks with esoteric dict keys.

v6-6-1

 -  Fix for `DeepDiff raises decimal exception when using significant
    digits <https://github.com/seperman/deepdiff/issues/426>`__
 -  Introducing group_by_sort_key
 -  Adding group_by 2D. For example
    ``group_by=['last_name', 'zip_code']``

v6-6-0

-  Numpy 2.0 support
-  Adding
   `Delta.to_flat_dicts <https://zepworks.com/deepdiff/current/serialization.html#delta-serialize-to-flat-dictionaries>`__

v6-5-0

-  Adding
   ```parse_path`` <https://github.com/seperman/deepdiff/pull/419>`__

v6-4-1

-  Bugfix: Keep Numpy Optional
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants