Errors with dictionary_item_removed and dictionary_item_added

[tpcalhoun]

Please checkout the F.A.Q page before creating a bug ticket to make sure it is not already addressed.

Describe the bug
When using a dictionary and a Delta containing either dictionary_item_removed or dictionary_item_added to generate a resulting dictionary, will cause the following error:
TypeError: 'list' object is not callable

To Reproduce
You can run this sample code to demonstrate both error conditions:

import pytest
import json
import copy
from deepdiff import DeepDiff, Delta

"""
Sample code (Unit Test) to demonstrate DeepDiff errors when trying
to generate a result dict from a starting dict + Delta
    
    execute:
    pytest -vv demo_deepdiff_fail.py
"""


def test_demonstrate_fail_dictionary_item_removed():
    """
    Demonstrate fail with:
    {'dictionary_item_removed': [root['individualCustomer']['employment']]}
    """
    dict1 = {"individualCustomer": {
        "gender": "Male",
        "citizenship": {
            "country": "United States"
        },
        "employment": {
            "employerName": "ABC LTD",
            "jobTitle": "Bookkeeper",
            "status": "Full-time"
        }
    }}

    # Make a copy of dict1
    dict2 = copy.deepcopy(dict1)
    # Remove the inner object employment from dict2 only.
    dict2['individualCustomer'].pop('employment')

    # find the difference between the two dictionaries
    # putting dict1 FIRST...
    updates = DeepDiff(dict1, dict2)
    updates_dict = updates.to_dict()

    # now using a Delta and dict1, try to generate dict2
    delta = Delta(updates_dict)
    generated_dict = dict1 + delta

    # ############################ #
    # Errors on statement above with:
    # E  TypeError: 'list' object is not callable
    # C:\DEV\Tools\Python\Python38\lib\site-packages\deepdiff\delta.py:466: TypeError
    # ############################ #

    assert dict2 == generated_dict

def test_demonstrate_fail_dictionary_item_added():
    """
    Demonstrate fail with:
    {'dictionary_item_added': [root['individualCustomer']['employment']]}
    """
    dict1 = {"individualCustomer": {
        "gender": "Male",
        "citizenship": {
            "country": "United States"
        },
        "employment": {
            "employerName": "ABC LTD",
            "jobTitle": "Bookkeeper",
            "status": "Full-time"
        }
    }}

    # Make a copy of dict1
    dict2 = copy.deepcopy(dict1)
    # Remove the inner object employment from dict2 only.
    dict2['individualCustomer'].pop('employment')

    # find the difference between the two dictionaries
    # putting dict2 FIRST...
    updates = DeepDiff(dict2, dict1)
    updates_dict = updates.to_dict()

    # now using a Delta and dict2, try to generate dict1
    delta = Delta(updates_dict)
    generated_dict = dict2 + delta

    # ############################ #
    # Errors on statement above with:
    # E   TypeError: 'list' object is not callable
    # C:\DEV\Tools\Python\Python38\lib\site-packages\deepdiff\delta.py:357: TypeError
    # ############################ #

    assert dict1 == generated_dict

Expected behavior
A clear and concise description of what you expected to happen.

OS, DeepDiff version and Python version (please complete the following information):

• OS: [e.g. Ubuntu] Windows10
• Version [e.g. 20LTS]
• Python Version [e.g. 3.9.12] 3.8
• DeepDiff Version [e.g. 5.8.0] 6.7.0

Additional context
Add any other context about the problem here.

[seperman]

Hi @tpcalhoun
The reason you are running into this issue is that you are passing updates.to_dict() to Delta instead of the diff object itself.
You can either serialize the DeepDiff object, then recreate the DeepDiff object and pass it to Delta to recreate Delta, or you can create the Delta object and serialize Delta itself and restore Delta from the serialized Delta.

        diff = DeepDiff(dict1, dict2)
        delta = Delta(diff)

Serialization options are documented here: https://zepworks.com/deepdiff/current/serialization.html

[tpcalhoun]

From: Sep
Sent: Tuesday, December 12, 2023 1:45 PM
To: Todd
Subject: RE: [EXTERNAL] Re: DeepDiff errors

If you want to have them pass the updates as a flat list of dictionaries, you can avoid DeepDiff and go straight to Delta:
https://zepworks.com/deepdiff/current/serialization.html#delta-load-from-flat-dictionaries

---- On Tue, 12 Dec 2023 11:34:08 -0800 Todd wrote ---

Thank you for the quick reply.
Ok, I have not tried your suggestion yet but…..

My upstream team is going to send a single JSON like:

event = {
“beforeImage”: {“key”: “value”},
“updates”: {“key”: “value”}
}

So I would need to generate my “afterImage”.

Sort of like this:
afterimage = event.get(‘beforeImage’) + Delta(event.get(‘updates’))

Is there a way for me to convert my “updates” JSON (python dictionary) to a “diff object” to pass into my Delta? In short, I will not be using the DeepDiff class normally, just the Delta class.

Please reply at your convenience.
Todd

---- On Tue, 12 Dec 2023 12:57 PM Sep wrote ---
From: Sep
Sent: Tuesday, December 12, 2023 12:57 PM
To: Todd
Subject: [EXTERNAL] Re: DeepDiff errors
Importance: High

Hi Todd,
Closed your ticket on Github. Please read my comments.
Thanks,
Sep

[tpcalhoun]

THANK YOU SEP. Your suggestion worked for me, see corrected code examples below:

def test_demonstrate_corrected_dictionary_item_removed():
    """
    Demonstrate fail with:
    {'dictionary_item_removed': [root['individualCustomer']['employment']]}
    """
    dict1 = {"individualCustomer": {
        "gender": "Male",
        "citizenship": {
            "country": "United States"
        },
        "employment": {
            "employerName": "ABC LTD",
            "jobTitle": "Bookkeeper",
            "status": "Full-time"
        }
    }}

    # Make a copy of dict1
    dict2 = copy.deepcopy(dict1)
    # Remove the inner object employment from dict2 only.
    dict2['individualCustomer'].pop('employment')

    # find the difference between the two dictionaries
    # putting dict1 FIRST...
    diff = DeepDiff(dict1, dict2)
    delta = Delta(diff)
    flat_dict_list = delta.to_flat_dicts()

    # Note this flat_dict_list is what we need to pass as event['updates']
    updates = flat_dict_list

    # ----------- Simulate processing event received ---------------
    event = {"beforeImage": dict1, "updates": updates}

    # now using a Delta and beforeImage, try to generate afterImage
    delta = Delta(flat_dict_list=event.get('updates'))
    after_image = event.get('beforeImage') + delta
    # --------------------------------------------------------------

    final_diff = DeepDiff(dict2, after_image)
    assert final_diff == {}
    assert dict2 == after_image


def test_demonstrate_corrected_dictionary_item_added():
    """
    Demonstrate fail with:
    {'dictionary_item_added': [root['individualCustomer']['employment']]}
    """
    dict1 = {"individualCustomer": {
        "gender": "Male",
        "citizenship": {
            "country": "United States"
        },
        "employment": {
            "employerName": "ABC LTD",
            "jobTitle": "Bookkeeper",
            "status": "Full-time"
        }
    }}

    # Make a copy of dict1
    dict2 = copy.deepcopy(dict1)
    # Remove the inner object employment from dict2 only.
    dict1['individualCustomer'].pop('employment')

    # find the difference between the two dictionaries
    # putting dict2 FIRST...
    diff = DeepDiff(dict1, dict2)
    delta = Delta(diff)
    flat_dict_list = delta.to_flat_dicts()

    # Note this flat_dict_list is what we need to pass as event['updates']
    updates = flat_dict_list

    # ----------- Simulate processing event received ---------------
    event = {"beforeImage": dict1, "updates": updates}

    # now using a Delta and beforeImage, try to generate afterImage
    delta = Delta(flat_dict_list=event.get('updates'))
    after_image = event.get('beforeImage') + delta
    # --------------------------------------------------------------

    final_diff = DeepDiff(dict2, after_image)
    assert final_diff == {}
    assert dict2 == after_image