Dictionaries Should Be Equal should support ignoring keys

[bbpatel2001]

Problem:
Current implementation of "Dictionaries Should Be Equal" returns true if two dictionaries are identical. Sometimes we need to compare two dictionaries by ignoring some keys. i.e. If we have large dictionaries and one dictionary does not have some keys at nth level but still keyword should return True for all other keys if matches . Take example of following two dictionaries. expected_dict is my expected dictionary and Actual dictionary is created from UI data. Actual dictionary does not contain "Forged Source" key. If key is 1st or 2nd level we can pop key and compare it but if we want to ignore any key which is at the nth level it is very difficult.

expected_dict = {
  "alert_main_list": {
  "test":"test1",
  "IP": {
      "1.1.1.16": {
        "#_of_CLUSTERS": "1",
        "#_of_FINGERPRINTS": "1",
        "IP_ADDRESS": "1.1.1.1",
        "HIGHEST_CONFIDENCE": "49",
        "EVENT_COUNT": "1",
        "abcd" : ['I','Love','Robot', 'Framework'],
        "numbers" : [ 1, 2, 3, 4, 5,"aa"]
      }
    },
  },
  "User": {
      "user@test.com": {
        "ALERT_TRIGGERS": {
          "Suspicious Behavior": {
            "Username is an email address": "1"
          },
          "Suspicious Source": {
            "Forum Spam Blacklist": "1"
          },
          "Forged Source": {
            "Web Browser : Watson Lookup": "1"
          }
        },        
      }
    },
}
Actual_dict = {
  "alert_main_list": {
  "test":"test1",
  "IP": {
      "1.1.1.16": {
        "#_of_CLUSTERS": "1",
        "#_of_FINGERPRINTS": "1",
        "IP_ADDRESS": "1.1.1.1",
        "HIGHEST_CONFIDENCE": "49",
        "EVENT_COUNT": "1",
        "abcd" : ['I','Love','Robot', 'Framework'],
        "numbers" : [ 1, 2, 3, 4, 5,"aa"]
      }
    },
  },
  "User": {
      "user@test.com": {
        "ALERT_TRIGGERS": {
          "Suspicious Behavior": {
            "Username is an email address": "1"
          },
          "Suspicious Source": {
            "Forum Spam Blacklist": "1"
          },
        },        
      }
    },
}

I am proposing
new keyword "Dictionaries Should Ignore And Equal"
that will take three parameters: (I am OK to change name of keyword if accepted this proposal. )

1st parameter : Actual dictionary [Type: Dict]
2nd parameter : Expected dictionary [Type: Dict]
3rd Parameter : Ignore list [Type: List] : List of keys to be ignored

This keyword will not match key which is listed in third parameter
Dictionaries Should Ignore And Equal | d1 | d2 | ['Forged Source'] This will return True

I will add few more example if required.

[bbpatel2001]

Other problem is if we have list value for any specific key and we want to compare it with same key in other dictionary, both list should be identical. Order should be exactly same. Current keyword does not return true for UN-ordered list.

In above example we have two keys with values in list.

"abcd" : ['I','Love','Robot', 'Framework'],
"numbers" : [ 1, 2, 3, 4, 5,"aa"]

it will not match if we have following two keys in actual dictionary

"abcd" : ['Love','I','Robot', 'Framework'],
"numbers" : [ 5, 3, 4, 2, 5, 1, "aa"]

My above proposed keyword will take care this as well.

[pekkaklarck]

Ignoring keys sounds useful, but I think it's better to enhance the existing Dictionaries Should Be Equal to support it instead of adding a new keyword. I guess it could get an optional argument like ignore=None or ignore_keys=None that could be used to specify a list of keys to ignore.

I can see that ignoring order of internal lists can be useful too, but I'm not sure is it an often enough needed feature to make sense to add it. Similarly we could argue that support for ignoring duplicates in an internal list or making some internal validation case-insensitive could be useful, but adding keywords or optional arguments to handle all such cases is not possible. Collections library should support getting an internal list from a dictionary and having other keywords then for validating it in un-ordered manner. In other words, we could enhance list related keywords in Collections to support it.

Notice also that if you have complex data structures, it's often easiest to implement custom keywords for handling them.

[pekkaklarck]

We actually already have #2703 about ignoring list order.

[bbpatel2001]

Developed new keyword that compare two dictionaries. For time being, I have kept keyword name as Dictionaries Should Be Equal New Will be changed as per review comments. Also I have compared original v/s new keyword output. New keyword gives exact location where key is missing or value is not matching whereas original keyword gives long

Also if you provide optional list of keys as parameter which will be ignored while comparing dictionaries. It will not check key and key value of ignored keys and keyword will pass if remaining keys are matching.
Please compare result of two keywords.
Test name ends with New Keyword : Used new keyword Dictionaries Should Be Equal New
Test name ends with Original Keword : Used Original keyword Dictionaries Should Be Equal

Example                                                                       
==============================================================================
T1 Dictionaries Should Be Equal New                                   | PASS |
------------------------------------------------------------------------------
T2 Missing HIGHEST_CONFIDENCE In First Dict New Keyword               | FAIL |
Mistmatch found in second dictionary: 
following keys missing alert_main_list.IP.1.1.1.16.HIGHEST_CONFIDENCE
------------------------------------------------------------------------------
T2 Missing HIGHEST_CONFIDENCE In First Dict Original Keword           | FAIL |
Following keys have different values:
Key alert_main_list: {'test': 'test1', 'IP': {'1.1.1.16': {'abcd': ['I', 'Love', 'Robot', 'Framework'], '#_of_FINGERPRINTS': '1', 'EVENT_COUNT': '1', '#_of_CLUSTERS': '1', 'HIGHEST_CONFIDENCE': '49', 'IP_ADDRESS': '1.1.1.1', 'numbers': [1, 2, 3, 4, 5, 'aa']}}, 'User': {'user@test.com': {'ALERT_TRIGGERS': {'Suspicious Behavior': {'Username is an email address': '1'}, 'Suspicious Source': {'Forum Spam Blacklist': '1'}, 'Forged Source': {'Web Browser : Watson Lookup': '1'}}}}} != {'test': 'test1', 'IP': {'1.1.1.16': {'abcd': ['I', 'Love', 'Robot', 'Framework'], '#_of_FINGERPRINTS': '1', 'EVENT_COUNT': '1', '#_of_CLUSTERS': '1', 'numbers': [1, 2, 3, 4, 5, 'aa'], 'IP_ADDRESS': '1.1.1.1'}}, 'User': {'user@test.com': {'ALERT_TRIGGERS': {'Suspicious Behavior': {'Username is an email address': '1'}, 'Suspicious Source': {'Forum Spam Blacklist': '1'}, 'Forged Source': {'Web Browser : Watson Lookup': '1'}}}}}
------------------------------------------------------------------------------
T3 Missing HIGHEST_CONFIDENCE In Second Dict New Keyword              | FAIL |
Mistmatch found in first dictionary: 
following keys missing alert_main_list.IP.1.1.1.16.HIGHEST_CONFIDENCE
------------------------------------------------------------------------------
T3 Missing HIGHEST_CONFIDENCE In Second Dict Original Keword          | FAIL |
Following keys have different values:
Key alert_main_list: {'test': 'test1', 'IP': {'1.1.1.16': {'abcd': ['I', 'Love', 'Robot', 'Framework'], '#_of_FINGERPRINTS': '1', 'EVENT_COUNT': '1', '#_of_CLUSTERS': '1', 'numbers': [1, 2, 3, 4, 5, 'aa'], 'IP_ADDRESS': '1.1.1.1'}}, 'User': {'user@test.com': {'ALERT_TRIGGERS': {'Suspicious Behavior': {'Username is an email address': '1'}, 'Suspicious Source': {'Forum Spam Blacklist': '1'}, 'Forged Source': {'Web Browser : Watson Lookup': '1'}}}}} != {'test': 'test1', 'IP': {'1.1.1.16': {'abcd': ['I', 'Love', 'Robot', 'Framework'], '#_of_FINGERPRINTS': '1', 'EVENT_COUNT': '1', '#_of_CLUSTERS': '1', 'HIGHEST_CONFIDENCE': '49', 'IP_ADDRESS': '1.1.1.1', 'numbers': [1, 2, 3, 4, 5, 'aa']}}, 'User': {'user@test.com': {'ALERT_TRIGGERS': {'Suspicious Behavior': {'Username is an email address': '1'}, 'Suspicious Source': {'Forum Spam Blacklist': '1'}, 'Forged Source': {'Web Browser : Watson Lookup': '1'}}}}}
------------------------------------------------------------------------------
T4 Few Key Missing and Values Changed New Keyword                     | FAIL |
Mistmatch found in first dictionary: 
for key alert_main_list.IP.1.1.1.16.HIGHEST_CONFIDENCE value 59 is not matching: 59 != 49, 
for key alert_main_list.User.user@test.com.ALERT_TRIGGERS.Suspicious Behavior.Username is an email address value 10 is not matching: 10 != 1, 
following keys missing alert_main_list.IP.1.1.1.16.#_of_CLUSTERS , 
for key alert_main_list.IP.1.1.1.16.IP_ADDRESS value 1.1.1.1 is not matching: 1.1.1.1 != 1.1.2.2

Mistmatch found in second dictionary: 
for key alert_main_list.IP.1.1.1.16.HIGHEST_CONFIDENCE value 49 is not matching: 49 != 59, 
for key alert_main_list.User.user@test.com.ALERT_TRIGGERS.Suspicious Behavior.Username is an email address value 1 is not matching: 1 != 10, 
for key alert_main_list.IP.1.1.1.16.IP_ADDRESS value 1.1.2.2 is not matching: 1.1.2.2 != 1.1.1.1, 
following keys missing alert_main_list.User.user@test.com.ALERT_TRIGGERS.Forged Source.Web Browser : Watson Lookup , 
following keys missing alert_main_list.IP.1.1.1.16.EVENT_COUNT
------------------------------------------------------------------------------
T4 Few Key Missing and Values Changed Original Keword                 | FAIL |
Following keys have different values:
Key alert_main_list: {'test': 'test1', 'IP': {'1.1.1.16': {'abcd': ['I', 'Love', 'Robot', 'Framework'], '#_of_FINGERPRINTS': '1', 'EVENT_COUNT': '1', 'HIGHEST_CONFIDENCE': '49', 'IP_ADDRESS': '1.1.2.2', 'numbers': [1, 2, 3, 4, 5, 'aa']}}, 'User': {'user@test.com': {'ALERT_TRIGGERS': {'Suspicious Behavior': {'Username is an email address': '1'}, 'Suspicious Source': {'Forum Spam Blacklist': '1'}, 'Forged Source': {'Web Browser : Watson Lookup': '1'}}}}} != {'test': 'test1', 'IP': {'1.1.1.16': {'abcd': ['I', 'Love', 'Robot', 'Framework'], '#_of_FINGERPRINTS': '1', '#_of_CLUSTERS': '1', 'HIGHEST_CONFIDENCE': '59', 'IP_ADDRESS': '1.1.1.1', 'numbers': [1, 2, 3, 4, 5, 'aa']}}, 'User': {'user@test.com': {'ALERT_TRIGGERS': {'Suspicious Behavior': {'Username is an email address': '10'}, 'Suspicious Source': {'Forum Spam Blacklist': '1'}, 'Forged Source': {}}}}}
------------------------------------------------------------------------------
T5 Few Key Missing And Values Changed And Ignore One Key New Keyword  | FAIL |
Mistmatch found in first dictionary: 
for key alert_main_list.User.user@test.com.ALERT_TRIGGERS.Suspicious Behavior.Username is an email address value 10 is not matching: 10 != 1, 
following keys missing alert_main_list.IP.1.1.1.16.#_of_CLUSTERS , 
for key alert_main_list.IP.1.1.1.16.IP_ADDRESS value 1.1.1.1 is not matching: 1.1.1.1 != 1.1.2.2

Mistmatch found in second dictionary: 
for key alert_main_list.User.user@test.com.ALERT_TRIGGERS.Suspicious Behavior.Username is an email address value 1 is not matching: 1 != 10, 
for key alert_main_list.IP.1.1.1.16.IP_ADDRESS value 1.1.2.2 is not matching: 1.1.2.2 != 1.1.1.1, 
following keys missing alert_main_list.User.user@test.com.ALERT_TRIGGERS.Forged Source.Web Browser : Watson Lookup , 
following keys missing alert_main_list.IP.1.1.1.16.EVENT_COUNT
------------------------------------------------------------------------------
T5 Few Key Missing And Values Changed And Ignore One Key Original ... .No keyword to ignore key
T5 Few Key Missing And Values Changed And Ignore One Key Original ... | PASS |
------------------------------------------------------------------------------
T6 List Mismatched At Inner Level New Keyword                         | FAIL |
Mistmatch found in first dictionary: 
for key alert_main_list.IP.1.1.1.16.HIGHEST_CONFIDENCE value 59 is not matching: 59 != 49, 
for key alert_main_list.User.user@test.com.ALERT_TRIGGERS.Suspicious Behavior.Username is an email address value 10 is not matching: 10 != 1, 
following keys missing alert_main_list.IP.1.1.1.16.#_of_CLUSTERS , 
for key alert_main_list.IP.1.1.1.16.IP_ADDRESS value 1.1.1.1 is not matching: 1.1.1.1 != 1.1.2.2, 
for key alert_main_list.IP.1.1.1.16.numbers value [1, 2, 3, 4, 5, 'aa'] is not matching: [1, 2, 3, 4, 5, 'aa'] != [1, 2, 4, 5, 'aa']

Mistmatch found in second dictionary: 
for key alert_main_list.IP.1.1.1.16.HIGHEST_CONFIDENCE value 49 is not matching: 49 != 59, 
for key alert_main_list.User.user@test.com.ALERT_TRIGGERS.Suspicious Behavior.Username is an email address value 1 is not matching: 1 != 10, 
for key alert_main_list.IP.1.1.1.16.IP_ADDRESS value 1.1.2.2 is not matching: 1.1.2.2 != 1.1.1.1, 
for key alert_main_list.IP.1.1.1.16.numbers value [1, 2, 4, 5, 'aa'] is not matching: [1, 2, 4, 5, 'aa'] != [1, 2, 3, 4, 5, 'aa'], 
following keys missing alert_main_list.User.user@test.com.ALERT_TRIGGERS.Forged Source.Web Browser : Watson Lookup , 
following keys missing alert_main_list.IP.1.1.1.16.EVENT_COUNT
------------------------------------------------------------------------------
T6 List Mismatched At Inner Level New Keyword Original Keword         | FAIL |
Following keys have different values:
Key alert_main_list: {'test': 'test1', 'IP': {'1.1.1.16': {'abcd': ['I', 'Love', 'Robot', 'Framework'], '#_of_FINGERPRINTS': '1', 'EVENT_COUNT': '1', 'HIGHEST_CONFIDENCE': '49', 'IP_ADDRESS': '1.1.2.2', 'numbers': [1, 2, 4, 5, 'aa']}}, 'User': {'user@test.com': {'ALERT_TRIGGERS': {'Suspicious Behavior': {'Username is an email address': '1'}, 'Suspicious Source': {'Forum Spam Blacklist': '1'}, 'Forged Source': {'Web Browser : Watson Lookup': '1'}}}}} != {'test': 'test1', 'IP': {'1.1.1.16': {'abcd': ['I', 'Love', 'Robot', 'Framework'], '#_of_FINGERPRINTS': '1', '#_of_CLUSTERS': '1', 'HIGHEST_CONFIDENCE': '59', 'IP_ADDRESS': '1.1.1.1', 'numbers': [1, 2, 3, 4, 5, 'aa']}}, 'User': {'user@test.com': {'ALERT_TRIGGERS': {'Suspicious Behavior': {'Username is an email address': '10'}, 'Suspicious Source': {'Forum Spam Blacklist': '1'}, 'Forged Source': {}}}}}
------------------------------------------------------------------------------
Example                                                               | FAIL |
11 critical tests, 2 passed, 9 failed
11 tests total, 2 passed, 9 failed

Please check the output of tests [in following archive file ] which are starting with TC

TestLog_report_output.zip

[kormbrek-rally]

FYI we use jsoncompare.are_same for this.
jsoncompare

[pekkaklarck]

Are you @bbpatel2001 still interested in this one? I took a quick look at the PR and it seemed pretty complicated.

[sandeepvaidya]

I suppose the keyword, 'Dictionary Should Contain Sub Dictionary' can be used in some situations.

[rainydew]

this library written by me may help
recursion json(bytes str/unicode str/dict/list) compare
supports ignore path, regular expression compare, list ignore order and fuzzy float equal
https://pypi.org/project/jsoncomparedeep/

[yuriverweij]

Hi, can I try taking up this issue?

[pekkaklarck]

Thanks @yuriverweij for a great PR! I had missed it earlier but merged it now, just in time for RF 6.1. There are few smallish things, mentioned in the PR review, that I'll still fix.