BugFix for multiple responses of different type adding bulk attributes

[kirzaks]

Main problem is when I would like insert multiple attributes and one of attributes have error of "A similar attribute already exists for this event". MISP is sending back error message where KeyError occurs in IF sections

ipdb> new_attribute                                                                                                                                                                                                
{'Attribute': {'id': '472', 'event_id': '63', 'object_id': '0', 'object_relation': None, 'category': 'Network activity', 'type': 'hostname', 'value1': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbb.com', 'value2': '', 'to_ids': True, 'uuid': '5d3b198a-02d0-4b50-8d28-16f5c0a8383e', 'timestamp': '1564154250', 'distribution': '5', 'sharing_group_id': '0', 'comment': '', 'deleted': False, 'disable_correlation': False, 'value': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbb.com'}, 'errors': {'attribute_0': {'value': ['A similar attribute already exists for this event.']}}}
ipdb> new_attribute["errors"]                                                                                                                                                                                      
{'attribute_0': {'value': ['A similar attribute already exists for this event.']}}
ipdb> type(new_attribute["errors"])                                                                                                                                                                                
<class 'dict'>
ipdb> new_attribute['errors'][0]                                                                                                                                                                                   
*** KeyError: 0

[coveralls]

Coverage decreased (-0.05%) to 39.438% when pulling 2d057b9 on kirzaks:master into 549e3a5 on MISP:master.

[Rafiot]

Wow, wait. value1 and value2 should be discarded by MISP, that's not a valid way to push multiple attributes.

[Rafiot]

Sorry, that's something else.

Which version of MISP are you using?

[kirzaks]

Hi, MISP version: Currently installed version… v2.4.112 (3bfb2a28e13d75f115e23cf1ff2b185082bb0e95) PyMISP version: __version__ = '2.4.111.2' At this moment - func add_attribute in aping.py is allowing add multiple attributes. (POC_1) I'm adding two domains [OK] domain aaaaaaaaaaaaaaaaaaaaaaaaaaaaa.com [OK] domain bbbbbbbbbbbbbbbbbbbbbbbbbbbbb.com Response: [{'Attribute': {'id': '497', 'event_id': '70', 'object_id': '0', 'object_relation': None, 'category': 'Network activity', 'type': 'hostname', 'value1': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaa.com', 'value2': '', 'to_ids': True, 'uuid': '5d493c9d-3914-4787-9f7a-337ac0a8383e', 'timestamp': '1565080733', 'distribution': '5', 'sharing_group_id': '0', 'comment': '{new_comm}', 'deleted': False, 'disable_correlation': False, 'value': 'aaaaaaaaaaaaaaaaaaaaaaaaaaaaa.com'}, 'AttributeTag': []}, {'Attribute': {'id': '498', 'event_id': '70', 'object_id': '0', 'object_relation': None, 'category': 'Network activity', 'type': 'hostname', 'value1': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbb.com', 'value2': '', 'to_ids': True, 'uuid': '5d493c9d-5ce0-4936-8632-337ac0a8383e', 'timestamp': '1565080733', 'distribution': '5', 'sharing_group_id': '0', 'comment': '{new_comm}', 'deleted': False, 'disable_correlation': False, 'value': 'bbbbbbbbbbbbbbbbbbbbbbbbbbbbb.com'}, 'AttributeTag': []}] And there is the issue: If I would like to send multiple attributes where one is already added (POC_2) [OK] domain cccccccccccccccccccccccccccccccc.com [OK] domain bbbbbbbbbbbbbbbbbbbbbbbbbbbbb.com I'm getting error in code where domain bbbbbbbbbbbbbbbbbbbbbbbbbbbbb.com is giving me error "A similar attribute already exists for this event." what is not suitable whit next IF blocks if ('errors' in new_attribute and new_attribute['errors'][0] == 403

…

On Tue, Aug 6, 2019 at 11:08 AM Raphaël Vinot ***@***.***> wrote: Wow, wait. value1 and value2 should be discarded by MISP, that's not a valid way to push multiple attributes. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#433?email_source=notifications&email_token=AEHGGMMCWMXNTVCRDFD75YTQDEWRXA5CNFSM4IJMHXHKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3UJOUA#issuecomment-518559568>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEHGGMMOLM6R7JB62ZFP5PLQDEWRXANCNFSM4IJMHXHA> .

-- Armīns Palms

[Rafiot]

Alright, makes sense, I can reproduce. Bugfix incoming.

[Rafiot]

FYI, there is a bug on MISP side so we're fixing it over there too, I cannot promise you 100% that it will be out today.

[Rafiot]

Sooo, I'll have a fix on PyMISP side that avoid the exception and directly return the response from MISP.

Important point tho: the actual output returned by MISP will change starting in 2.4.113 (the current one is broken, see MISP/MISP#4959)

[kirzaks]

Thanks!

[Rafiot]

You're welcome, let me know if you have any problem.