Updating Existing Event, NewAttributeError "The info field of the new event is required."

[faustus25]

Searching for events belonging to an org and if the the tag is present then proceeds to tag the event only for 1 event. It fails with the following error message:

5bf16598-a1b4-4e5f-a979-e67dc0a8b280
Traceback (most recent call last):
File "cuckootag.py", line 50, in
misp.change_analysis_status({'Event':event}, analysis_status=2)
File "/usr/local/lib/python3.5/dist-packages/pymisp-2.4.99-py3.5.egg/pymisp/api.py", line 492, in change_analysis_status
e = self._make_mispevent(event)
File "/usr/local/lib/python3.5/dist-packages/pymisp-2.4.99-py3.5.egg/pymisp/api.py", line 281, in _make_mispevent
e.load(event)
File "/usr/local/lib/python3.5/dist-packages/pymisp-2.4.99-py3.5.egg/pymisp/mispevent.py", line 467, in load
self.from_dict(**event)
File "/usr/local/lib/python3.5/dist-packages/pymisp-2.4.99-py3.5.egg/pymisp/mispevent.py", line 491, in from_dict
raise NewAttributeError('The info field of the new event is required.')
pymisp.exceptions.NewAttributeError: The info field of the new event is required.

Code below:

result = misp.search_index(org="16")

for event in result['response']:
  if misp.search_index(tag=None):
    print(event['uuid'])
    misp.tag(event['uuid'], "Cuckoo")
    misp.tag(event['uuid'], "tlp:green")
    misp.change_threat_level({'Event':event}, threat_level_id=2)
    misp.change_analysis_status({'Event':event}, analysis_status=2)
    misp.change_distribution({'Event':event}, distribution=2)
    misp.publish({'Event':event})
    print("Event tagged: %s"%event)
    misp.pushEventToZMQ(event['uuid'])

Was there a recent change that requires passing the "eventinfo" in modifying existing events?

What is the best method to pass the existing "eventinfo" in order to publish a modified event/s?

[tomking2]

I've got the same issue

Take the following code block as an example:

pm = PyMISP("url", "apikey")
event = pm.get(1234)
# Printing the event will give you the fill event, with attributes and all)
print(event)
pm.add_domain(event, "exampledomain.com")
# Printing the event now contains considerably less data. For example, following keys aren't present:
# event['Event']['id']
# event['Event']['info']
print(event)

I'm under the impression that whatever is adding an attribute and updating is causing the event to get refreshed, and many of the fields are stripped or not returned

It's definitely a PyMISP issue - Old version of PyMISP works as expected on the same MISP

[tomking2]

Looks like it may be an issue with how the misp_event.py is popping all the records from the MISP event, such as:

PyMISP/pymisp/mispevent.py

Lines 489 to 491 in d4daa2f

	self.info = kwargs.pop('info', None)
	if self.info is None:
	raise NewEventError('The info field of the new event is required.')

What's happening is that if you pass the entire event to pm.add_domain for example, it's popping it off. Meaning that you can add at least one attribute, but anything prior to this won't have the necessary fields

@Rafiot - What is the recommended approach for adding/removing attributes from an event? Should we be passing the ID instead of the entire event, i.e. pm.add_domain(event['Event']['id'], "exampledomain.com")

[tomking2]

Also, what's interesting is that it's raising a NewEventError exception. As we are modifying events, may we need a different function to translate from JSON?

[tomking2]

This is the line which causes this to fire when adding attributes in my case

PyMISP/pymisp/mispevent.py

Line 467 in d4daa2f

self.from_dict(**event)

  File "<path>/pymisp/api.py", line 763, in add_domain
    return self.add_named_attribute(event, 'domain', domain, category, to_ids, comment, distribution, proposal, **kwargs)
  File "<path>/pymisp/api.py", line 612, in add_named_attribute
    return self._send_attributes(event, attributes, proposal)
  File "<path>/pymisp/api.py", line 547, in _send_attributes
    event_id = self._extract_event_id(event)
  File "<path>/pymisp/api.py", line 600, in _extract_event_id
    e.load(event)
  File "<path>/pymisp/mispevent.py", line 467, in load
    self.from_dict(**event)
  File "<path>/pymisp/mispevent.py", line 491, in from_dict
    raise NewEventError('The info field of the new event is required.')

[faustus25]

Same issue we have, the updated branch of pymisp seems to consider this modification as a new event so event_info is a required field to be passed now for existing events.

It worked on the older branch of pymisp but newer updates of MISP (and PyMISP) have impacted this.

Also, there is no specific method available to update an event's eventinfo which ideally needs a new method "change_eventinfo" but in my example I don't want the eventinfo to change.

[Rafiot]

Sorry for the late answer.

Okay, that was definitely a weird one to figure out. Basically, calling add_<attribute> with an event dictionary as parameter was loading said dictionary in a MISPEvent object, That load was modifying the event variable, so next time you try to pass it as parameter, it wasn't a valid event anymore, and failing.

If you use HEAD, it should work as expected.

[faustus25]

This is still not working for me when I update to HEAD or checkout that commit;

With that commit: (was on PyMISP v2.99)

cat .git/HEAD
4c60ed7

Traceback (most recent call last):
  File "cuckootag.py", line 42, in <module>
    misp.change_analysis_status({'Event':event},analysis_status=2)
  File "/home/user/python3-misp-virtual-environments/env/lib/python3.5/site-packages/pymisp/api.py", line 492, in change_analysis_status
    e = self._make_mispevent(event)
  File "/home/user/python3-misp-virtual-environments/env/lib/python3.5/site-packages/pymisp/api.py", line 281, in _make_mispevent
    e.load(event)
  File "/home/user/python3-misp-virtual-environments/env/lib/python3.5/site-packages/pymisp/mispevent.py", line 470, in load
    self.from_dict(**event)
  File "/home/user/python3-misp-virtual-environments/env/lib/python3.5/site-packages/pymisp/mispevent.py", line 496, in from_dict
    raise NewEventError('The info field of the new event is required.')
pymisp.exceptions.NewEventError: The info field of the new event is required.

Same error with latest HEAD: (PyMISP v2.102)

cat .git/HEAD
ref: refs/heads/master

git describe
v2.4.102-16-g5afdcb5

[Rafiot]

Where does cuckootag.py comes from? Can I see the code so I figure out what it does?

The sample code here works.

Oh, I see. It's failing further down in the code. Also, this piece of code uses a very old way making lots of calls to MISP and should be updated. If you point me to the project, I can probably fix it.

[Rafiot]

That should work now. Nevertheless, the code is extremely inefficient and should really be fixed.

[faustus25]

That should work now. Nevertheless, the code is extremely inefficient and should really be fixed.

The fix unfortunately didn't work @Rafiot

Traceback (most recent call last):
  File "cuckootag.py", line 50, in <module>
    misp.change_analysis_status({'Event':event}, analysis_status=2)
  File "/usr/local/lib/python3.5/dist-packages/pymisp-2.4.102-py3.5.egg/pymisp/api.py", line 493, in change_analysis_status
    e = self._make_mispevent(event)
  File "/usr/local/lib/python3.5/dist-packages/pymisp-2.4.102-py3.5.egg/pymisp/api.py", line 282, in _make_mispevent
    e.load(copy.copy(event))
  File "/usr/local/lib/python3.5/dist-packages/pymisp-2.4.102-py3.5.egg/pymisp/mispevent.py", line 468, in load
    self.from_dict(**event)
  File "/usr/local/lib/python3.5/dist-packages/pymisp-2.4.102-py3.5.egg/pymisp/mispevent.py", line 494, in from_dict
    raise NewEventError('The info field of the new event is required.')
pymisp.exceptions.NewEventError: The info field of the new event is required.

The script is still the same as the original post and yes could be refined. Hangs on the second edit "change_analysis_status") to the existing event(s):

    misp.change_threat_level({'Event':event}, threat_level_id=2)
    _misp.change_analysis_status({'Event':event}, analysis_status=2)_
    misp.change_distribution({'Event':event}, distribution=2)

Updated PyMISP

git describe
v2.4.102-17-gbd74a11

and in pip3

pip3 list | grep "pymisp"
pymisp (2.4.102)

Appreciate some guidance on how to improve the script and get it working again.

[Rafiot]

That would make everyone's life a lot easier if you could share the whole file, so I don't keep blindly patching things.

[Rafiot]

replace

    misp.tag(event['uuid'], "Cuckoo")
    misp.tag(event['uuid'], "tlp:green")
    misp.change_threat_level({'Event':event}, threat_level_id=2)
    misp.change_analysis_status({'Event':event}, analysis_status=2)
    misp.change_distribution({'Event':event}, distribution=2)
    misp.publish({'Event':event})
    print("Event tagged: %s"%event)
    misp.pushEventToZMQ(event['uuid'])

by:

me = MISPEvent()
me.load(event) 
me.add_tag('Cuckoo')
me.add_tag('tlp:green')
me.threat_level_id = 2 
me.analysis = 2    
me.distribution = 2
me.publish()     
event = misp.update(me)
print("Event tagged: %s"%event['Event']['info'])
misp.pushEventToZMQ(event['Event']['id'])

And it should (?) work. I don't even understand how it could have worked before, because the following call is invalid (it expects an ID)...

    misp.pushEventToZMQ(event['uuid'])

[faustus25]

Great thanks for the help @Rafiot, updated the code to that and it works now.

Was only importing PyMISP library previously for the old code, once MISPEvent library was imported it worked along with the updated code.

[Rafiot]

Cool, glad it works :)