Feature Request: provide a merge event function in PyMISP

[github-germ]

There are times that we have a need to merge many events into one. One case is related to MISP/MISP#5544 where we needed to change default feeds from daily events to a fixed event, and we need to merge all the prior daily events into the one new fixed event. (This need to merge is triggered by duplicate attributes spanning separate events causing huge and false correlation growth)

There is now a method to merge two events in the WebUI (see MISP/MISP#5544).

Would be great if PyMISP.merge could specify a list of existing events and a target event to merge them all into.

[mokaddem]

Indeed, that's a good idea to remediate feed misconfiguration and could be a nice contribution.
As a first step, what you could do is to provide a wrapper around the Merge event functionality already present in MISP. The url to post the merge data to is /events/merge/[event_id]
Feel free to ask if you need more help!

[Rafiot]

The merging strategy is highly dependent on the business logic, so it needs to be implemented by the end user (you). The merge method in the UI is not recommended.

From the top of my head, you need to be able to answer at least these questions in order to do a successful merge:

• which event do we use as master (and keep the metadata, tags, ...)
• what is a duplicate attribute? (value only, comment and value, timestamp, how we handle first seen/last seen, sightings ...)
• do we create a new event out of the merged events? Or use one of them?
• how do we handle synchronization?

And that's just for events that contain only attributes. I let you think on how you would do that for events with objects.

All that to say that where will be no such feature in the coming future in the default PyMISP package, but if you write your own script, feel free to share it in the comments here, it may help someone in your situation.

[github-germ]

I understand your points and appreciate your reply!

How are some of the factors handled in the existing merge that exists in the WebUI and server-side?

[Rafiot]

Badly :)

afaik, it doesn't support objects for example, so you could give it a try in your case, but it is very much not recommended if you really cannot lose data.

[github-germ]

Understood. Thx. We will have to see if/how we will merge the Events in our use cases. Your input has been helpful.

[Rafiot]

You're very welcome!