Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding custom object to event #462

Closed
thomasb454 opened this issue Sep 25, 2019 · 19 comments
Closed

Adding custom object to event #462

thomasb454 opened this issue Sep 25, 2019 · 19 comments
Assignees
Labels
T: support Type: support. This issue is seeking support on a problem or question

Comments

@thomasb454
Copy link

thomasb454 commented Sep 25, 2019

Hi,

I'm having issues when trying to update an event with a custom object.

PyMISP version: 2.4.114
Python version: 3.7

The object template is present on the web server (I can manually add the object to an event), however when using ExpandedPyMISP it returns a 403.
ERROR [aping.py:2039 - _check_response() ] Something went wrong (403): {'name': 'Could not add object', 'message': 'Could not add object', 'url': '/objects/add/9/', 'errors': 'No valid template found to edit the object.'}

from pymisp import ExpandedPyMISP
from pymisp import MISPObject

misp = ExpandedPyMISP('<url>', '<key>', False)

event = misp.search(uuid='5d89ea9a-0778-4a90-a4d0-05a7ac1002c7')

obj = MISPObject(name='mopr-report', misp_objects_path_custom='mopr')

obj.add_attribute('score', value=0.02)
obj.add_attribute('malicious', value=False)
obj.add_attribute('benign', value=True)

misp.add_object('5d89ea9a-0778-4a90-a4d0-05a7ac1002c7', misp_object=obj)

Any advice is appreciated.

@Rafiot
Copy link
Member

Rafiot commented Sep 25, 2019

The problem is that MISP

  1. doesn't know the object template
  2. tries to update an existing event on the platform

when the two conditions are true, the error is expected.

You can either add the template on MISP, or create a new object (by changing the UUID).

@Rafiot Rafiot added the T: support Type: support. This issue is seeking support on a problem or question label Sep 25, 2019
@thomasb454
Copy link
Author

thomasb454 commented Sep 25, 2019

Hi,

The object template is present on MISP? Like I said, I can add the object to an event using the web app.
Additionally, you can see I point the MISPObject to the custom template locally (so it can load all the related metadata and perform validiation)

@Rafiot
Copy link
Member

Rafiot commented Sep 25, 2019

Yes, you can add it, but you should not be able to update it from the web interface (?)

If you can, I'm confused and will ask @iglocska to the rescue.

@thomasb454
Copy link
Author

thomasb454 commented Sep 25, 2019

Hi,

I can add the object to an event and edit it (on the web app).

Okay haha - thank you.

@thomasb454
Copy link
Author

Hi,

I've found a less than ideal workaround. Hopefully the above issue is fixed because these methods are deprecated.

Instead of using the new ExpandedPyMISP API, revert to PyMISP API and provide the template UUID when adding the object to the event.

@kovacsbalu
Copy link
Contributor

kovacsbalu commented Sep 25, 2019

Hi @thomasb454, I tried to reproduce you issue. There is no error for me.
My MISP version is 2.4.109 pymisp 2.4.114
In your error msg there is an 'url': '/objects/add/9/'. Object index 9 is you custom object id?
Did you try debug mode?

@thomasb454
Copy link
Author

Hi @kovacsbalu,

9 is the ID of the event that I'm trying to add the object to.
Debug output produces the following:

DEBUG [aping.py:2073 - _prepare_request() ] GET - http://<url>/servers/getPyMISPVersion.json
DEBUG [aping.py:2075 - _prepare_request() ] {}
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.7', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'm3uBBfGH94cDSSsDvZv7Doqpihgsjg1Zc3WPYs5r', 'content-type': 'application/json'}
DEBUG [aping.py:2047 - _check_response() ] {'version': '2.4.99'}
INFO [aping.py:79 - __init__() ] The version of PyMISP recommended by the MISP instance (response['version']) is older than the one you're using now (2.4.114). If you have a problem, please upgrade the MISP instance or use an older PyMISP version.
DEBUG [aping.py:2073 - _prepare_request() ] GET - http://<url>/servers/getVersion.json
DEBUG [aping.py:2075 - _prepare_request() ] {}
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.7', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'm3uBBfGH94cDSSsDvZv7Doqpihgsjg1Zc3WPYs5r', 'content-type': 'application/json'}
DEBUG [aping.py:2047 - _check_response() ] {'version': '2.4.100', 'perm_sync': True}
DEBUG [aping.py:2073 - _prepare_request() ] GET - http://<url>/attributes/describeTypes.json
DEBUG [aping.py:2075 - _prepare_request() ] {}
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.7', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'm3uBBfGH94cDSSsDvZv7Doqpihgsjg1Zc3WPYs5r', 'content-type': 'application/json'}
DEBUG [aping.py:2047 - _check_response() ] {'result': {'sane_defaults': {'md5': {'default_category': 'Payload delivery', 'to_ids': 1}, 'sha1': {'default_category': 'Payload delivery', 'to_ids': 1}, 'sha256': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename': {'default_category': 'Payload delivery', 'to_ids': 1}, 'pdb': {'default_category': 'Artifacts dropped', 'to_ids': 0}, 'filename|md5': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|sha1': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|sha256': {'default_category': 'Payload delivery', 'to_ids': 1}, 'ip-src': {'default_category': 'Network activity', 'to_ids': 1}, 'ip-dst': {'default_category': 'Network activity', 'to_ids': 1}, 'hostname': {'default_category': 'Network activity', 'to_ids': 1}, 'domain': {'default_category': 'Network activity', 'to_ids': 1}, 'domain|ip': {'default_category': 'Network activity', 'to_ids': 1}, 'email-src': {'default_category': 'Payload delivery', 'to_ids': 1}, 'email-dst': {'default_category': 'Network activity', 'to_ids': 1}, 'email-subject': {'default_category': 'Payload delivery', 'to_ids': 0}, 'email-attachment': {'default_category': 'Payload delivery', 'to_ids': 1}, 'email-body': {'default_category': 'Payload delivery', 'to_ids': 0}, 'float': {'default_category': 'Other', 'to_ids': 0}, 'url': {'default_category': 'Network activity', 'to_ids': 1}, 'http-method': {'default_category': 'Network activity', 'to_ids': 0}, 'user-agent': {'default_category': 'Network activity', 'to_ids': 0}, 'ja3-fingerprint-md5': {'default_category': 'Network activity', 'to_ids': 1}, 'regkey': {'default_category': 'Persistence mechanism', 'to_ids': 1}, 'regkey|value': {'default_category': 'Persistence mechanism', 'to_ids': 1}, 'AS': {'default_category': 'Network activity', 'to_ids': 0}, 'snort': {'default_category': 'Network activity', 'to_ids': 1}, 'bro': {'default_category': 'Network activity', 'to_ids': 1}, 'pattern-in-file': {'default_category': 'Payload installation', 'to_ids': 1}, 'pattern-in-traffic': {'default_category': 'Network activity', 'to_ids': 1}, 'pattern-in-memory': {'default_category': 'Payload installation', 'to_ids': 1}, 'yara': {'default_category': 'Payload installation', 'to_ids': 1}, 'stix2-pattern': {'default_category': 'Payload installation', 'to_ids': 1}, 'sigma': {'default_category': 'Payload installation', 'to_ids': 1}, 'gene': {'default_category': 'Artifacts dropped', 'to_ids': 0}, 'mime-type': {'default_category': 'Artifacts dropped', 'to_ids': 0}, 'identity-card-number': {'default_category': 'Person', 'to_ids': 0}, 'cookie': {'default_category': 'Network activity', 'to_ids': 0}, 'vulnerability': {'default_category': 'External analysis', 'to_ids': 0}, 'attachment': {'default_category': 'External analysis', 'to_ids': 0}, 'malware-sample': {'default_category': 'Payload delivery', 'to_ids': 1}, 'link': {'default_category': 'External analysis', 'to_ids': 0}, 'comment': {'default_category': 'Other', 'to_ids': 0}, 'text': {'default_category': 'Other', 'to_ids': 0}, 'hex': {'default_category': 'Other', 'to_ids': 0}, 'other': {'default_category': 'Other', 'to_ids': 0}, 'named pipe': {'default_category': 'Artifacts dropped', 'to_ids': 0}, 'mutex': {'default_category': 'Artifacts dropped', 'to_ids': 1}, 'target-user': {'default_category': 'Targeting data', 'to_ids': 0}, 'target-email': {'default_category': 'Targeting data', 'to_ids': 0}, 'target-machine': {'default_category': 'Targeting data', 'to_ids': 0}, 'target-org': {'default_category': 'Targeting data', 'to_ids': 0}, 'target-location': {'default_category': 'Targeting data', 'to_ids': 0}, 'target-external': {'default_category': 'Targeting data', 'to_ids': 0}, 'btc': {'default_category': 'Financial fraud', 'to_ids': 1}, 'xmr': {'default_category': 'Financial fraud', 'to_ids': 1}, 'iban': {'default_category': 'Financial fraud', 'to_ids': 1}, 'bic': {'default_category': 'Financial fraud', 'to_ids': 1}, 'bank-account-nr': {'default_category': 'Financial fraud', 'to_ids': 1}, 'aba-rtn': {'default_category': 'Financial fraud', 'to_ids': 1}, 'bin': {'default_category': 'Financial fraud', 'to_ids': 1}, 'cc-number': {'default_category': 'Financial fraud', 'to_ids': 1}, 'prtn': {'default_category': 'Financial fraud', 'to_ids': 1}, 'phone-number': {'default_category': 'Person', 'to_ids': 0}, 'threat-actor': {'default_category': 'Attribution', 'to_ids': 0}, 'campaign-name': {'default_category': 'Attribution', 'to_ids': 0}, 'campaign-id': {'default_category': 'Attribution', 'to_ids': 0}, 'malware-type': {'default_category': 'Payload delivery', 'to_ids': 0}, 'uri': {'default_category': 'Network activity', 'to_ids': 1}, 'authentihash': {'default_category': 'Payload delivery', 'to_ids': 1}, 'ssdeep': {'default_category': 'Payload delivery', 'to_ids': 1}, 'imphash': {'default_category': 'Payload delivery', 'to_ids': 1}, 'pehash': {'default_category': 'Payload delivery', 'to_ids': 1}, 'impfuzzy': {'default_category': 'Payload delivery', 'to_ids': 1}, 'sha224': {'default_category': 'Payload delivery', 'to_ids': 1}, 'sha384': {'default_category': 'Payload delivery', 'to_ids': 1}, 'sha512': {'default_category': 'Payload delivery', 'to_ids': 1}, 'sha512/224': {'default_category': 'Payload delivery', 'to_ids': 1}, 'sha512/256': {'default_category': 'Payload delivery', 'to_ids': 1}, 'tlsh': {'default_category': 'Payload delivery', 'to_ids': 1}, 'cdhash': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|authentihash': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|ssdeep': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|imphash': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|impfuzzy': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|pehash': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|sha224': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|sha384': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|sha512': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|sha512/224': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|sha512/256': {'default_category': 'Payload delivery', 'to_ids': 1}, 'filename|tlsh': {'default_category': 'Payload delivery', 'to_ids': 1}, 'windows-scheduled-task': {'default_category': 'Artifacts dropped', 'to_ids': 0}, 'windows-service-name': {'default_category': 'Artifacts dropped', 'to_ids': 0}, 'windows-service-displayname': {'default_category': 'Artifacts dropped', 'to_ids': 0}, 'whois-registrant-email': {'default_category': 'Attribution', 'to_ids': 0}, 'whois-registrant-phone': {'default_category': 'Attribution', 'to_ids': 0}, 'whois-registrant-name': {'default_category': 'Attribution', 'to_ids': 0}, 'whois-registrant-org': {'default_category': 'Attribution', 'to_ids': 0}, 'whois-registrar': {'default_category': 'Attribution', 'to_ids': 0}, 'whois-creation-date': {'default_category': 'Attribution', 'to_ids': 0}, 'x509-fingerprint-sha1': {'default_category': 'Network activity', 'to_ids': 1}, 'x509-fingerprint-md5': {'default_category': 'Network activity', 'to_ids': 1}, 'x509-fingerprint-sha256': {'default_category': 'Network activity', 'to_ids': 1}, 'dns-soa-email': {'default_category': 'Attribution', 'to_ids': 0}, 'size-in-bytes': {'default_category': 'Other', 'to_ids': 0}, 'counter': {'default_category': 'Other', 'to_ids': 0}, 'datetime': {'default_category': 'Other', 'to_ids': 0}, 'cpe': {'default_category': 'Other', 'to_ids': 0}, 'port': {'default_category': 'Network activity', 'to_ids': 0}, 'ip-dst|port': {'default_category': 'Network activity', 'to_ids': 1}, 'ip-src|port': {'default_category': 'Network activity', 'to_ids': 1}, 'hostname|port': {'default_category': 'Network activity', 'to_ids': 1}, 'mac-address': {'default_category': 'Network activity', 'to_ids': 0}, 'mac-eui-64': {'default_category': 'Network activity', 'to_ids': 0}, 'email-dst-display-name': {'default_category': 'Payload delivery', 'to_ids': 0}, 'email-src-display-name': {'default_category': 'Payload delivery', 'to_ids': 0}, 'email-header': {'default_category': 'Payload delivery', 'to_ids': 0}, 'email-reply-to': {'default_category': 'Payload delivery', 'to_ids': 0}, 'email-x-mailer': {'default_category': 'Payload delivery', 'to_ids': 0}, 'email-mime-boundary': {'default_category': 'Payload delivery', 'to_ids': 0}, 'email-thread-index': {'default_category': 'Payload delivery', 'to_ids': 0}, 'email-message-id': {'default_category': 'Payload delivery', 'to_ids': 0}, 'github-username': {'default_category': 'Social network', 'to_ids': 0}, 'github-repository': {'default_category': 'Social network', 'to_ids': 0}, 'github-organisation': {'default_category': 'Social network', 'to_ids': 0}, 'jabber-id': {'default_category': 'Social network', 'to_ids': 0}, 'twitter-id': {'default_category': 'Social network', 'to_ids': 0}, 'first-name': {'default_category': 'Person', 'to_ids': 0}, 'middle-name': {'default_category': 'Person', 'to_ids': 0}, 'last-name': {'default_category': 'Person', 'to_ids': 0}, 'date-of-birth': {'default_category': 'Person', 'to_ids': 0}, 'place-of-birth': {'default_category': 'Person', 'to_ids': 0}, 'gender': {'default_category': 'Person', 'to_ids': 0}, 'passport-number': {'default_category': 'Person', 'to_ids': 0}, 'passport-country': {'default_category': 'Person', 'to_ids': 0}, 'passport-expiration': {'default_category': 'Person', 'to_ids': 0}, 'redress-number': {'default_category': 'Person', 'to_ids': 0}, 'nationality': {'default_category': 'Person', 'to_ids': 0}, 'visa-number': {'default_category': 'Person', 'to_ids': 0}, 'issue-date-of-the-visa': {'default_category': 'Person', 'to_ids': 0}, 'primary-residence': {'default_category': 'Person', 'to_ids': 0}, 'country-of-residence': {'default_category': 'Person', 'to_ids': 0}, 'special-service-request': {'default_category': 'Person', 'to_ids': 0}, 'frequent-flyer-number': {'default_category': 'Person', 'to_ids': 0}, 'travel-details': {'default_category': 'Person', 'to_ids': 0}, 'payment-details': {'default_category': 'Person', 'to_ids': 0}, 'place-port-of-original-embarkation': {'default_category': 'Person', 'to_ids': 0}, 'place-port-of-clearance': {'default_category': 'Person', 'to_ids': 0}, 'place-port-of-onward-foreign-destination': {'default_category': 'Person', 'to_ids': 0}, 'passenger-name-record-locator-number': {'default_category': 'Person', 'to_ids': 0}, 'mobile-application-id': {'default_category': 'Payload delivery', 'to_ids': 1}, 'cortex': {'default_category': 'External analysis', 'to_ids': 0}, 'boolean': {'default_category': 'Other', 'to_ids': 0}}, 'types': ['md5', 'sha1', 'sha256', 'filename', 'pdb', 'filename|md5', 'filename|sha1', 'filename|sha256', 'ip-src', 'ip-dst', 'hostname', 'domain', 'domain|ip', 'email-src', 'email-dst', 'email-subject', 'email-attachment', 'email-body', 'float', 'url', 'http-method', 'user-agent', 'ja3-fingerprint-md5', 'regkey', 'regkey|value', 'AS', 'snort', 'bro', 'pattern-in-file', 'pattern-in-traffic', 'pattern-in-memory', 'yara', 'stix2-pattern', 'sigma', 'gene', 'mime-type', 'identity-card-number', 'cookie', 'vulnerability', 'attachment', 'malware-sample', 'link', 'comment', 'text', 'hex', 'other', 'named pipe', 'mutex', 'target-user', 'target-email', 'target-machine', 'target-org', 'target-location', 'target-external', 'btc', 'xmr', 'iban', 'bic', 'bank-account-nr', 'aba-rtn', 'bin', 'cc-number', 'prtn', 'phone-number', 'threat-actor', 'campaign-name', 'campaign-id', 'malware-type', 'uri', 'authentihash', 'ssdeep', 'imphash', 'pehash', 'impfuzzy', 'sha224', 'sha384', 'sha512', 'sha512/224', 'sha512/256', 'tlsh', 'cdhash', 'filename|authentihash', 'filename|ssdeep', 'filename|imphash', 'filename|impfuzzy', 'filename|pehash', 'filename|sha224', 'filename|sha384', 'filename|sha512', 'filename|sha512/224', 'filename|sha512/256', 'filename|tlsh', 'windows-scheduled-task', 'windows-service-name', 'windows-service-displayname', 'whois-registrant-email', 'whois-registrant-phone', 'whois-registrant-name', 'whois-registrant-org', 'whois-registrar', 'whois-creation-date', 'x509-fingerprint-sha1', 'x509-fingerprint-md5', 'x509-fingerprint-sha256', 'dns-soa-email', 'size-in-bytes', 'counter', 'datetime', 'cpe', 'port', 'ip-dst|port', 'ip-src|port', 'hostname|port', 'mac-address', 'mac-eui-64', 'email-dst-display-name', 'email-src-display-name', 'email-header', 'email-reply-to', 'email-x-mailer', 'email-mime-boundary', 'email-thread-index', 'email-message-id', 'github-username', 'github-repository', 'github-organisation', 'jabber-id', 'twitter-id', 'first-name', 'middle-name', 'last-name', 'date-of-birth', 'place-of-birth', 'gender', 'passport-number', 'passport-country', 'passport-expiration', 'redress-number', 'nationality', 'visa-number', 'issue-date-of-the-visa', 'primary-residence', 'country-of-residence', 'special-service-request', 'frequent-flyer-number', 'travel-details', 'payment-details', 'place-port-of-original-embarkation', 'place-port-of-clearance', 'place-port-of-onward-foreign-destination', 'passenger-name-record-locator-number', 'mobile-application-id', 'cortex', 'boolean'], 'categories': ['Internal reference', 'Targeting data', 'Antivirus detection', 'Payload delivery', 'Artifacts dropped', 'Payload installation', 'Persistence mechanism', 'Network activity', 'Payload type', 'Attribution', 'External analysis', 'Financial fraud', 'Support Tool', 'Social network', 'Person', 'Other'], 'category_type_mappings': {'Internal reference': ['text', 'link', 'comment', 'other', 'hex'], 'Targeting data': ['target-user', 'target-email', 'target-machine', 'target-org', 'target-location', 'target-external', 'comment'], 'Antivirus detection': ['link', 'comment', 'text', 'hex', 'attachment', 'other'], 'Payload delivery': ['md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512', 'sha512/224', 'sha512/256', 'ssdeep', 'imphash', 'impfuzzy', 'authentihash', 'pehash', 'tlsh', 'cdhash', 'filename', 'filename|md5', 'filename|sha1', 'filename|sha224', 'filename|sha256', 'filename|sha384', 'filename|sha512', 'filename|sha512/224', 'filename|sha512/256', 'filename|authentihash', 'filename|ssdeep', 'filename|tlsh', 'filename|imphash', 'filename|impfuzzy', 'filename|pehash', 'mac-address', 'mac-eui-64', 'ip-src', 'ip-dst', 'ip-dst|port', 'ip-src|port', 'hostname', 'domain', 'email-src', 'email-dst', 'email-subject', 'email-attachment', 'email-body', 'url', 'user-agent', 'AS', 'pattern-in-file', 'pattern-in-traffic', 'stix2-pattern', 'yara', 'sigma', 'mime-type', 'attachment', 'malware-sample', 'link', 'malware-type', 'comment', 'text', 'hex', 'vulnerability', 'x509-fingerprint-sha1', 'x509-fingerprint-md5', 'x509-fingerprint-sha256', 'ja3-fingerprint-md5', 'other', 'hostname|port', 'email-dst-display-name', 'email-src-display-name', 'email-header', 'email-reply-to', 'email-x-mailer', 'email-mime-boundary', 'email-thread-index', 'email-message-id', 'mobile-application-id', 'whois-registrant-email'], 'Artifacts dropped': ['md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512', 'sha512/224', 'sha512/256', 'ssdeep', 'imphash', 'impfuzzy', 'authentihash', 'cdhash', 'filename', 'filename|md5', 'filename|sha1', 'filename|sha224', 'filename|sha256', 'filename|sha384', 'filename|sha512', 'filename|sha512/224', 'filename|sha512/256', 'filename|authentihash', 'filename|ssdeep', 'filename|tlsh', 'filename|imphash', 'filename|impfuzzy', 'filename|pehash', 'regkey', 'regkey|value', 'pattern-in-file', 'pattern-in-memory', 'pdb', 'stix2-pattern', 'yara', 'sigma', 'attachment', 'malware-sample', 'named pipe', 'mutex', 'windows-scheduled-task', 'windows-service-name', 'windows-service-displayname', 'comment', 'text', 'hex', 'x509-fingerprint-sha1', 'x509-fingerprint-md5', 'x509-fingerprint-sha256', 'other', 'cookie', 'gene', 'mime-type'], 'Payload installation': ['md5', 'sha1', 'sha224', 'sha256', 'sha384', 'sha512', 'sha512/224', 'sha512/256', 'ssdeep', 'imphash', 'impfuzzy', 'authentihash', 'pehash', 'tlsh', 'cdhash', 'filename', 'filename|md5', 'filename|sha1', 'filename|sha224', 'filename|sha256', 'filename|sha384', 'filename|sha512', 'filename|sha512/224', 'filename|sha512/256', 'filename|authentihash', 'filename|ssdeep', 'filename|tlsh', 'filename|imphash', 'filename|impfuzzy', 'filename|pehash', 'pattern-in-file', 'pattern-in-traffic', 'pattern-in-memory', 'stix2-pattern', 'yara', 'sigma', 'vulnerability', 'attachment', 'malware-sample', 'malware-type', 'comment', 'text', 'hex', 'x509-fingerprint-sha1', 'x509-fingerprint-md5', 'x509-fingerprint-sha256', 'mobile-application-id', 'other', 'mime-type'], 'Persistence mechanism': ['filename', 'regkey', 'regkey|value', 'comment', 'text', 'other', 'hex'], 'Network activity': ['ip-src', 'ip-dst', 'ip-dst|port', 'ip-src|port', 'port', 'hostname', 'domain', 'domain|ip', 'mac-address', 'mac-eui-64', 'email-dst', 'url', 'uri', 'user-agent', 'http-method', 'AS', 'snort', 'pattern-in-file', 'stix2-pattern', 'pattern-in-traffic', 'attachment', 'comment', 'text', 'x509-fingerprint-md5', 'x509-fingerprint-sha1', 'x509-fingerprint-sha256', 'ja3-fingerprint-md5', 'other', 'hex', 'cookie', 'hostname|port', 'bro'], 'Payload type': ['comment', 'text', 'other'], 'Attribution': ['threat-actor', 'campaign-name', 'campaign-id', 'whois-registrant-phone', 'whois-registrant-email', 'whois-registrant-name', 'whois-registrant-org', 'whois-registrar', 'whois-creation-date', 'comment', 'text', 'x509-fingerprint-sha1', 'x509-fingerprint-md5', 'x509-fingerprint-sha256', 'other', 'dns-soa-email'], 'External analysis': ['md5', 'sha1', 'sha256', 'filename', 'filename|md5', 'filename|sha1', 'filename|sha256', 'ip-src', 'ip-dst', 'ip-dst|port', 'ip-src|port', 'mac-address', 'mac-eui-64', 'hostname', 'domain', 'domain|ip', 'url', 'user-agent', 'regkey', 'regkey|value', 'AS', 'snort', 'bro', 'pattern-in-file', 'pattern-in-traffic', 'pattern-in-memory', 'vulnerability', 'attachment', 'malware-sample', 'link', 'comment', 'text', 'x509-fingerprint-sha1', 'x509-fingerprint-md5', 'x509-fingerprint-sha256', 'ja3-fingerprint-md5', 'github-repository', 'other', 'cortex'], 'Financial fraud': ['btc', 'xmr', 'iban', 'bic', 'bank-account-nr', 'aba-rtn', 'bin', 'cc-number', 'prtn', 'phone-number', 'comment', 'text', 'other', 'hex'], 'Support Tool': ['link', 'text', 'attachment', 'comment', 'other', 'hex'], 'Social network': ['github-username', 'github-repository', 'github-organisation', 'jabber-id', 'twitter-id', 'email-src', 'email-dst', 'comment', 'text', 'other', 'whois-registrant-email'], 'Person': ['first-name', 'middle-name', 'last-name', 'date-of-birth', 'place-of-birth', 'gender', 'passport-number', 'passport-country', 'passport-expiration', 'redress-number', 'nationality', 'visa-number', 'issue-date-of-the-visa', 'primary-residence', 'country-of-residence', 'special-service-request', 'frequent-flyer-number', 'travel-details', 'payment-details', 'place-port-of-original-embarkation', 'place-port-of-clearance', 'place-port-of-onward-foreign-destination', 'passenger-name-record-locator-number', 'comment', 'text', 'other', 'phone-number', 'identity-card-number'], 'Other': ['comment', 'text', 'other', 'size-in-bytes', 'counter', 'datetime', 'cpe', 'port', 'float', 'hex', 'phone-number', 'boolean']}}}
DEBUG [aping.py:2073 - _prepare_request() ] GET - http://<url>/events/5d89ea9a-0778-4a90-a4d0-05a7ac1002c7
DEBUG [aping.py:2075 - _prepare_request() ] {}
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.7', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'm3uBBfGH94cDSSsDvZv7Doqpihgsjg1Zc3WPYs5r', 'content-type': 'application/json'}
DEBUG [aping.py:2047 - _check_response() ] {'Event': {'id': '9', 'orgc_id': '1', 'org_id': '1', 'date': '2019-09-24', 'threat_level_id': '3', 'info': "Today's event", 'published': False, 'uuid': '5d89ea9a-0778-4a90-a4d0-05a7ac1002c7', 'attribute_count': '6', 'analysis': '0', 'timestamp': '1569402304', 'distribution': '1', 'proposal_email_lock': False, 'locked': False, 'publish_timestamp': '0', 'sharing_group_id': '0', 'disable_correlation': False, 'extends_uuid': '', 'event_creator_email': 'admin@admin.test', 'Org': {'id': '1', 'name': '--', 'uuid': '5c34585c-ac5c-46bd-bafa-45570c8b271e'}, 'Orgc': {'id': '1', 'name': '--', 'uuid': '5c34585c-ac5c-46bd-bafa-45570c8b271e'}, 'Attribute': [], 'ShadowAttribute': [], 'RelatedEvent': [], 'Galaxy': [], 'Object': [{'id': '10', 'name': 'file', 'meta-category': 'file', 'description': 'File object describing a file with meta-information', 'template_uuid': '688c46fb-5edb-40a3-8273-1af7923e2215', 'template_version': '15', 'event_id': '9', 'uuid': '5d89f086-8630-4aed-b010-03d9ac1002c7', 'timestamp': '1569321094', 'distribution': '5', 'sharing_group_id': '0', 'comment': 'dsf', 'deleted': False, 'ObjectReference': [], 'Attribute': [{'id': '52', 'type': 'malware-sample', 'category': 'Payload delivery', 'to_ids': True, 'uuid': '5d89f086-defc-4241-a463-03d9ac1002c7', 'event_id': '9', 'distribution': '5', 'timestamp': '1569321094', 'comment': '', 'sharing_group_id': '0', 'deleted': False, 'disable_correlation': False, 'object_id': '10', 'object_relation': 'malware-sample', 'value': 'MISP Tags.png|ec90a596bdb7cf99857427ec0934cded', 'Galaxy': [], 'data': '<data>', 'ShadowAttribute': [], 'Tag': [{'id': '1', 'name': 'MOPR_Benign', 'colour': '#00ff19', 'exportable': True, 'user_id': '0', 'hide_tag': False, 'numerical_value': None}]}, {'id': '53', 'type': 'filename', 'category': 'Payload delivery', 'to_ids': False, 'uuid': '5d89f086-88a4-455f-a154-03d9ac1002c7', 'event_id': '9', 'distribution': '5', 'timestamp': '1569321094', 'comment': '', 'sharing_group_id': '0', 'deleted': False, 'disable_correlation': False, 'object_id': '10', 'object_relation': 'filename', 'value': 'MISP Tags.png', 'Galaxy': [], 'ShadowAttribute': []}, {'id': '54', 'type': 'md5', 'category': 'Payload delivery', 'to_ids': True, 'uuid': '5d89f086-6aa8-4638-9b6a-03d9ac1002c7', 'event_id': '9', 'distribution': '5', 'timestamp': '1569321094', 'comment': '', 'sharing_group_id': '0', 'deleted': False, 'disable_correlation': False, 'object_id': '10', 'object_relation': 'md5', 'value': 'ec90a596bdb7cf99857427ec0934cded', 'Galaxy': [], 'ShadowAttribute': []}, {'id': '55', 'type': 'sha1', 'category': 'Payload delivery', 'to_ids': True, 'uuid': '5d89f086-64f0-4a56-80b3-03d9ac1002c7', 'event_id': '9', 'distribution': '5', 'timestamp': '1569321094', 'comment': '', 'sharing_group_id': '0', 'deleted': False, 'disable_correlation': False, 'object_id': '10', 'object_relation': 'sha1', 'value': '1f170b71d5c6f0d5c36593a890c4717bff06cd2e', 'Galaxy': [], 'ShadowAttribute': []}, {'id': '56', 'type': 'sha256', 'category': 'Payload delivery', 'to_ids': True, 'uuid': '5d89f086-7450-455f-b2eb-03d9ac1002c7', 'event_id': '9', 'distribution': '5', 'timestamp': '1569321094', 'comment': '', 'sharing_group_id': '0', 'deleted': False, 'disable_correlation': False, 'object_id': '10', 'object_relation': 'sha256', 'value': '7d25eeb7bfa3c66ee2123c5c5cd99c141318e44bae0022bc824467b71cb52a00', 'Galaxy': [], 'ShadowAttribute': []}, {'id': '57', 'type': 'size-in-bytes', 'category': 'Other', 'to_ids': False, 'uuid': '5d89f086-efd8-4594-a197-03d9ac1002c7', 'event_id': '9', 'distribution': '5', 'timestamp': '1569321094', 'comment': '', 'sharing_group_id': '0', 'deleted': False, 'disable_correlation': True, 'object_id': '10', 'object_relation': 'size-in-bytes', 'value': '3159', 'Galaxy': [], 'ShadowAttribute': []}]}], 'Tag': [{'id': '4', 'name': 'Signal', 'colour': '#0057ff', 'exportable': True, 'user_id': '0', 'hide_tag': False, 'numerical_value': None}, {'id': '1', 'name': 'MOPR_Benign', 'colour': '#00ff19', 'exportable': True, 'user_id': '0', 'hide_tag': False, 'numerical_value': None}]}}
Adding MOPR-report Object
DEBUG [aping.py:2073 - _prepare_request() ] POST - http://<url>/objects/add/5d89ea9a-0778-4a90-a4d0-05a7ac1002c7
DEBUG [aping.py:2075 - _prepare_request() ] <MISPObject(name=mopr-report)
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.7', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Content-Length': '833', 'Authorization': 'm3uBBfGH94cDSSsDvZv7Doqpihgsjg1Zc3WPYs5r', 'content-type': 'application/json'}
ERROR [aping.py:2039 - _check_response() ] Something went wrong (403): {'name': 'Could not add object', 'message': 'Could not add object', 'url': '/objects/add/9/', 'errors': 'No valid template found to edit the object.'}

@kovacsbalu
Copy link
Contributor

My debug output is only differ in this request:
Your: GET - http://<url>/events/5d89ea9a-0778-4a90-a4d0-05a7ac1002c7
My: POST - https://misp.url/events/restSearch
and the result :)

@thomasb454
Copy link
Author

thomasb454 commented Sep 26, 2019

@kovacsbalu That's strange, could you try on the same version as me and report the results?

@Rafiot Rafiot self-assigned this Sep 26, 2019
@kovacsbalu
Copy link
Contributor

@thomasb454 I created MISP 2.4.99 in docker env and I could reproduce

DEBUG [aping.py:2073 - _prepare_request() ] GET - http://localhost/servers/getPyMISPVersion.json
DEBUG [aping.py:2075 - _prepare_request() ] {}
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.6', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'gY0n252vbcOi8EHgmM0xmw7eEGGhEwI9TfUmUoS7', 'content-type': 'application/json'}
DEBUG [aping.py:2047 - _check_response() ] {'version': '2.4.98'}
INFO [aping.py:79 - __init__() ] The version of PyMISP recommended by the MISP instance (response['version']) is older than the one you're using now (2.4.114). If you have a problem, please upgrade the MISP instance or use an older PyMISP version.
DEBUG [aping.py:2073 - _prepare_request() ] GET - http://localhost/servers/getVersion.json
DEBUG [aping.py:2075 - _prepare_request() ] {}
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.6', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'gY0n252vbcOi8EHgmM0xmw7eEGGhEwI9TfUmUoS7', 'content-type': 'application/json'}
DEBUG [aping.py:2047 - _check_response() ] {'version': '2.4.99', 'perm_sync': True}
DEBUG [aping.py:2073 - _prepare_request() ] GET - http://localhost/attributes/describeTypes.json
DEBUG [aping.py:2075 - _prepare_request() ] {}
....
DEBUG [aping.py:2073 - _prepare_request() ] POST - http://localhost/objects/add/5d8d10bc-1b9c-4379-b46e-0034ac130003
DEBUG [aping.py:2075 - _prepare_request() ] <MISPObject(name=myobj)
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.6', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Content-Length': '456', 'Authorization': 'gY0n252vbcOi8EHgmM0xmw7eEGGhEwI9TfUmUoS7', 'content-type': 'application/json'}
ERROR [aping.py:2039 - _check_response() ] Something went wrong (403): {'name': 'Could not add object', 'message': 'Could not add object', 'url': '/objects/add/1/', 'errors': 'No valid template found to edit the object.'}

So probably you need to upgrade your MISP.
I will try the same with the latest version 2.4.116

@kovacsbalu
Copy link
Contributor

MISP 2.4.116

DEBUG [aping.py:2073 - _prepare_request() ] GET - http://localhost/servers/getPyMISPVersion.json
DEBUG [aping.py:2075 - _prepare_request() ] {}
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.6', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'ISSB8xUulacfME7NyUUSPyT9eWjr9rW3WSTLrWpJ', 'content-type': 'application/json'}
DEBUG [aping.py:2047 - _check_response() ] {'version': '2.4.114'}
DEBUG [aping.py:2073 - _prepare_request() ] GET - http://localhost/servers/getVersion.json
DEBUG [aping.py:2075 - _prepare_request() ] {}
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.6', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'ISSB8xUulacfME7NyUUSPyT9eWjr9rW3WSTLrWpJ', 'content-type': 'application/json'}
DEBUG [aping.py:2047 - _check_response() ] {'version': '2.4.116', 'perm_sync': True}
DEBUG [aping.py:2073 - _prepare_request() ] GET - http://localhost/attributes/describeTypes.json
DEBUG [aping.py:2075 - _prepare_request() ] {}
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.6', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Authorization': 'ISSB8xUulacfME7NyUUSPyT9eWjr9rW3WSTLrWpJ', 'content-type': 'application/json'}
.....
DEBUG [aping.py:2073 - _prepare_request() ] POST - http://localhost/objects/add/5d8d1511-c514-4d05-b6b2-010aac180003
DEBUG [aping.py:2075 - _prepare_request() ] <MISPObject(name=myobj)
DEBUG [aping.py:2100 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.114 - Python 3.6', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Content-Length': '456', 'Authorization': 'ISSB8xUulacfME7NyUUSPyT9eWjr9rW3WSTLrWpJ', 'content-type': 'application/json'}
DEBUG [aping.py:2047 - _check_response() ] {'Object': {'id': '1', 'name': 'myobj', 'meta-category': 'misc', 'description': 'myobj details', 'template_uuid': 'a41d6985-ec35-4930-9565-a4fef440b616', 'template_version': '1', 'event_id': '1', 'uuid': '3b295fb3-42cc-4496-bfe1-e1537e59d692', 'timestamp': '1569527129', 'distribution': '5', 'sharing_group_id': '0', 'comment': '', 'deleted': False, 'Attribute': [{'id': '1', 'event_id': '1', 'object_id': '1', 'object_relation': 'url', 'category': 'Network activity', 'type': 'url', 'value1': 'valami.hu', 'value2': '', 'to_ids': True, 'uuid': '67434fb0-dab0-4966-8614-2dc88f261528', 'timestamp': '1569527129', 'distribution': '5', 'sharing_group_id': '0', 'comment': '', 'deleted': False, 'disable_correlation': False, 'value': 'valami.hu'}]}}

@thomasb454
Copy link
Author

Hi @kovacsbalu,
Based on the output from MISP 2.4.116 it was successful?

@kovacsbalu
Copy link
Contributor

Yes, and as I wrote before also works with 2.4.109

@thomasb454
Copy link
Author

thomasb454 commented Sep 27, 2019 via email

@thomasb454
Copy link
Author

Hi,

After revisiting this I believe I found the cause of this bug.
PyMISP version: 2.4.117.2

My code is as follows:

for result in results:
    new_event = MISPEvent()
    new_event.extends_uuid = result.uuid
    new_event.info = "TESTING EXTENDS"
    res = misp.add_event(new_event)
    new_uuid = res['Event']['uuid']
    new_id = res['Event']['id']

    mopr_obj = MISPObject(name='mopr-report', misp_objects_path_custom='mopr')
    mopr_obj.add_attribute('score', value=result.score)

    misp.add_object(new_id, misp_object=mopr_obj)

This doesn't work and produces the following debug output:

DEBUG [aping.py:2192 - _prepare_request() ] POST - http://<host>/objects/add/33
DEBUG [aping.py:2194 - _prepare_request() ] <MISPObject(name=mopr-report)
DEBUG [aping.py:2219 - _prepare_request() ] {'User-Agent': 'PyMISP 2.4.117.2 - Python 3.7', 'Accept-Encoding': 'gzip, deflate', 'Accept': 'application/json', 'Connection': 'keep-alive', 'Content-Length': '664', 'Authorization': '<key>', 'content-type': 'application/json'}
ERROR [aping.py:2158 - _check_response() ] Something went wrong (403): {'name': 'Could not add object', 'message': 'Could not add object', 'url': '/objects/add/33/', 'errors': 'No valid template found to edit the object.'}

As you can see it's sending the request to /objects/add/33 - but if you try to do the same via the MISP UI it will send the request to /objects/add/[event id]/[object template id].

I fixed this issue by changing the following line in aping.py (line 280)
BEFORE:

        new_object = self._prepare_request('POST', f'objects/add/{event_id}', data=misp_object)

AFTER:

        new_object = self._prepare_request('POST', f'objects/add/{event_id}/116', data=misp_object)

In this case I hard-coded 116, which is the ID of my object template - when fixed this obviously needs to be dynamically applied.

Rafiot added a commit that referenced this issue Nov 15, 2019
@Rafiot
Copy link
Member

Rafiot commented Nov 15, 2019

You should never pass the template ID when you're using PyMISP, I'm not sure how you end-up in this situation.

I patched ExpandedPyMISP to print the json blob and make the debug easier, but canyou show me the content of mopr_obj?

@Rafiot
Copy link
Member

Rafiot commented Nov 15, 2019

This commit may also solve your problem: MISP/MISP@e4c82eb

@thomasb454
Copy link
Author

Hi,
My original work around was not for ExpandedPyMISP - just PyMISP. I'm out of the office right now so I'll have to get back to you Monday.

@Rafiot
Copy link
Member

Rafiot commented Nov 15, 2019

Oh, right, PyMISP will not be fixed at this point, as it will go away in ~45 days.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T: support Type: support. This issue is seeking support on a problem or question
Projects
None yet
Development

No branches or pull requests

3 participants