Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] MISP 2.0 analyzer search crashes the MISP instance #602

Closed
Kamforka opened this issue Jan 29, 2020 · 9 comments
Closed

[Bug] MISP 2.0 analyzer search crashes the MISP instance #602

Kamforka opened this issue Jan 29, 2020 · 9 comments
Labels
bug
Milestone

Comments

@Kamforka
Copy link

@Kamforka Kamforka commented Jan 29, 2020

Describe the bug
The MISP analyzer is "crashing" the MISP server when pymisp version 2.4.120 is used.

To Reproduce
Steps to reproduce the behavior:

  1. Install pymisp client version 2.4.120
  2. Run the MISP analyzer with any observable
  3. Memory consumption of the MISP server will increase up to a set resource limit if such exists, otherwise just conumes all available system memory (at least I tested it with a 16GB machine and it always ate up all the mem and crashed)

Expected behavior
To run as before.

Work environment

  • Client OS: Ubuntu 18.04
  • Server OS: Ubuntu 18.04
  • Browse type and version:
  • Cortex version: 3.0.0-1
  • Cortex Analyzer/Responder name: MISP
  • Cortex Analyzer/Responder version: 2.0

Possible solutions
As a short term solution it's enough to pin the pymisp requirements to version 2.4.119.
For a long term solution I'd revisit the search queries used by the analyzer and limit them in a sensible way not to make MISP load a lot of resources into memory, although it's a shame that you can crash a system via its ReST API.

Additional context
The analyzer is using the legacy/deprecated client of pymisp which is never a good thing.

@Kamforka Kamforka added the bug label Jan 29, 2020
@3c7

This comment has been minimized.

Copy link
Member

@3c7 3c7 commented Jan 29, 2020

Not able to reproduce, is your MISP commit newer than MISP/MISP@f88743c?

@3c7

This comment has been minimized.

Copy link
Member

@3c7 3c7 commented Jan 29, 2020

Oh. Forget that, somehow mistyped the pip command to update PyMISP on my test machine.

@Kamforka

This comment has been minimized.

Copy link
Author

@Kamforka Kamforka commented Jan 29, 2020

Not able to reproduce, is your MISP commit newer than MISP/MISP@f88743c?

I tried with both MISP server version 2.4.119 and 2.4.120.

To be honest this bug should be there for quite some time, but the analyzer is using the legacy interface of pymisp that's why it worked, because the ExpandedPyMISP class was killing MISP with earlier pymisp versions with the same search command.

3c7 added a commit that referenced this issue Jan 30, 2020
@3c7

This comment has been minimized.

Copy link
Member

@3c7 3c7 commented Jan 30, 2020

There were two issues, first the reponse dictionary changed and the search request must use value instead of values as parameter. Fixed in 2.4.0. Can you give the release/2.4.0 branch a shot, if you have the time?

@Kamforka

This comment has been minimized.

Copy link
Author

@Kamforka Kamforka commented Jan 30, 2020

Yep I tried with value instead of values already, so I think it should be fine.

Other than that, I'd suggest you to use the new ExpandedPyMISP client instead of the deprecated PyMISP one.

@3c7

This comment has been minimized.

Copy link
Member

@3c7 3c7 commented Jan 30, 2020

ExpandedPyMISP == PyMISP since 2.4.120.

@3c7

This comment has been minimized.

@Kamforka

This comment has been minimized.

Copy link
Author

@Kamforka Kamforka commented Jan 30, 2020

@3c7 I know that but in case someone has an earlier version of pymisp installed on the Cortex machine, then the MISP 2.1 analyzer will be using two different classes in the background, (the legacy) PyMISP for versions <2.4.120 and (the new) ExpandedPyMISP for versions >=2.4.120.

So this search:

pymisp.search(value="192.168.0.1", type_attribute=["ip-src", "ip-dst"])

Will send out the following bodies to MISP:

pymisp < 2.4.120:

{'returnFormat': 'json', 'type': ['ip-src', 'ip-dst'], 'value': '192.168.0.1'}

pymisp >= 2.4.120:

{'category': None,
 'deleted': None,
 'enforceWarninglist': 0,
 'eventid': None,
 'eventinfo': None,
 'from': None,
 'headerless': 0,
 'includeContext': 0,
 'includeCorrelations': 0,
 'includeEventTags': 0,
 'includeEventUuid': 0,
 'includeSightings': 0,
 'limit': None,
 'metadata': 0,
 'org': None,
 'page': None,
 'published': None,
 'quickFilter': None,
 'requested_attributes': None,
 'returnFormat': 'json',
 'searchall': None,
 'sgReferenceOnly': 0,
 'tags': None,
 'to': None,
 'type': ['ip-src', 'ip-dst'],
 'uuid': None,
 'value': '192.168.0.1',
 'withAttachments': 0}

The newer body is basically a superset of the older body, so normally it should cause no problems, but I don't know if all the other body values could cause any inconsistency between the two searches.

3c7 added a commit that referenced this issue Jan 30, 2020
@3c7

This comment has been minimized.

Copy link
Member

@3c7 3c7 commented Jan 30, 2020

That makes sense, thank you.

Edit: Also, because of the response it would crash with an earlier version of PyMISP.

@3c7 3c7 added this to the 2.4.0 milestone Jan 30, 2020
3c7 added a commit that referenced this issue Jan 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.