Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add malware_signature for Memory protection alert #155

Merged
merged 12 commits into from
Jul 28, 2021
40 changes: 40 additions & 0 deletions custom_schemas/custom_malware_signature.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
- name: malware_signature
title: Malware Signature
group: 2
short: These fields contain the yara malware signature.
description: >
These fields contain the yara malware signature.
reusable:
order: 1
top_level: false
expected:
- process.Ext
- file.Ext
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a heads up it doesn't look like malware_signature is being placed under file.Ext 🤔 that might be because the file subset file doesn't include it 🤷‍♂️ is that ok?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the title. All fields added here are for Memory Protection plugin.

type: group
fields:
- name: all_names
level: custom
type: keyword
example: "Windows.EICAR.Not-a-virus"
description: A sequence of signature names matched.

- name: identifier
level: custom
type: keyword
description: The model's unique identifier.

- name: primary
level: custom
type: object
gogochan marked this conversation as resolved.
Show resolved Hide resolved
description: The first matching details.

- name: secondary
level: custom
type: array
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Elasticsearch there isn't an array type. Everything can be an array out of the box: https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-types.html#types-array-handling

Could we change this to the structure and type that will be contained within secondary? If secondary will be the exact same as primary, you could abstract it out to a new file and have it overlaid as both primary and secondary using the ECS reuse mechanism.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

secondary will be a list of object that are subset of the primary. Should I just say object here then?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we use object then users won't be able to search for it beyond just using a search based on its existence. If we don't expect users to search for the secondary fields then you could exclude it from the definition here. The only reason we want to define it in the endpoint package is so that users can search for it. If you want to include it just as a description of a field that could exist in an alert, we can add enabled: false (e.g. https://github.com/elastic/endpoint-package/blob/master/custom_schemas/custom_endpoint.yml#L28) that way Elasticsearch will not index it.

description: Additional matching details if available.

- name: version
level: custom
type: keyword
description: The version of the model used.
4 changes: 4 additions & 0 deletions custom_subsets/elastic_endpoint/alerts/malware_event.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -315,6 +315,8 @@ fields:
buffer: {}
decompressed_size: {}
encoding: {}
malware_signature:
fields: "*"
gogochan marked this conversation as resolved.
Show resolved Hide resolved
macro:
fields:
errors:
Expand Down Expand Up @@ -469,6 +471,8 @@ fields:
exceptionable: true
status:
exceptionable: true
malware_signature:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this go into memory_protection_event, since Memory Scan is presented to the user as part of Memory Protection? https://github.com/elastic/endpoint-package/blob/c23a052ef81ea39ab2934e2e1ea5b32bff102196/custom_subsets/elastic_endpoint/alerts/memory_protection_event.yaml

Copy link
Contributor

@ferullo ferullo Jun 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should stay where it is at the very least for malware (FileScore) events. We could duplicate it to another location as well for use by memory alerts though I'd vote against that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I take it back. I now see you're commenting on process.Ext.malware_signature not file.Exe.malware_signature.

Currently Linux only sends file activity based malware (FileScore) alerts (open, write, etc) so wouldn't ever need this location. But Windows and macOS can send process based malware alerts (process start). So I still think this is needed for Windows and macOS signature based FileScore alerts (and wonder why testing has been OK without this so far).

But that's only whether or not this update is beneficial to malware alerts. My presumption would be that a standardized key name for yara signature matches regardless of the protection type would be better than it going in different places for different alert types. But I don't feel strongly, if my presumption is wrong I'll stay silent.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per our discussion, I will remove this line from this PR. We will add it back in 7.15

fields: "*"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything else here looks like this:

Suggested change
fields: "*"
fields: {}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should update the docs on "*" vs {} because I always forget haha. {} only works when the field is a leaf aka if we only wanted malware_signature.all_names in the mapping we could write it like:

malware_signature:
  fields:
    all_names: {}

Since we want everything under malware_signature and fields is an array we have to use the "*" syntax like Chan has here.

malware_classification:
fields:
score: {}
Expand Down
108 changes: 108 additions & 0 deletions generated/alerts/ecs/ecs_flat.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

108 changes: 108 additions & 0 deletions generated/alerts/ecs/subset/malware_event/ecs_flat.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading