Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Filebeat module for monitoring Santa #9540

Merged
merged 4 commits into from Dec 17, 2018
Merged

Add Filebeat module for monitoring Santa #9540

merged 4 commits into from Dec 17, 2018

Conversation

andrewkroh
Copy link
Member

@andrewkroh andrewkroh commented Dec 14, 2018 •
edited

This adds a Filebeat module for monitoring and parsing the log file generated
by the Google Santa, a process monitoring tool for MacOS. This module includes
an overview dashboard.

https://github.com/google/santa

filebeat-santa-log-overview-dashboard

ruflin
ruflin reviewed Dec 14, 2018
View reviewed changes
Copy link
Member

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎅 Overall LGTM. I wonder if that is a module we should keep in the long term? ;-)

filebeat/module/santa/_meta/fields.yml
description: >
Santa Module
fields:
- name: santa
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume all fields are on the module level on purpose?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'm never expecting there to be anything other than the "log" dataset so dropped the .log. Hopefully that's ok.

webmat
webmat approved these changes Dec 14, 2018
View reviewed changes
Copy link
Contributor

@webmat webmat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Suggesting a minor improvement to make conversion to int easier, and also convert user/group.id to int.

filebeat/module/santa/log/ingest/pipeline.json Outdated
"grok": {
"field": "message",
"patterns": [
"\\[%{TIMESTAMP_ISO8601:process.start}\\] I santad: action=%{NOT_SEPARATOR:santa.action}\\|decision=%{NOT_SEPARATOR:santa.decision}\\|reason=%{NOT_SEPARATOR:santa.reason}\\|sha256=%{NOT_SEPARATOR:hash.sha256}\\|path=%{NOT_SEPARATOR:process.executable}(\\|args=%{NOT_SEPARATOR:process.args})?(\\|cert_sha256=%{NOT_SEPARATOR:certificate.sha256})?(\\|cert_cn=%{NOT_SEPARATOR:certificate.common_name})?\\|pid=%{NUMBER:process.pid}\\|ppid=%{NUMBER:process.ppid}\\|uid=%{NUMBER:user.id}\\|user=%{NOT_SEPARATOR:user.name}\\|gid=%{NUMBER:group.id}\\|group=%{NOT_SEPARATOR:group.name}\\|mode=%{WORD:santa.mode}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"\\[%{TIMESTAMP_ISO8601:process.start}\\] I santad: action=%{NOT_SEPARATOR:santa.action}\\|decision=%{NOT_SEPARATOR:santa.decision}\\|reason=%{NOT_SEPARATOR:santa.reason}\\|sha256=%{NOT_SEPARATOR:hash.sha256}\\|path=%{NOT_SEPARATOR:process.executable}(\\|args=%{NOT_SEPARATOR:process.args})?(\\|cert_sha256=%{NOT_SEPARATOR:certificate.sha256})?(\\|cert_cn=%{NOT_SEPARATOR:certificate.common_name})?\\|pid=%{NUMBER:process.pid}\\|ppid=%{NUMBER:process.ppid}\\|uid=%{NUMBER:user.id}\\|user=%{NOT_SEPARATOR:user.name}\\|gid=%{NUMBER:group.id}\\|group=%{NOT_SEPARATOR:group.name}\\|mode=%{WORD:santa.mode}"
"\\[%{TIMESTAMP_ISO8601:process.start}\\] I santad: action=%{NOT_SEPARATOR:santa.action}\\|decision=%{NOT_SEPARATOR:santa.decision}\\|reason=%{NOT_SEPARATOR:santa.reason}\\|sha256=%{NOT_SEPARATOR:hash.sha256}\\|path=%{NOT_SEPARATOR:process.executable}(\\|args=%{NOT_SEPARATOR:process.args})?(\\|cert_sha256=%{NOT_SEPARATOR:certificate.sha256})?(\\|cert_cn=%{NOT_SEPARATOR:certificate.common_name})?\\|pid=%{NUMBER:process.pid:int}\\|ppid=%{NUMBER:process.ppid:int}\\|uid=%{NUMBER:user.id:int}\\|user=%{NOT_SEPARATOR:user.name}\\|gid=%{NUMBER:group.id:int}\\|group=%{NOT_SEPARATOR:group.name}\\|mode=%{WORD:santa.mode}"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion to simplify the pipeline, and remove the need for a few convert blocks below.

Notice that I'm also converting the user.id and group.id in this suggestion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated as suggested, but I did not change user.id or group.id because they are keyword in ECS (I assume this is in order to accommodate SID values from Windows).

filebeat/module/santa/_meta/fields.yml
description: SHA256 hash of code signing certificate.

# Auditbeat FIM is using this field for the same purpose.
- name: hash.sha256
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it common elsewhere to use certificate and keyword at the top level?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The closest thing we have is tls.{client,server}_certificate.fingerprint.sha256.

@webmat
Copy link
Contributor

webmat commented Dec 14, 2018

@ruflin The project has been alive for about 4 years. I doubt it's a holiday prank LOL.

@andrewkroh andrewkroh requested a review from a team as a code owner December 17, 2018 03:13
@andrewkroh
Add Filebeat module for monitoring Santa
5dd44e8 
This adds a Filebeat module for monitoring and parsing the log file generated
by the Google Santa, a process monitoring tools for MacOS. This module includes
an overview dashboard.

https://github.com/google/santa
@andrewkroh
Make pid and ppid integers in grok pattern
0a6362a
@jsoriano
Copy link
Member

jsoriano commented Dec 17, 2018 •
edited

Oh when I read the title I thought this was going to be an easter egg! 😄 🎅

@tsg tsg added the SecOps label Dec 17, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/secops

ruflin
ruflin approved these changes Dec 17, 2018
View reviewed changes
Copy link
Member

@ruflin ruflin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Let's backport to 6.x?

@andrewkroh andrewkroh merged commit 4962a59 into elastic:master Dec 17, 2018
@andrewkroh andrewkroh added the needs_backport PR is waiting to be backported to other branches. label Dec 18, 2018
@andrewkroh
Copy link
Member Author

It can be backported. I'll add the label as a reminder. I'm pretty sure the backport will need some fields added to fields.yml since this currently depends on some ECS fields.

@cwurm cwurm mentioned this pull request Feb 6, 2019
Merged
@MikePaquette
Copy link

@andrewkroh should this be included in our announcement of features in the 7.0 release? I don't think we've mentioned it anywhere yet. Should it be listed as ECS-compatible? Thanks.

@andrewkroh
Copy link
Member Author

@MikePaquette Absolutely. And it is ECS-compatible.

@andrewkroh andrewkroh removed the needs_backport PR is waiting to be backported to other branches. label Aug 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@webmat webmat webmat approved these changes

@jsoriano jsoriano jsoriano approved these changes

@ruflin ruflin ruflin approved these changes

Assignees
No one assigned
Labels
Filebeat Filebeat module review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@andrewkroh @webmat @jsoriano @elasticmachine @MikePaquette @ruflin @tsg