-
Notifications
You must be signed in to change notification settings - Fork 444
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[cisco_asa] Add grok support for 722022, 722023 logs #9222
[cisco_asa] Add grok support for 722022, 722023 logs #9222
Conversation
Adds support for ingesting Cisco ASA logs with message ids 722022 and 722023.
- Add grok patterns for Cisco ASA logs with message ids 722022, 722023 - Add sample data for pipeline tests
🚀 Benchmarks reportTo see the full report comment with |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The grok patterns aren't extracting the inner values when a field is wrapped in angle brackets. I think for now, it is sufficient to have two patterns, one with angle brackets and one without. In theory, the first pattern should exit as soon as it doesn't see the angle bracket and it will try the next one. This happens very early in the string (it's not like it has to get all the way to the end of the pattern to find that out).
The other thing to consider is that there is a pretty well established pattern in the integration with the other grok processors, where they have two patterns, one with angle brackets, one without. I say we stick to that pattern for now. I tried a few different ideas myself, but I can't get it to quite work. If you want to have one pattern with optionals, you'd have to change the NOTSPACE
pattern to something more restrictive to get it to work (basically something that doesn't include <
and >
).
When testing grok patterns, I like to use the pipeline simulator to do rapid testing:
POST /_ingest/pipeline/_simulate?verbose=true
{
"pipeline" :
{
"description": "_description",
"processors": [
{
"grok": {
"field": "message",
"tag": "grok_message",
"patterns": [
"^Group <%{NOTSPACE:source.user.group.name}> User <%{NOTSPACE:source.user.name}> IP <%{IP:source.address}> %{GREEDYDATA:event.reason}$",
"^Group %{NOTSPACE:source.user.group.name} User %{NOTSPACE:source.user.name} IP %{IP:source.address} %{GREEDYDATA:event.reason}$"
]
}
}
]
},
"docs": [
{
"_index": "index",
"_id": "id",
"_source": {
"message": "Group <MY-MFA> User <src_user@myorg.com> IP <67.43.156.118> TCP SVC connection established without compression"
}
},
{
"_index": "index",
"_id": "id",
"_source": {
"message": "Group MY-MFA User src_user@myorg.com IP 67.43.156.118 TCP SVC connection established without compression"
}
}
]
}
One other thing to note, it is recommended to add anchors to the patterns (^
and $
). I forget the exact reasons why, but it does help with performance.
packages/cisco_asa/data_stream/log/_dev/test/pipeline/test-asa-missing-groups.log-expected.json
Outdated
Show resolved
Hide resolved
packages/cisco_asa/data_stream/log/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
packages/cisco_asa/data_stream/log/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
packages/cisco_asa/data_stream/log/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices) |
- As this is an enhancement, we get a new minor version - Simplify the grok patterns - Force the sample data to conform
💚 Build Succeeded
History
|
Quality Gate passedKudos, no new issues were introduced! 0 New issues |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Package cisco_asa - 2.31.0 containing this change is available at https://epr.elastic.co/search?package=cisco_asa |
[cisco_asa] Add grok support for 722022, 722023 logs - Add grok patterns for Cisco ASA logs with message id 722022 and 722023. - Added sample data to validate the addition during pipeline tests.
Type of change
Proposed commit message
Add grok patterns for Cisco ASA logs with message id 722022 and 722023.
Added sample data to validate the addition during pipeline tests.
Checklist
changelog.yml
file.How to test this PR locally
Related issues