[Juniper SRX] Support SRX system logs pattern #6183
Conversation
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsExpand to view the GitHub comments
To re-run your PR in the CI, just comment with:
|
🌐 Coverage report
|
kcreddy
changed the title
|
Pinging @elastic/security-external-integrations (Team:Security-External Integrations) |
| - _temp_.to_be_parsed | ||
| - _temp_ |
There was a problem hiding this comment.
You would only need _temp_ here, as it removes the whole object
There was a problem hiding this comment.
Updated as suggested
| type: keyword | ||
| - name: argument1 | ||
| type: keyword | ||
| - name: index1 |
There was a problem hiding this comment.
There are quite a few new fields here, many which seems related to ECS. If these fields are converted or moved to ECS, they should not require its own mapping. Do we know if any of these might be obsolete?
There was a problem hiding this comment.
Removed few fields which could have been ECS. Added processors to extract
| - juniper.srx.firewall | ||
| ignore_missing: true | ||
|
|
||
| on_failure: |
There was a problem hiding this comment.
Since we are tagging so many of the processors, let's ensure that we also set the proper event.kind. Also if we are append to error.message, then this also needs to be append instead of set
Could you add these lines to both this ingest pipeline, but also at the end of the others?
on_failure:
- appendt:
field: error.message
value: |-
Processor "{{ _ingest.on_failure_processor_type }}" with tag "{{ _ingest.on_failure_processor_tag }}" in pipeline "{{ _ingest.on_failure_pipeline }}" failed with message "{{ _ingest.on_failure_message }}"
- set:
field: event.kind
value: pipeline_error
There was a problem hiding this comment.
Updated as suggested
|
After talking a bit more with Krishna, there are a few concerns with the testdata:
|
|
Moved the logic to system pipeline and added dissect patterns for few sample messages |
|
The PR itself looks good, though there are still some concerns around the data format inconsistencies, and unless we manage to resolve all those questions we should wait with merging. Some discussions on Slack led us to currently plan on making a custom build of the integration that certain key users can test with, together with finalizing the open questions around the format should result in a new datastream being merged, so let's await a bit further first. |
Added new samples to the tests. No grok failures, but most of the logs need new grok/dissect patterns to extract. |
|
Package juniper_srx - 1.13.0 containing this change is available at https://epr.elastic.co/search?package=juniper_srx |
What does this PR do?
Current Juniper SRX package doesn't support system logs. This PR adds support for SRX system logs. Samples requested from customer and anonymized.
Checklist
changelog.ymlfile.How to test this PR locally
Related issues
Screenshots