Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pulse_connect_secure: add syslog priority format parsing support #2552

Merged
merged 2 commits into from
Feb 4, 2022
Merged

pulse_connect_secure: add syslog priority format parsing support #2552

merged 2 commits into from
Feb 4, 2022

Conversation

efd6
Copy link
Contributor

@efd6 efd6 commented Jan 23, 2022

What does this PR do?

This adds support for syslog priority value parsing.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

First commit is mechanical; this just runs the new version of elastic-package on to reduce diff noise in commit two.

How to test this PR locally

elastic-package test in package/pulse_connect_secure

Related issues

Screenshots

N/A

@elasticmachine
Copy link

elasticmachine commented Jan 23, 2022 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Reason: null

  • Start Time: 2022-02-04T03:05:29.299+0000

  • Duration: 22 min 26 sec

  • Commit: 72ad407

Test stats 🧪

Test Results
Failed 0
Passed 10
Skipped 0
Total 10

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

legoguy1000
legoguy1000 reviewed Jan 23, 2022
View reviewed changes
packages/pulse_connect_secure/data_stream/log/elasticsearch/ingest_pipeline/default.yml Outdated
@@ -23,6 +23,7 @@ processors:
field: event.original
patterns:
- '%{SYSLOGTIMESTAMP} %{SYSLOGHOST:host.hostname} %{INT} %{TIMESTAMP_ISO8601:_tmp.timestamp} %{IP:observer.ip} PulseSecure: - - - %{DATE2} - %{SYSLOGHOST:observer.name} - \[%{IPORHOST:client.address}\] %{USERNAME:user.name}?\(%{DATA:pulse_secure.realm}?\)\[%{DATA:pulse_secure.role}\] - %{GREEDYDATA:message}'
- '^<%{NONNEGINT}>%{NUMBER}? %{TIMESTAMP_ISO8601:_tmp.timestamp} %{IP:observer.ip} PulseSecure: - - - %{DATE2} - %{SYSLOGHOST:observer.name} - \[%{IPORHOST:client.address}\] %{USERNAME:user.name}?\(%{DATA:pulse_secure.realm}?\)\[%{DATA:pulse_secure.role}\] - %{GREEDYDATA:message}'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would combine the 2 patterns and just add the <%{NONNEGINT}>%{NUMBER}? as a optional item. Besides that, the pattern seems the same thought u changed the host.* to observer.* which might have been the right thing to begin with.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. Yes, it is identical beyond the prefix (no change to host.*. The issue with just making the PRI optional is that the %{SYSLOGTIMESTAMP} %{SYSLOGHOST:host.hostname} %{INT} prefix components do not appear in test cases that prompted the change, so they need to be made optional too, and then an order for the case that they both appear has to be decided. What would you suggest there?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't spot that difference, THen the best thing i'd do is consolidate the similar sections into sub grok patterns as much as you can so its easier to maintain long term

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL

andrewkroh
andrewkroh reviewed Jan 27, 2022
View reviewed changes
packages/pulse_connect_secure/data_stream/log/elasticsearch/ingest_pipeline/default.yml Outdated
@@ -22,7 +22,7 @@ processors:
- grok:
field: event.original
patterns:
- '%{SYSLOGTIMESTAMP} %{SYSLOGHOST:host.hostname} %{INT} %{TIMESTAMP_ISO8601:_tmp.timestamp} %{IP:observer.ip} PulseSecure: - - - %{DATE2} - %{SYSLOGHOST:observer.name} - \[%{IPORHOST:client.address}\] %{USERNAME:user.name}?\(%{DATA:pulse_secure.realm}?\)\[%{DATA:pulse_secure.role}\] - %{GREEDYDATA:message}'
- '^(<%{NONNEGINT}>%{NUMBER}?|%{SYSLOGTIMESTAMP} %{SYSLOGHOST:host.hostname} %{INT}) %{TIMESTAMP_ISO8601:_tmp.timestamp} %{IP:observer.ip} PulseSecure: - - - %{DATE2} - %{SYSLOGHOST:observer.name} - \[%{IPORHOST:client.address}\] %{USERNAME:user.name}?\(%{DATA:pulse_secure.realm}?\)\[%{DATA:pulse_secure.role}\] - %{GREEDYDATA:message}'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about NONNEGINT:log.syslog.priority:long to store the value?

https://www.elastic.co/guide/en/ecs/current/ecs-log.html#field-log-syslog-priority

@efd6 efd6 requested a review from andrewkroh February 3, 2022 08:20
@efd6 efd6 merged commit 84bc5ea into elastic:main Feb 4, 2022
@efd6 efd6 deleted the pulsesyslog branch February 4, 2022 23:11
@andrewkroh andrewkroh mentioned this pull request Feb 9, 2022
Merged
eyalkraft pushed a commit to build-security/integrations that referenced this pull request Mar 30, 2022
@efd6
pulse_connect_secure: add syslog priority format parsing support (ela…
d6ea1c7 
…stic#2552)

* remove event.ingested
* pulse_connect_secure: add syslog priority format parsing support
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@legoguy1000 legoguy1000 legoguy1000 left review comments

@andrewkroh andrewkroh andrewkroh approved these changes

Assignees
No one assigned
Labels
8.1-candidate enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Pulse Secure Connect integration "Provided Grok expressions do not match field value"
4 participants
@efd6 @elasticmachine @andrewkroh @legoguy1000