Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HAProxy] Fix the Grok pattern for parsing TCP logs #8255

Merged
merged 8 commits into from Oct 26, 2023

Conversation

devamanv
Copy link
Contributor

@devamanv devamanv commented Oct 19, 2023

Proposed commit message

The PR contains changes to fix the Grok Pattern for parsing the HAProxy TCP logs. Currently, the Grok patterns in the ingest pipelines don't conform to the official TCP log format. The change is in the existing Grok pattern to add some optional fields, as well some sample raw logs for the ingest pipeline tests.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • Grok pattern is able to parse the TCP log format
  • Pipeline tests pass

How to test this PR locally

  • Clone the integrations repository
  • Install elastic-package locally
  • Spin up an elastic stack using elastic-package
  • Run elastic-package test pipeline -v from the integrations/packages/haproxy directory

Related issues

@elasticmachine
Copy link

elasticmachine commented Oct 19, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-10-26T12:43:41.656+0000

  • Duration: 16 min 52 sec

Test stats 🧪

Test Results
Failed 0
Passed 37
Skipped 0
Total 37

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

elasticmachine commented Oct 19, 2023

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (1/1) 💚
Files 100.0% (1/1) 💚 3.837
Classes 100.0% (1/1) 💚 3.837
Methods 90.476% (19/21) 👎 -1.636
Lines 97.238% (176/181) 👍 8.644
Conditionals 100.0% (0/0) 💚

@devamanv devamanv changed the title [HAProxy] Fix Grok pattern for parsing TCP logs [HAProxy] Add missing Grok pattern for parsing TCP logs Oct 20, 2023
@devamanv devamanv marked this pull request as ready for review October 20, 2023 19:30
@devamanv devamanv requested a review from a team as a code owner October 20, 2023 19:30
@devamanv devamanv changed the title [HAProxy] Add missing Grok pattern for parsing TCP logs [HAProxy] Fix the Grok pattern for parsing TCP logs Oct 20, 2023
@ishleenk17
Copy link
Contributor

haproxy log format is only TCP? Or is it one of them?

@devamanv
Copy link
Contributor Author

devamanv commented Oct 23, 2023

@ishleenk17 HAProxy supports multiple log formats, TCP is just one of them.

@aliabbas-elastic
Copy link
Contributor

Testing Details

Test Case Scenarios

Test Case Summaries Status
Verify by ingesting a log matching the TCP log format according to official documentation Pass 💚
Check that the TCP logs are getting ingested when the optional fields are not present Pass 💚
Verify that all the fields are getting mapped with their respective fields with correct type of the fields Pass 💚
Verify that there are no conflicting fields after upgrading the package and ingesting TCP logs Pass 💚
Verify that the visualizations are loading properly under the "[Logs HAProxy] Overview" dashboard after ingesting TCP Logs Pass 💚

Copy link

@HGS9761 HGS9761 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is cristal clear!

@devamanv devamanv requested a review from a team October 26, 2023 11:37
Copy link
Contributor

@ishleenk17 ishleenk17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@devamanv devamanv merged commit 4a57b22 into elastic:main Oct 26, 2023
4 checks passed
@elasticmachine
Copy link

Package haproxy - 1.10.1 containing this change is available at https://epr.elastic.co/search?package=haproxy

@HGS9761
Copy link

HGS9761 commented Oct 27, 2023

It is still not OK!
It still does not match the missing format

I suggested a new GROK pattern

#6605

@devamanv devamanv deleted the haproxy-grok-fix branch October 27, 2023 13:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix TCP Logs Grok pattern for HAProxy Integration package
5 participants