Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File Structure Finder should handle whitespace better #51167

Open
benwtrent opened this issue Jan 17, 2020 · 1 comment
Open

File Structure Finder should handle whitespace better #51167

benwtrent opened this issue Jan 17, 2020 · 1 comment
Labels
:ml Machine learning

Comments

@benwtrent
Copy link
Member

Log lines like the following:

[2020-01-17T09:25:35,792][DEBUG][o.e.a.a.c.n.t.c.TransportCancelTasksAction] [Benjamins-MacBook-Pro.local] Removing ban for the parent [edx7HRvUTr6_4AuIUlziIQ:5242648] on the node [edx7HRvUTr6_4AuIUlziIQ]
[2020-01-17T10:17:47,664][INFO ][o.e.n.Node               ] [Benjamins-MacBook-Pro.local] stopping ...

Are automatically Grok'd as the pattern

\[%{TIMESTAMP_ISO8601:timestamp}\]\[%{LOGLEVEL:loglevel}.*

I think that a better pattern might have been discovered if the grok pattern discovery accounted for trailing whitespaces before and after grok patterns.

Something like

\[%{TIMESTAMP_ISO8601:timestamp}\]\[%{LOGLEVEL:loglevel}\s*\]\[.*

It would be even better if DATA grok patterns could be used, but those are pretty general and might could only be used if there are some closing brackets (like in the standard ES logs)

I personally would love the pattern to ultimately result in something like

\[%{TIMESTAMP_ISO8601:timestamp}\]\[%{LOGLEVEL:loglevel}\s*\]\[%{JAVACLASS:class1}\s*\][%{DATA:data1}\].*"
@benwtrent benwtrent added the :ml Machine learning label Jan 17, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/ml-core (:ml)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:ml Machine learning
Projects
None yet
Development

No branches or pull requests

2 participants