Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot create GROK pattern with underscores in field name. #5563

Closed
ddimick opened this issue Jan 21, 2019 · 4 comments
Closed

Cannot create GROK pattern with underscores in field name. #5563

ddimick opened this issue Jan 21, 2019 · 4 comments
Assignees

Comments

@ddimick
Copy link

@ddimick ddimick commented Jan 21, 2019

Expected Behavior

Graylog should accept underscores in field name and create the grok pattern.

Current Behavior

Graylog returns "Could not test Grok pattern
Testing Grok pattern "test" failed with status: cannot POST https://graylog.my.domain/api/system/grok (400)"

Steps to Reproduce (for bugs)

Create a grok pattern like (?<test_field>test), it will fail. Remove the underscore and create (?<testfield>test), it will succeed.

Your Environment

  • Graylog Version: 3.0.0-beta.3-1
  • Elasticsearch Version: 6.5.4
  • MongoDB Version: 4.0.5
  • Operating System: Server CoreOS 4.14.88, docker 18.06.1-ce, client Windows 10 Pro
  • Browser version: Chrome 71.0.3578.98, Firefox 62.0.3 and Edge 42.17134.1.0
@kmerz
Copy link
Member

@kmerz kmerz commented Jan 22, 2019

Unfortunately this is not fixable. The java Pattern matcher does not allow to have underscores in its named groups:

https://docs.oracle.com/javase/7/docs/api/java/util/regex/Pattern.html#groupname

A capturing group can also be assigned a "name", a named-capturing group, and then be back-
referenced later by the "name". Group names are composed of the following characters. The first
character must be a letter.

The uppercase letters 'A' through 'Z' ('\u0041' through '\u005a'),
The lowercase letters 'a' through 'z' ('\u0061' through '\u007a'),
The digits '0' through '9' ('\u0030' through '\u0039'),

The only alternative would be to implement our own Pattern Matcher which would be slightly out of scope.

@kmerz kmerz closed this Jan 22, 2019
bernd added a commit that referenced this issue Mar 25, 2019
To support underscores ("_") in Grok match group names, we had to modify
the java-grok library to use the old regexp engine again.

See: graylog-labs/java-grok#2

This also adds a test for the Grok extractor to make sure that using
underscores works.

Fixes #5704
Fixes #5563
kmerz added a commit that referenced this issue Mar 26, 2019
* Switch back to a repackaged and fixed version of java-grok

To support underscores ("_") in Grok match group names, we had to modify
the java-grok library to use the old regexp engine again.

See: graylog-labs/java-grok#2

This also adds a test for the Grok extractor to make sure that using
underscores works.

Fixes #5704
Fixes #5563

* Fix GrokPatternService#extractPatternNames and add a test for it

* Add missing license header to GrokPatternServiceTest

* Add test for named group with underscore

Prior to this change, there was no test for named groups
with underscores in the FunctionSnippetsTest

This change enhances the grok() test to run with a
named group with underscore.
@bernd
Copy link
Member

@bernd bernd commented Mar 26, 2019

@ddimick We fixed this issue in master and it will be backported into the upcoming 3.0.1. release.

@bernd bernd self-assigned this Mar 26, 2019
bernd added a commit that referenced this issue Mar 26, 2019
* Switch back to a repackaged and fixed version of java-grok

To support underscores ("_") in Grok match group names, we had to modify
the java-grok library to use the old regexp engine again.

See: graylog-labs/java-grok#2

This also adds a test for the Grok extractor to make sure that using
underscores works.

Fixes #5704
Fixes #5563

* Fix GrokPatternService#extractPatternNames and add a test for it

* Add missing license header to GrokPatternServiceTest

* Add test for named group with underscore

Prior to this change, there was no test for named groups
with underscores in the FunctionSnippetsTest

This change enhances the grok() test to run with a
named group with underscore.

(cherry picked from commit e642a41)
kmerz added a commit that referenced this issue Mar 26, 2019
…5807)

* Switch back to a repackaged and fixed version of java-grok

To support underscores ("_") in Grok match group names, we had to modify
the java-grok library to use the old regexp engine again.

See: graylog-labs/java-grok#2

This also adds a test for the Grok extractor to make sure that using
underscores works.

Fixes #5704
Fixes #5563

* Fix GrokPatternService#extractPatternNames and add a test for it

* Add missing license header to GrokPatternServiceTest

* Add test for named group with underscore

Prior to this change, there was no test for named groups
with underscores in the FunctionSnippetsTest

This change enhances the grok() test to run with a
named group with underscore.

(cherry picked from commit e642a41)
@ayoublab92
Copy link

@ayoublab92 ayoublab92 commented May 8, 2019

hello everyone, please i need hepl i have the same problem with Graylog 2.5 I am blocked for creating GROK PATTERN , Graylog returns " Saving Grok pattern “name of grok pattern ” failed with status cannot POST http://@ip of graylog/api/system/grok (400)"

@kmerz
Copy link
Member

@kmerz kmerz commented May 13, 2019

@ayoublab92 github is meant for reporting bugs and discussing them. The reported bug is solved and backported to 3.0.1. Please upgrade your graylog to the fixed version. Try https//community.graylog.com to find help otherwise. This is not a forum.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants