Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

US-ASCII error on default grok-pattern file #32

Closed
Kuckkuck opened this issue Jul 1, 2017 · 5 comments
Closed

US-ASCII error on default grok-pattern file #32

Kuckkuck opened this issue Jul 1, 2017 · 5 comments

Comments

@Kuckkuck
Copy link

Kuckkuck commented Jul 1, 2017

Hi,

since fluent version 0.14.17/0.14.18 with the fluent-plugin-grok-parser plugin 2.3.1, which is reading the default grok-patterns file, I get a /var/lib/gems/2.3.0/gems/fluent-plugin-grok-parser-2.1.3/lib/fluent/plugin/grok.rb:39:in `block in add_patterns_from_file': invalid byte sequence in US-ASCII (ArgumentError).
The default grok pattern file contains month where the german character "ä" ( May/März) is not us-ascii compatible. This was working with 0.14.16 with the same plugin version for fluent-plugin-grok-parser 2.3.1., but now it is crashing. I am using an Ubuntu container, where I have just changed the version number for fluentd and now it is crashing with: block in add_patterns_from_file': invalid byte sequence in US-ASCII (ArgumentError).

2017-06-30 09:57:27 +0000 [info]: reading config file path="/etc/fluent/fluent.conf"
/var/lib/gems/2.3.0/gems/fluent-plugin-grok-parser-2.1.3/lib/fluent/plugin/grok.rb:39:in `block in add_patterns_from_file': invalid byte sequence in US-ASCII (ArgumentError)
from /var/lib/gems/2.3.0/gems/fluent-plugin-grok-parser-2.1.3/lib/fluent/plugin/grok.rb:38:in each_line'
from /var/lib/gems/2.3.0/gems/fluent-plugin-grok-parser-2.1.3/lib/fluent/plugin/grok.rb:38:in add_patterns_from_file'
from /var/lib/gems/2.3.0/gems/fluent-plugin-grok-parser-2.1.3/lib/fluent/plugin/parser_grok.rb:30:in block in configure'
from /var/lib/gems/2.3.0/gems/fluent-plugin-grok-parser-2.1.3/lib/fluent/plugin/parser_grok.rb:29:in glob'
from /var/lib/gems/2.3.0/gems/fluent-plugin-grok-parser-2.1.3/lib/fluent/plugin/parser_grok.rb:29:in configure'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/plugin.rb:164:in configure'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/plugin_helper/parser.rb:90:in block in configure'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/plugin_helper/parser.rb:85:in each'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/plugin_helper/parser.rb:85:in configure'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/plugin/filter_parser.rb:42:in configure'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/plugin.rb:164:in configure'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/agent.rb:152:in add_filter'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/agent.rb:70:in block in configure'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/agent.rb:64:in each'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/agent.rb:64:in configure'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/root_agent.rb:112:in configure'
from /var/lib/gems/2.3.0/gems/fluentd-0.14.18/lib/fluent/engine.rb:131:in configure'

Any idea, what was changed?

Regards,

Olaf
@okkez
Copy link
Collaborator

okkez commented Jul 3, 2017

Same as #27 ???

@okkez
Copy link
Collaborator

okkez commented Jul 3, 2017

I will check this issue later.

@okkez
Copy link
Collaborator

okkez commented Jul 3, 2017

Related to fluent/fluentd#1606 ?

@okkez okkez closed this as completed in 49a2aa3 Jul 3, 2017
@okkez
Copy link
Collaborator

okkez commented Jul 3, 2017

Released 2.1.4

@Kuckkuck
Copy link
Author

Kuckkuck commented Jul 3, 2017

great, thanks it worked.

openstack-gerrit pushed a commit to openstack/kolla that referenced this issue Nov 16, 2017
@chenxingc
Fix the Ubuntu Fluentd image
cc646d1 
For ubuntu, we actually use fluentd 0.14.22 and ruby 2.1.10 now.
Thus, we should use the specific gem version for the plugins:

 - fluent-plugin-rewrite-tag-filter:2.0.0
According to the README.md of this plugin[1]
It shows:
----------------------------------------------------------
| fluent-plugin-rewrite-tag-filter | Fluentd    | Ruby   |
|----------------------------------|------------|--------|
| >= 2.0.0                         | >= v0.14.2 | >= 2.1 |
| < 2.0.0                          | >= v0.12.0 | >= 1.9 |
----------------------------------------------------------

 - fluent-plugin-grok-parser:2.1.4
When using the fluent-plugin-rewrite-tag-filter 2.0.0 to fix this bug, another
bug[1] which is the same as this one[2] was triggered.
According to https://github.com/fluent/fluent-plugin-grok-parser, it shows:
---------------------------------------------------
| fluent-plugin-grok-parser | fluentd    | Ruby   |
|---------------------------|------------|--------|
| >= 1.0.0                  | >= v0.14.0 | >= 2.1 |
| < 1.0.0                   | >= v0.12.0 | >= 1.9 |
---------------------------------------------------

We actually use fluentd 0.14.22 now, but the version of fluent-plugin-grok-parser
is still pinned in 0.3.1. So we need to upgrade the fluent-plugin-grok-parser to
2.1.4.

[1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter
[2] http://paste.openstack.org/show/626145/
[3] fluent/fluent-plugin-grok-parser#32

backport: pike

Change-Id: I47985113fe732569d640a262ca832c6edc8c2bb6
Partial-Bug: #1730664
openstack-gerrit pushed a commit to openstack/kolla that referenced this issue Nov 22, 2017
@chenxingc
Fix the Ubuntu Fluentd image
ed669e2 
For ubuntu, we actually use fluentd 0.14.22 and ruby 2.1.10 now.
Thus, we should use the specific gem version for the plugins:

 - fluent-plugin-rewrite-tag-filter:2.0.0
According to the README.md of this plugin[1]
It shows:
----------------------------------------------------------
| fluent-plugin-rewrite-tag-filter | Fluentd    | Ruby   |
|----------------------------------|------------|--------|
| >= 2.0.0                         | >= v0.14.2 | >= 2.1 |
| < 2.0.0                          | >= v0.12.0 | >= 1.9 |
----------------------------------------------------------

 - fluent-plugin-grok-parser:2.1.4
When using the fluent-plugin-rewrite-tag-filter 2.0.0 to fix this bug, another
bug[1] which is the same as this one[2] was triggered.
According to https://github.com/fluent/fluent-plugin-grok-parser, it shows:
---------------------------------------------------
| fluent-plugin-grok-parser | fluentd    | Ruby   |
|---------------------------|------------|--------|
| >= 1.0.0                  | >= v0.14.0 | >= 2.1 |
| < 1.0.0                   | >= v0.12.0 | >= 1.9 |
---------------------------------------------------

We actually use fluentd 0.14.22 now, but the version of fluent-plugin-grok-parser
is still pinned in 0.3.1. So we need to upgrade the fluent-plugin-grok-parser to
2.1.4.

[1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter
[2] http://paste.openstack.org/show/626145/
[3] fluent/fluent-plugin-grok-parser#32

Change-Id: I47985113fe732569d640a262ca832c6edc8c2bb6
Partial-Bug: #1730664
yankcrime pushed a commit to stackhpc/kolla that referenced this issue Feb 23, 2018
@chenxingc @yankcrime
Fix the Ubuntu Fluentd image
1973f22 
For ubuntu, we actually use fluentd 0.14.22 and ruby 2.1.10 now.
Thus, we should use the specific gem version for the plugins:

 - fluent-plugin-rewrite-tag-filter:2.0.0
According to the README.md of this plugin[1]
It shows:
----------------------------------------------------------
| fluent-plugin-rewrite-tag-filter | Fluentd    | Ruby   |
|----------------------------------|------------|--------|
| >= 2.0.0                         | >= v0.14.2 | >= 2.1 |
| < 2.0.0                          | >= v0.12.0 | >= 1.9 |
----------------------------------------------------------

 - fluent-plugin-grok-parser:2.1.4
When using the fluent-plugin-rewrite-tag-filter 2.0.0 to fix this bug, another
bug[1] which is the same as this one[2] was triggered.
According to https://github.com/fluent/fluent-plugin-grok-parser, it shows:
---------------------------------------------------
| fluent-plugin-grok-parser | fluentd    | Ruby   |
|---------------------------|------------|--------|
| >= 1.0.0                  | >= v0.14.0 | >= 2.1 |
| < 1.0.0                   | >= v0.12.0 | >= 1.9 |
---------------------------------------------------

We actually use fluentd 0.14.22 now, but the version of fluent-plugin-grok-parser
is still pinned in 0.3.1. So we need to upgrade the fluent-plugin-grok-parser to
2.1.4.

[1] https://github.com/fluent/fluent-plugin-rewrite-tag-filter
[2] http://paste.openstack.org/show/626145/
[3] fluent/fluent-plugin-grok-parser#32

backport: pike

Change-Id: I47985113fe732569d640a262ca832c6edc8c2bb6
Partial-Bug: #1730664
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@okkez @Kuckkuck