Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure to include charset in getBytes and other relevant code sections #2567

Closed
kroepke opened this issue Jul 29, 2016 · 3 comments
Closed

Make sure to include charset in getBytes and other relevant code sections #2567

kroepke opened this issue Jul 29, 2016 · 3 comments
Labels
Milestone

Comments

@kroepke
Copy link
Member

@kroepke kroepke commented Jul 29, 2016

Problem description

On systems where UTF-8 isn't the default character encoding loading messages with the default character set can fail if they contain non-ASCII characters.

Include the workaound mentioned in Graylog2/graylog-plugin-pipeline-processor#61 (comment) to prevent the wrong encoding to be used.

Steps to reproduce the problem

  1. Use a Russian locale on your server
  2. Use the pipeline simulator to send a cyrillic message
  3. Press simulate
  4. The non-ascii chars are replaced by ? chars

Environment

  • Graylog Version: 2.1.0
  • Elasticsearch Version: n/a
  • MongoDB Version: n/a
  • Operating System: n/a
  • Browser version: n/a
@kroepke kroepke added the improvement label Jul 29, 2016
@kroepke kroepke added this to the 2.1.0 milestone Jul 29, 2016
@joschi
Copy link
Contributor

@joschi joschi commented Jul 29, 2016

This particular error has been fixed in 2ccef59.

IMHO, Graylog should always work, independent of the configured system locale (which is why we add an explicit encoding hint on all relevant methods, e. g. String.getBytes()).

@bernd bernd changed the title Add UTF-8 as default file encoding to start scripts Make sure to include charset in getBytes and other relevant code sections Aug 1, 2016
@bernd
Copy link
Member

@bernd bernd commented Aug 1, 2016

We are going to fix this in the code.

@kroepke
Copy link
Member Author

@kroepke kroepke commented Aug 3, 2016

fixed by #2574

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants