New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make sure to include charset in getBytes and other relevant code sections #2567

Closed
kroepke opened this Issue Jul 29, 2016 · 3 comments

Comments

Projects
None yet
3 participants
@kroepke
Member

kroepke commented Jul 29, 2016

Problem description

On systems where UTF-8 isn't the default character encoding loading messages with the default character set can fail if they contain non-ASCII characters.

Include the workaound mentioned in Graylog2/graylog-plugin-pipeline-processor#61 (comment) to prevent the wrong encoding to be used.

Steps to reproduce the problem

  1. Use a Russian locale on your server
  2. Use the pipeline simulator to send a cyrillic message
  3. Press simulate
  4. The non-ascii chars are replaced by ? chars

Environment

  • Graylog Version: 2.1.0
  • Elasticsearch Version: n/a
  • MongoDB Version: n/a
  • Operating System: n/a
  • Browser version: n/a

@kroepke kroepke added the improvement label Jul 29, 2016

@kroepke kroepke added this to the 2.1.0 milestone Jul 29, 2016

@joschi

This comment has been minimized.

Contributor

joschi commented Jul 29, 2016

This particular error has been fixed in 2ccef59.

IMHO, Graylog should always work, independent of the configured system locale (which is why we add an explicit encoding hint on all relevant methods, e. g. String.getBytes()).

@bernd bernd changed the title from Add UTF-8 as default file encoding to start scripts to Make sure to include charset in getBytes and other relevant code sections Aug 1, 2016

@bernd

This comment has been minimized.

Member

bernd commented Aug 1, 2016

We are going to fix this in the code.

@kroepke

This comment has been minimized.

Member

kroepke commented Aug 3, 2016

fixed by #2574

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment