Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Shellescaped utf-8 string misbehaving in backticks #3046

Closed
mmustala opened this Issue Jun 15, 2015 · 6 comments

Comments

Projects
None yet
2 participants
@mmustala
Copy link

mmustala commented Jun 15, 2015

Just tried my app with JRuby 9.0.0.0.rc1 and noticed that shellescape is returning utf-8 characters with a couple of backslashes in front of them. And because of this the escaped string does not work with the backticks.

This is almost the same situation as in #2258 but now I'm using shellescape to sanitize the input.

#encoding: utf-8
filename = "neliö" # Any string with multibyte chars should do
File.write(filename, "content")
mime = `file -b --mime #{filename.shellescape}`
# => "ERROR: cannot open `neliö' (No such file or directory)\n"

In the 1.7 JRuby the shellescape was working without issues.

@mmustala

This comment has been minimized.

Copy link
Author

mmustala commented Jun 15, 2015

The shellescape method seems to be the same in 1.7.20.1 and 9.0.0.0.rc1. So it must be the backticks that is behaving differently and not treating the backslashes correctly.

@mmustala mmustala changed the title utf-8 characters misbehaving in shellescape Shellescaped utf-8 string misbehaving in backticks Jun 15, 2015

@headius

This comment has been minimized.

Copy link
Member

headius commented Jun 15, 2015

Our shellwords library is identical to that in MRI, so I think you're right...the problem isn't in shellwords.

However, I was unable to reproduce your issue with my HEAD version of JRuby 9k. What platform are you on? Do you have an unusual system encoding (i.e. non-UTF-8)?

@headius headius added this to the JRuby 9.0.0.0.rc2 milestone Jun 15, 2015

@mmustala

This comment has been minimized.

Copy link
Author

mmustala commented Jun 15, 2015

I just installed jruby-head with rvm and tested that it reproduces. My test file content was:

#encoding: utf-8
require 'shellwords'

filename = "neliö" # Any string with multibyte chars should do
File.write(filename, "content")
mime = `file -b --mime #{filename.shellescape}`
puts mime

This will output

ERROR: cannot open `neliö' (No such file or directory)

My environment is Ubuntu 14.04.

The same test file run with JRuby 1.7.20.1 outputs

text/plain; charset=us-ascii
@headius

This comment has been minimized.

Copy link
Member

headius commented Jun 15, 2015

Can you show me your LANG env var please? I ran on OS X and your script works ok.

@mmustala

This comment has been minimized.

Copy link
Author

mmustala commented Jun 15, 2015

LANG=en_US.UTF-8

@headius

This comment has been minimized.

Copy link
Member

headius commented Jun 15, 2015

Ok, I'll give it a shot on Linux and try to reproduce.

@enebo enebo closed this in c9c2390 Jul 1, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.