New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File.expand_path doesn't respect args encoding #3849

Closed
ahorek opened this Issue May 4, 2016 · 3 comments

Comments

Projects
None yet
2 participants
@ahorek
Contributor

ahorek commented May 4, 2016

Environment

jruby -v
jruby 9.1.0.0 (2.3.0) 2016-05-02 a633c63 Java HotSpot(TM) 64-Bit Server VM 25.92-b14 on 1.8.0_92-b14 +jit [mswin32-x86_64]
also doesn't work with jruby 9.0.5.0

java -version
java version "1.8.0_92"
Java(TM) SE Runtime Environment (build 1.8.0_92-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode)

Expected Behavior

Path should be encoded in UTF8

Actual Behavior

C:\jruby-9.1.0.0\bin\devel>..\jruby -S -J-Dfile.encoding=UTF-8 bundle install
UTF-8 .bundle
UTF-8 .bundle
IBM852 C:/jruby-9.1.0.0/bin/devel
IBM852 C:/jruby-9.1.0.0/bin/devel
UTF-8 config
UTF-8 config
UTF-8 C:/jruby-9.1.0.0/bin/devel/.bundle
UTF-8 C:/jruby-9.1.0.0/bin/devel/.bundle
UTF-8 .bundle
UTF-8 .bundle
IBM852 C:/Users/Pavel RosickÔöť┼╗
UTF-8 .bundle
UTF-8 .bundle
IBM852 C:/jruby-9.1.0.0/bin/devel
IBM852 C:/jruby-9.1.0.0/bin/devel
UTF-8 config
UTF-8 config
UTF-8 C:/jruby-9.1.0.0/bin/devel/.bundle
UTF-8 C:/jruby-9.1.0.0/bin/devel/.bundle
UTF-8 .bundle
UTF-8 .bundle
<b>IBM852 C:/Users/Pavel RosickÔöť┼╗</b>
TypeError: no implicit conversion from nil to integer
                        [] at org/jruby/RubyString.java:2959
             chop_basename at C:/jruby-9.1.0.0/lib/ruby/stdlib/pathname.rb:45
                      plus at C:/jruby-9.1.0.0/lib/ruby/stdlib/pathname.rb:371
                         + at C:/jruby-9.1.0.0/lib/ruby/stdlib/pathname.rb:351
                      join at C:/jruby-9.1.0.0/lib/ruby/stdlib/pathname.rb:417
          user_bundle_path at C:/jruby-9.1.0.0/lib/ruby/gems/shared/gems/bundler-1.12.1/lib/bundler.rb:142
        global_config_file at C:/jruby-9.1.0.0/lib/ruby/gems/shared/gems/bundler-1.12.1/lib/bundler/settings.rb:216
                initialize at C:/jruby-9.1.0.0/lib/ruby/gems/shared/gems/bundler-1.12.1/lib/bundler/settings.rb:13
                  settings at C:/jruby-9.1.0.0/lib/ruby/gems/shared/gems/bundler-1.12.1/lib/bundler.rb:198
                    report at C:/jruby-9.1.0.0/lib/ruby/gems/shared/gems/bundler-1.12.1/lib/bundler/env.rb:28
  request_issue_report_for at C:/jruby-9.1.0.0/lib/ruby/gems/shared/gems/bundler-1.12.1/lib/bundler/friendly_errors.rb:74
                 log_error at C:/jruby-9.1.0.0/lib/ruby/gems/shared/gems/bundler-1.12.1/lib/bundler/friendly_errors.rb:40
      with_friendly_errors at C:/jruby-9.1.0.0/lib/ruby/gems/shared/gems/bundler-1.12.1/lib/bundler/friendly_errors.rb:100
                     <top> at C:/jruby-9.1.0.0/lib/ruby/gems/shared/gems/bundler-1.12.1/exe/bundle:19
                      load at org/jruby/RubyKernel.java:962
                     <top> at C:/jruby-9.1.0.0/bin/bundle:22
@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius May 9, 2016

Member

Running this on any Gemfile produces the same result?

It does appear that we're continuing to encode paths from the system as the system's encoding. If you can narrow this down to something specific in pathname.rb it would be a huge help.

We also need to know what the default encoding/locale/character page is for your Windows system.

Member

headius commented May 9, 2016

Running this on any Gemfile produces the same result?

It does appear that we're continuing to encode paths from the system as the system's encoding. If you can narrow this down to something specific in pathname.rb it would be a huge help.

We also need to know what the default encoding/locale/character page is for your Windows system.

@ahorek

This comment has been minimized.

Show comment
Hide comment
@ahorek

ahorek May 10, 2016

Contributor

Running this on any Gemfile produces the same result? yes
what is the default encoding/locale/character page: IBM852

I've found the cause

jruby

irb(main):012:0> File.expand_path('/').encoding
=> #<Encoding:IBM852>
irb(main):013:0> File.expand_path('/'.force_encoding('UTF-8')).encoding
=> #<Encoding:IBM852>
irb(main):014:0>

mri

irb(main):007:0> File.expand_path('/').encoding
=> #<Encoding:IBM852>
irb(main):008:0> File.expand_path('/'.force_encoding('UTF-8')).encoding
=> #<Encoding:UTF-8>

related code https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/RubyFile.java#L786

    @JRubyMethod(name = "expand_path", required = 1, optional = 1, meta = true)
    public static IRubyObject expand_path19(ThreadContext context, IRubyObject recv, IRubyObject[] args) {
        RubyString path = (RubyString) expandPathInternal(context, recv, args, true, false);
        path.force_encoding(context, context.runtime.getEncodingService().getDefaultExternal());

        return path;
    }
Contributor

ahorek commented May 10, 2016

Running this on any Gemfile produces the same result? yes
what is the default encoding/locale/character page: IBM852

I've found the cause

jruby

irb(main):012:0> File.expand_path('/').encoding
=> #<Encoding:IBM852>
irb(main):013:0> File.expand_path('/'.force_encoding('UTF-8')).encoding
=> #<Encoding:IBM852>
irb(main):014:0>

mri

irb(main):007:0> File.expand_path('/').encoding
=> #<Encoding:IBM852>
irb(main):008:0> File.expand_path('/'.force_encoding('UTF-8')).encoding
=> #<Encoding:UTF-8>

related code https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/RubyFile.java#L786

    @JRubyMethod(name = "expand_path", required = 1, optional = 1, meta = true)
    public static IRubyObject expand_path19(ThreadContext context, IRubyObject recv, IRubyObject[] args) {
        RubyString path = (RubyString) expandPathInternal(context, recv, args, true, false);
        path.force_encoding(context, context.runtime.getEncodingService().getDefaultExternal());

        return path;
    }

@ahorek ahorek changed the title from Encoding issue with bundler to File.expand_path doesn't respect args encoding May 10, 2016

headius added a commit that referenced this issue May 10, 2016

Further fixes for File.expand_path encoding.
See #3849 and the PR this is based on #3866.
@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius May 10, 2016

Member

Thanks to @ahorek we have merged fixes and tests to master that should at least incrementally improve our expand_path encoding logic.

Please file any remaining incompatibilities as new issues.

Member

headius commented May 10, 2016

Thanks to @ahorek we have merged fixes and tests to master that should at least incrementally improve our expand_path encoding logic.

Please file any remaining incompatibilities as new issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment