Encoding.find("filesystem") on windows

[ahorek]

Environment

jruby 9.1.0.0 (2.3.0) 2016-05-02 a633c63 Java HotSpot(TM) 64-Bit Server VM 25.92-b14 on 1.8.0_92-b14 +jit [mswin32-x86_64]

Expected Behavior

mri

irb(main):007:0> Encoding.find("filesystem")
=> #<Encoding:Windows-1252>
irb(main):008:0> Encoding.default_external
=> #<Encoding:IBM852>

Actual Behavior

jruby

irb(main):023:0> Encoding.find("filesystem")
=> #<Encoding:IBM852>
irb(main):024:0> Encoding.default_external
=> #<Encoding:IBM852>

it looks like Encoding.find("filesystem") is just an alias to Encoding.default_external

            case FILESYSTEM:
                // This needs to do something different on Windows. See encoding.c,
                // in the enc_set_filesystem_encoding function.
                return runtime.getDefaultExternalEncoding();

https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/runtime/encoding/EncodingService.java#L545

for windows the mri's logic is a little bit different

int Init_enc_set_filesystem_encoding(void)
{
    int idx;
#if defined NO_LOCALE_CHARMAP
# error NO_LOCALE_CHARMAP defined
#elif defined _WIN32 || defined __CYGWIN__
    char cp[SIZEOF_CP_NAME];
    CP_FORMAT(cp, AreFileApisANSI() ? GetACP() : GetOEMCP());
    idx = rb_enc_find_index(cp);
    if (idx < 0) idx = ENCINDEX_ASCII;
#else
    idx = rb_enc_to_index(rb_default_external_encoding());
#endif
    return idx;
}

[headius]

We can probably bind these few methods, but we don't have a compile time for it (and I expect ArgFileApisANSI is a compiler macro). What does MRI produce for filesystem encoding on your system?

@ahorek Does the output of ENV_JAVA['file.encoding'] match what you would expect from the filesystem Encoding in Ruby?

[ahorek]

irb(main):034:0> ENV_JAVA['file.encoding']
=> "Cp1252"

or

irb(main):033:0>  java.nio.charset.Charset.defaultCharset.to_s
=> "windows-1252"

    private static Charset getDefaultCharset() {
        String encoding = System.getProperty("file.encoding", "UTF-8");
        try {
            return Charset.forName(encoding);
        } catch (UnsupportedCharsetException e) {
            return Charset.forName("UTF-8");
        }
    }

it matches, but I don't know how safe that is.

[headius]

Does it match what MRI reports?

[ahorek]

yes

irb(main):007:0> Encoding.find("filesystem")
=> #<Encoding:Windows-1252>

[headius]

@ahorek Seems like a quick fix then would be for us to use the default charset or file.encoding for the filesystem encoding on Windows.

[headius]

@ahorek I have committed a change to try to use the value set for file.encoding as the "filesystem" encoding. Please confirm it resolves your issues.

Could you please add a spec for this to https://github.com/ruby/spec, or otherwise a spec for our regression suite under spec/regression?