Parser does not convert newlines reading \r\n on Windows

[headius]

I've been investigating ruby/json#867 and I think I've narrowed it down to a JRuby bug.

I believe it's an issue in JRuby's parser, whereby we do not convert \r\n to \n. The outputting of \n for newlines in json is correct in the JRuby extension.

Source, and this file has \r\n line terminators:

#!/usr/bin/env ruby
# frozen_string_literal: true

expected = <<~'JSON'.chomp
  {
  <i>"1" :   {
  <i><i>"2" :   3,
  <i><i>"4" :   [<a_nl><i><i><i>5,<a_nl><i><i><i>6<a_nl><i><i>]
  <i>}
  }
JSON
p expected

Contents of the file as an inspected Ruby string (read as binary to avoid newline conversion):

"#!/usr/bin/env ruby\r\n# frozen_string_literal: true\r\n\r\nexpected = <<~'JSON'.chomp\r\n  {\r\n  <i>\"1\" :   {\r\n  <i><i>\"2\" :   3,\r\n  <i><i>\"4\" :   [<a_nl><i><i><i>5,<a_nl><i><i><i>6<a_nl><i><i>]\r\n  <i>}\r\n  }\r\nJSON\r\np expected\r\n"

Output with Ruby 3.4.7 on Windows:

C:\Users\headius\work\json>ruby test\json\json_generator_test.rb
"{\n<i>\"1\" :   {\n<i><i>\"2\" :   3,\n<i><i>\"4\" :   [<a_nl><i><i><i>5,<a_nl><i><i><i>6<a_nl><i><i>]\n<i>}\n}"

Output with JRuby 10-head on Windows:

C:\Users\headius\work\json>jruby test\json\json_generator_test.rb
"{\r\n<i>\"1\" :   {\r\n<i><i>\"2\" :   3,\r\n<i><i>\"4\" :   [<a_nl><i><i><i>5,<a_nl><i><i><i>6<a_nl><i><i>]\r\n<i>}\r\n}"

Interestingly, on macos JRuby will not even properly run the file with \r\n line terminators in it.

[headius]

This is likely due to the parser using standard IO operations to read files, and those operations not properly handling newline conversion:

>jruby -e "File.readlines('test\json\json_generator_test.rb').each {|line| p line}"
"#!/usr/bin/env ruby\r\n"
"# frozen_string_literal: true\r\n"
"\r\n"
"expected = <<~'JSON'.chomp\r\n"
"  {\r\n"
"  <i>\"1\" :   {\r\n"
"  <i><i>\"2\" :   3,\r\n"
"  <i><i>\"4\" :   [<a_nl><i><i><i>5,<a_nl><i><i><i>6<a_nl><i><i>]\r\n"
"  <i>}\r\n"
"  }\r\n"
"JSON\r\n"
"p expected\r\n"

These line terminators should be \n (and are on CRuby).