JRuby 1.7.24 - REXML bad encoding issue

[kimptoc]

When we switched from 1.7.23 to 1.7.24, we started getting intermittent (1 out of 10) failures on this line.

xml = REXML::Document.new(req.body)

10:50:58:115|2047-00060:TH21 ERROR: #<ArgumentError: Bad encoding name utf-8>
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/encoding.rb:11:in `encoding='
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/source.rb:55:in `encoding='
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/baseparser.rb:215:in `pull_event'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/baseparser.rb:183:in `pull'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/treeparser.rb:22:in `parse'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/document.rb:249:in `build'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/document.rb:43:in `initialize'
C:/rbidev/IonPlatformPostMerge/config/RaboWriter/integration_tests/tests/mock_shareco/mock_shareco_webservice.rb:94:in `handle_bond'
C:/rbidev/IonPlatformPostMerge/config/RaboWriter/integration_tests/tests/mock_shareco/mock_shareco_webservice.rb:20:in `service'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/webrick/httpserver.rb:138:in `service'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/webrick/httpserver.rb:94:in `run'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/webrick/server.rb:191:in `start_thread'
...
Bad encoding name utf-8
Line: 1
Position: 40
Last 80 unconsumed characters:

This is the XML sent – largely the same as the other 9 times when it does work ok.

10:50:58:115|0313-00060:TH21 INFO: <?xml version="1.0" encoding="utf-8" ?>
<bond><exchange>186</exchange><isin>XSA3</isin><sequenceNumber>1453978215004</sequenceNumber><ohlc><high>102.03</high><close>102.03</close><recentPrice>102.03</recentPrice><recentPriceDatetime>2016-01-28T12:12:03Z</recentPriceDatetime></ohlc></bond>

Running on Windows with JDK 1.7.0_80-b15. Seems to be ok on Linux - at least we did not see any issues and that is what we run the main app on.

Will see if I can try some tests, eg via git bisect and/or JRuby 9 to see where it starts happening

[kimptoc]

I have setup a small-ish test case here: https://github.com/kimptoc/docker-jruby-webrick-tester

Runs ok with 1.7.23, but fails with 1.7.24.

Now trying to setup the Docker/jruby env for tracking down the problem commit. :)

[headius]

Have you managed to bisect or try JRuby 9k? There's unfortunately not a lot to go on here, but 9k should have significantly better encoding support.

If I were to take a guess, we're not properly looking up the encoding coming in from rexml, and so we don't find "UTF-8" encoding for "utf-8".

[headius]

I believe this is the same bug as #3670. After poking around the failing code, the only thing remaining that could be failing was the regular expression match /\AUTF-8\z/i == 'utf-8' happening inside rexml/encoding.rb logic for encoding=.

I came up with a reproduction script:

10.times.map {
  Thread.new {
    loop {
      fail unless /\AUTF-8\z/i === "utf-8"
    }
  }
}.map(&:join)

Obviously the expression should match every time, but this fails very quickly on JRuby 1.7.24 and passes on JRuby 9.1.

The fix was by @lopex in f23bffa9acf741fe3faaec89ce286af0465c6af5.

@lopex Do we have any tests for this problem? I may have found a pretty simple one if not.

[kimptoc]

Great - thanks for looking into this.
JRuby 9000 is on my "to try" list :)

[lopex]

@headius No, but also thinking about this interrupt thingie.