New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JRuby 1.7.24 - REXML bad encoding issue #3632

Closed
kimptoc opened this Issue Feb 2, 2016 · 5 comments

Comments

Projects
None yet
3 participants
@kimptoc

kimptoc commented Feb 2, 2016

When we switched from 1.7.23 to 1.7.24, we started getting intermittent (1 out of 10) failures on this line.

xml = REXML::Document.new(req.body)
10:50:58:115|2047-00060:TH21 ERROR: #<ArgumentError: Bad encoding name utf-8>
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/encoding.rb:11:in `encoding='
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/source.rb:55:in `encoding='
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/baseparser.rb:215:in `pull_event'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/baseparser.rb:183:in `pull'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/parsers/treeparser.rb:22:in `parse'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/document.rb:249:in `build'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/rexml/document.rb:43:in `initialize'
C:/rbidev/IonPlatformPostMerge/config/RaboWriter/integration_tests/tests/mock_shareco/mock_shareco_webservice.rb:94:in `handle_bond'
C:/rbidev/IonPlatformPostMerge/config/RaboWriter/integration_tests/tests/mock_shareco/mock_shareco_webservice.rb:20:in `service'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/webrick/httpserver.rb:138:in `service'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/webrick/httpserver.rb:94:in `run'
/C:/Users/kimptonc/.m2/repository/org/jruby/jruby-complete/1.7.24/jruby-complete-1.7.24.jar!/META-INF/jruby.home/lib/ruby/1.9/webrick/server.rb:191:in `start_thread'
...
Bad encoding name utf-8
Line: 1
Position: 40
Last 80 unconsumed characters:

This is the XML sent – largely the same as the other 9 times when it does work ok.

10:50:58:115|0313-00060:TH21 INFO: <?xml version="1.0" encoding="utf-8" ?>
<bond><exchange>186</exchange><isin>XSA3</isin><sequenceNumber>1453978215004</sequenceNumber><ohlc><high>102.03</high><close>102.03</close><recentPrice>102.03</recentPrice><recentPriceDatetime>2016-01-28T12:12:03Z</recentPriceDatetime></ohlc></bond>

Running on Windows with JDK 1.7.0_80-b15. Seems to be ok on Linux - at least we did not see any issues and that is what we run the main app on.

Will see if I can try some tests, eg via git bisect and/or JRuby 9 to see where it starts happening

@kimptoc

This comment has been minimized.

Show comment
Hide comment
@kimptoc

kimptoc Mar 3, 2016

I have setup a small-ish test case here: https://github.com/kimptoc/docker-jruby-webrick-tester

Runs ok with 1.7.23, but fails with 1.7.24.

Now trying to setup the Docker/jruby env for tracking down the problem commit. :)

kimptoc commented Mar 3, 2016

I have setup a small-ish test case here: https://github.com/kimptoc/docker-jruby-webrick-tester

Runs ok with 1.7.23, but fails with 1.7.24.

Now trying to setup the Docker/jruby env for tracking down the problem commit. :)

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Mar 4, 2016

Member

Have you managed to bisect or try JRuby 9k? There's unfortunately not a lot to go on here, but 9k should have significantly better encoding support.

If I were to take a guess, we're not properly looking up the encoding coming in from rexml, and so we don't find "UTF-8" encoding for "utf-8".

Member

headius commented Mar 4, 2016

Have you managed to bisect or try JRuby 9k? There's unfortunately not a lot to go on here, but 9k should have significantly better encoding support.

If I were to take a guess, we're not properly looking up the encoding coming in from rexml, and so we don't find "UTF-8" encoding for "utf-8".

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Mar 4, 2016

Member

I believe this is the same bug as #3670. After poking around the failing code, the only thing remaining that could be failing was the regular expression match /\AUTF-8\z/i == 'utf-8' happening inside rexml/encoding.rb logic for encoding=.

I came up with a reproduction script:

10.times.map {
  Thread.new {
    loop {
      fail unless /\AUTF-8\z/i === "utf-8"
    }
  }
}.map(&:join)

Obviously the expression should match every time, but this fails very quickly on JRuby 1.7.24 and passes on JRuby 9.1.

The fix was by @lopex in f23bffa9acf741fe3faaec89ce286af0465c6af5.

@lopex Do we have any tests for this problem? I may have found a pretty simple one if not.

Member

headius commented Mar 4, 2016

I believe this is the same bug as #3670. After poking around the failing code, the only thing remaining that could be failing was the regular expression match /\AUTF-8\z/i == 'utf-8' happening inside rexml/encoding.rb logic for encoding=.

I came up with a reproduction script:

10.times.map {
  Thread.new {
    loop {
      fail unless /\AUTF-8\z/i === "utf-8"
    }
  }
}.map(&:join)

Obviously the expression should match every time, but this fails very quickly on JRuby 1.7.24 and passes on JRuby 9.1.

The fix was by @lopex in f23bffa9acf741fe3faaec89ce286af0465c6af5.

@lopex Do we have any tests for this problem? I may have found a pretty simple one if not.

@headius headius closed this Mar 4, 2016

@headius headius added this to the JRuby 9.1.0.0 milestone Mar 4, 2016

@kimptoc

This comment has been minimized.

Show comment
Hide comment
@kimptoc

kimptoc Mar 4, 2016

Great - thanks for looking into this.
JRuby 9000 is on my "to try" list :)

kimptoc commented Mar 4, 2016

Great - thanks for looking into this.
JRuby 9000 is on my "to try" list :)

@lopex

This comment has been minimized.

Show comment
Hide comment
@lopex

lopex Mar 6, 2016

Member

@headius No, but also thinking about this interrupt thingie.

Member

lopex commented Mar 6, 2016

@headius No, but also thinking about this interrupt thingie.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment