Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Zlib crc32 resulting into Java::JavaLang::ArrayIndexOutOfBoundsException (32) #5290

Closed
vipulnsward opened this Issue Aug 26, 2018 · 7 comments

Comments

Projects
None yet
4 participants
@vipulnsward
Copy link
Contributor

vipulnsward commented Aug 26, 2018

After https://github.com/jruby/jruby/pull/5088/files was merged:

irb(main):027:0> n =  "p" ; key = ~305419896
=> -305419897
irb(main):028:0> Zlib.crc32(n, key)
Traceback (most recent call last):
       16: from org.jruby.RubyKernel.eval19(RubyKernel.java:1037)
       15: from org.jruby.RubyKernel.evalCommon(RubyKernel.java:1070)
       14: from org.jruby.ir.interpreter.Interpreter.evalWithBinding(Interpreter.java:197)
       13: from org.jruby.ir.interpreter.Interpreter.evalCommon(Interpreter.java:173)
       12: from org.jruby.ir.interpreter.Interpreter.INTERPRET_EVAL(Interpreter.java:119)
       11: from org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
       10: from org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:324)
        9: from org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:201)
        8: from org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:365)
        7: from org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:210)
        6: from org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:801)
        5: from org.jruby.ext.zlib.RubyZlib$INVOKER$s$0$2$crc32.call(RubyZlib$INVOKER$s$0$2$crc32.gen)
        4: from org.jruby.ext.zlib.RubyZlib.crc32(RubyZlib.java:206)
        3: from com.jcraft.jzlib.JZlib.crc32_combine(JZlib.java:90)
        2: from com.jcraft.jzlib.CRC32.combine(CRC32.java:108)
        1: from com.jcraft.jzlib.CRC32.gf2_matrix_times(CRC32.java:134)
Java::JavaLang::ArrayIndexOutOfBoundsException (32)

This passes successfully on ruby 2.5.1 as:

irb(main):010:0> n =  "p" ; key = ~305419896
=> -305419897
irb(main):011:0> Zlib.crc32(n, key)
=> 4046865307

Related on why ruby seems to allows this: ruby/ruby@778796b

@prashantvithani

This comment has been minimized.

Copy link
Contributor

prashantvithani commented Aug 28, 2018

@vipulnsward I'm facing same issue. Have you found any workaround?

@enebo enebo added this to the JRuby 9.2.1.0 milestone Aug 28, 2018

@headius

This comment has been minimized.

Copy link
Member

headius commented Aug 28, 2018

Huh...is this our bug or a bug in jzlib?

@vipulnsward

This comment has been minimized.

Copy link
Contributor Author

vipulnsward commented Aug 28, 2018

Its just deviation from functionality in jruby vs ruby.

ruby/ruby@778796b takes care of adjusting byte width in MRI.

@headius

This comment has been minimized.

Copy link
Member

headius commented Aug 28, 2018

I'm having a little trouble figuring out the right way to address this.

The fix in MRI seems to try to force longs into UINT, because that's what the zlib functions accept. Am I following correctly?

We use a pure-Java zlib port called jzlib, which always uses 64-bit signed longs for this parameter. So it seems like the only thing we need to do is to normalize negative long values passed in to their positive equivalents, rather than doing this extra loop to handle UINT-sized chunks. But I'm not sure if this is going to produce the equivalent values as MRI.

@headius

This comment has been minimized.

Copy link
Member

headius commented Aug 28, 2018

Ping @ymnk for input.

@prashantvithani

This comment has been minimized.

Copy link
Contributor

prashantvithani commented Oct 8, 2018

This error will occur only for negative long numbers. CRC32 requires the initial value (say crc1) to be of 32 bits only and expects it to become 0 after crc1 >> 32.

https://github.com/ymnk/jzlib/blob/a21be20213d66eff15904d925e9b721956a01ef7/src/main/java/com/jcraft/jzlib/CRC32.java#L129-L139

The size of lookup table mat is fixed at 32. The return value of RubyNumeric.num2long(-305419897) is equal to 0xFFFFFFFFEDCBA987, 64-bit number. As it doesn't resolve to 0 after 0xFFFFFFFFEDCBA987 >> 32 (i.e. 32 times (>>1) in loop), the loop goes on for the index 32, which is not expected.

The immediate fix for this would be to trim the 64-bit hex to 32-bit using start & 0xFFFFFFFFL (i.e. 0xEDCBA987). So it becomes 0 after 32 loops.

prashantvithani added a commit to adwyze/jruby that referenced this issue Oct 8, 2018

Trim 'start' value to 32-bit for negative long integers
As discussed in jruby#5290 (comment), The while loop in gf2_matrix_times should terminate after 32 iterations. That means the value of 'start' should be 0 after (start>>32). Trimming the value beyond 32 bits ensures that behavior.
@prashantvithani

This comment has been minimized.

Copy link
Contributor

prashantvithani commented Oct 8, 2018

@headius: The solution in #5350 seems to produce the equivalent value as in MRI for the example quoted by @vipulnsward.

@headius headius closed this in #5350 Oct 9, 2018

eregon added a commit to ruby/spec that referenced this issue Oct 27, 2018

Trim 'start' value to 32-bit for negative long integers
As discussed in jruby/jruby#5290 (comment), The while loop in gf2_matrix_times should terminate after 32 iterations. That means the value of 'start' should be 0 after (start>>32). Trimming the value beyond 32 bits ensures that behavior.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.