Skip to content

Loading…

UnmarshalStream: Fixed two off-by-one errors in unmarshalInt #1338

Merged
merged 1 commit into from

2 participants

@DavidEGrayson

While working on fixing issue #1329, I discovered something tangentially related that I think is a tiny bug and can be easily fixed. MRI's Marshal.load recognizes three different ways of encoding the integer 0 in a single byte, but JRuby 1.7.9 only handles one and raises the "marshal data too short" exception for the other two. This is because there are two off-by-one errors in JRuby's unmarshalInt method.

I think that unmarshalInt is supposed to be a clone of MRI's r_long function, so here is the source code of r_long:
https://github.com/ruby/ruby/blob/v2_0_0_352/marshal.c#L1114-1145

Here is the code I used to demonstrate the difference between MRI 2.0.0p0 and JRuby:

# MRI returns 0 for each, JRuby says marshal data is too short
puts Marshal.load("\x04\x08i\x05")
puts Marshal.load("\x04\x08i\xFB")

I suppose this isn't too important, because the number 0 will probably be marshaled as the 0x00 byte instead of 0xFB or 0x05, but if MRI treats those bytes as representing 0 then I don't think it is helpful for JRuby to raise an exception (or to try to construct an integer by reading 5 bytes from the stream).

I am happy to do any more investigating or documentation that is needed. Should I write a test for this? If so, where should I put it?

@DavidEGrayson DavidEGrayson UnmarshalStream: Fixed two off-by-one errors in unmarshalInt; it is n…
…ow consistent with r_long in marshal.c of MRI. This also means there are three different ways to marshal the number 0.
36fdcfb
@DavidEGrayson

The Travis CI build seems to have failed because it had trouble connecting to rubyforge.org port 443 and I don't think my change would have caused that.

@enebo enebo merged commit cd01094 into jruby:jruby-1_7

1 check failed

Details default The Travis CI build failed
@DavidEGrayson DavidEGrayson deleted the DavidEGrayson:jruby_pull_request_1 branch
@enebo enebo modified the milestone: JRuby 1.7.10, JRuby 1.7.11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Dec 15, 2013
  1. @DavidEGrayson

    UnmarshalStream: Fixed two off-by-one errors in unmarshalInt; it is n…

    DavidEGrayson committed
    …ow consistent with r_long in marshal.c of MRI. This also means there are three different ways to marshal the number 0.
Showing with 2 additions and 2 deletions.
  1. +2 −2 core/src/main/java/org/jruby/runtime/marshal/UnmarshalStream.java
View
4 core/src/main/java/org/jruby/runtime/marshal/UnmarshalStream.java
@@ -323,9 +323,9 @@ public int unmarshalInt() throws IOException {
int c = readSignedByte();
if (c == 0) {
return 0;
- } else if (5 < c && c < 128) {
+ } else if (4 < c && c < 128) {
return c - 5;
- } else if (-129 < c && c < -5) {
+ } else if (-129 < c && c < -4) {
return c + 5;
}
long result;
Something went wrong with that request. Please try again.