Incompatible error message around marshal data that is "too short"

[rwstauner]

I found a test in the dalli test suite that tests against 2 specific error messages one might get from Marshal.load: undefined class and marshal data too short

https://github.com/petergoldstein/dalli/blob/43baad2e890af746ac67fadc3a69fa47367b1af6/test/protocol/test_value_serializer.rb#L282-L309
https://github.com/petergoldstein/dalli/blob/43baad2e890af746ac67fadc3a69fa47367b1af6/lib/dalli/protocol/value_serializer.rb#L41

For reference, the marshal dump of an empty object is Marshal.dump(Object.new) => "\x04\bo:\vObject\x00"

$ ruby -ve 'Marshal.load("\x04\bo:\vObj")'
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]
<internal:marshal>:34:in `load': marshal data too short (ArgumentError)
        from -e:1:in `<main>'

vs

$ truffleruby -ve 'Marshal.load("\x04\bo:\vObj")'
truffleruby 23.1.0-dev-e539565e, like ruby 3.1.3, GraalVM CE JVM [aarch64-darwin]
<internal:core> core/marshal.rb:598:in `block in const_lookup': undefined class/module Obj (ArgumentError)
        from <internal:core> core/marshal.rb:591:in `each'
        from <internal:core> core/marshal.rb:591:in `const_lookup'
        from <internal:core> core/marshal.rb:1333:in `construct_object'
        from <internal:core> core/marshal.rb:676:in `construct'
        from <internal:core> core/marshal.rb:1482:in `load'
        from -e:1:in `<main>'
<internal:core> core/marshal.rb:596:in `const_missing': uninitialized constant Obj (NameError)
        from <internal:core> core/marshal.rb:596:in `block in const_lookup'
        from <internal:core> core/marshal.rb:591:in `each'
        from <internal:core> core/marshal.rb:591:in `const_lookup'
        from <internal:core> core/marshal.rb:1333:in `construct_object'
        from <internal:core> core/marshal.rb:676:in `construct'
        from <internal:core> core/marshal.rb:1482:in `load'
        from -e:1:in `<main>'

I see this error message:

• in CRuby: https://github.com/ruby/ruby/blob/381475f02e6b44ae729f9403637b30c445b622e5/marshal.c#L1220-L1226
• in TruffleRuby:

  truffleruby/src/main/ruby/truffleruby/core/marshal.rb

  Lines 1414 to 1426 in e539565

  	def consume(bytes)
  	raise ArgumentError, 'marshal data too short' if @consumed > @stream.bytesize
  	data = @stream.byteslice @consumed, bytes
  	@consumed += bytes
  	data
  	end
  	def consume_byte
  	raise ArgumentError, 'marshal data too short' if @consumed >= @stream.bytesize
  	data = @byte_array[@consumed]
  	@consumed += 1
  	data
  	end

If I add a null byte, CRuby's message is the same, but TruffleRuby's changes slightly:

$ ruby -ve 'Marshal.load("\x04\bo:\vObj\x00")'
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]
<internal:marshal>:34:in `load': marshal data too short (ArgumentError)
        from -e:1:in `<main>'
$ truffleruby -ve 'Marshal.load("\x04\bo:\vObj\x00")'
truffleruby 23.1.0-dev-e539565e, like ruby 3.1.3, GraalVM CE JVM [aarch64-darwin]
<internal:core> core/module.rb:109:in `const_defined?': wrong constant name Obj (NameError)
        from <internal:core> core/post.rb:70:in `const_exists?'
        from <internal:core> core/marshal.rb:592:in `block in const_lookup'
        from <internal:core> core/marshal.rb:591:in `each'
        from <internal:core> core/marshal.rb:591:in `const_lookup'
        from <internal:core> core/marshal.rb:1333:in `construct_object'
        from <internal:core> core/marshal.rb:676:in `construct'
        from <internal:core> core/marshal.rb:1482:in `load'
        from -e:1:in `<main>'

The error message is the same on both plaforms when using the correct marshal string for a class that legitimately does not exist:

$ ruby -ve 'p Marshal.load("\x04\bo:\x0EObjection\x00")'
ruby 3.2.2 (2023-03-30 revision e51014f9c0) [arm64-darwin22]
<internal:marshal>:34:in `load': undefined class/module Objection (ArgumentError)
        from -e:1:in `<main>'
$ truffleruby -ve 'p Marshal.load("\x04\bo:\x0EObjection\x00")'
truffleruby 23.1.0-dev-e539565e, like ruby 3.1.3, GraalVM CE JVM [aarch64-darwin]
<internal:core> core/marshal.rb:598:in `block in const_lookup': undefined class/module Objection (ArgumentError)
        from <internal:core> core/marshal.rb:591:in `each'
        from <internal:core> core/marshal.rb:591:in `const_lookup'
        from <internal:core> core/marshal.rb:1333:in `construct_object'
        from <internal:core> core/marshal.rb:676:in `construct'
        from <internal:core> core/marshal.rb:1482:in `load'
        from -e:1:in `<main>'
<internal:core> core/marshal.rb:596:in `const_missing': uninitialized constant Objection (NameError)
        from <internal:core> core/marshal.rb:596:in `block in const_lookup'
        from <internal:core> core/marshal.rb:591:in `each'
        from <internal:core> core/marshal.rb:591:in `const_lookup'
        from <internal:core> core/marshal.rb:1333:in `construct_object'
        from <internal:core> core/marshal.rb:676:in `construct'
        from <internal:core> core/marshal.rb:1482:in `load'
        from -e:1:in `<main>'

I haven't tried to figure out the scenarios where CRuby says "too short" but TruffleRuby seems to use different conditions for that.