Skip to content

Loading…

[FFI] Struct Arrays behave differently on JRuby vs FFI gem #630

Closed
Burgestrand opened this Issue · 3 comments

1 participant

@Burgestrand

On Ruby 2.0 (FFI gem), the below code runs without errors. On JRuby it will fail on two accounts:

  • A struct by reference (i.e. it’s given a pointer at initialization, memory is not allocated by FFI) will use a VariableLengthArray as field type, where Ruby uses an InlineArray of size 0. VariableLengthArray does not exist in Ruby FFI (I believe?), and while useful is maybe not what one would mean with [:type, 0] — perhaps nil is a better value than 0 to coerce into a VariableLengthArray instead of an Array, and always use Array for any integer value?
  • When FFI allocates the memory, one cannot access the field at all, and I am assuming this is because FFI does not allocate memory for the VariableLengthArray because it does not know how much memory is needed.

Using nil instead of 0 as a signal for a field being of variable length (and not having an explicit size) might break backwards-compatibility, but only on JRuby. Ruby FFI on MRI and Rubinius does not have the VariableLengthArray at all, and always return an Array field.

require "ffi"

class Zero < FFI::Struct
  layout dummy: :uint,
         pointers: [ :pointer, 0 ]
end

class One < FFI::Struct
  layout dummy: :uint,
         pointers: [ :pointer, 1 ]
end

fzero = Zero.layout.fields[1]
fone = One.layout.fields[1]

p [fzero, fzero.size] # => StructLayout::Array, size 0
p [fone, fone.size] # => StructLayout::Array, size 8

zero = Zero.new(FFI::Pointer::NULL) # existing pointer
one = One.new(FFI::Pointer::NULL) # existing pointer

p zero[:pointers] # => Struct::InlineArray (MRI), StructLayout::VariableLengthArrayProxy (JRuby)
p zero[:pointers].size # => 0
# ^ fails on JRuby, NoMethodError: undefined method `size' for #<FFI::StructLayout::VariableLengthArrayProxy:0x5edea768>
p one[:pointers] # => Struct::InlineArray (MRI), Struct::ArrayProxy (JRuby)
p one[:pointers].size # => 1

zero = Zero.new # allocate memory
one = One.new # allocate memory

p zero[:pointers]
# ^ fails on JRuby, IndexError: Memory access offset=8 size=1 is out of bounds
      # [] at org/jruby/ext/ffi/Struct.java:27
p one[:pointers]

Array fields of size 0, instead of being variable length when length is set as 0, are convenient for fields you map over. I’ve defined structs, where the size of the array is defined at runtime (and could be 0), which allows me to write code like this:

class MyStruct < FFI::Struct
  layout count: :uint, names: [:pointer, 0]

  def initialize(…)
    # logic for calculating the *true* layout, runtime, when handed a pointer
    super(…, *new_layout)
  end
end

MyStruct.new(some_pointer)[:names].map(&:read_string) # => returns [] on MRI and Rubinius, raises an error on JRuby
@ghost ghost was assigned
@ghost

You're definitely into "undocumented behaviour" here.

Support for size=0 arrays in structs is fairly recent, and is intended to support the GNU C variable-length trailing array behaviour - i.e. if the last element of a struct is an array of size zero, then generally it is expected that it is being used as a variable-length trailing array to hold an unknown amount of data.

e.g.

struct Packet {
    int length;
    char data[0];
};

It is ruby-ffi that is doing the wrong thing here - when I implemented zero-length array support, I just did a quick-and-dirty check for size==0 and turned off bounds checking - it should be stricter about the other side-effects of that situation. The memory bounds checking should fail the same as it does on JRuby.

I'll have to think about this one - "fixing" the ruby-ffi gem may break some existing code, but "fixing" JRuby to comply with some of the undocumented ruby-ffi side-effects will be a lot more work.

@ghost Unknown added a commit that closed this issue
Wayne Meissner Fix #630 - FFI::Struct Arrays behave differently on JRuby vs FFI gem 5a6279d
@ghost ghost closed this in 5a6279d
@Burgestrand

Support for size=0 arrays in structs is fairly recent, and is intended to support the GNU C variable-length trailing array behaviour - i.e. if the last element of a struct is an array of size zero, then generally it is expected that it is being used as a variable-length trailing array to hold an unknown amount of data.
Ah, it does make sense to follow the C convention. I understand.

I’m having trouble interpreting the diff (could be the early morning). But if I understand correctly it now still returns a variable length array when the size is 0 (but it has a size of 0), and bounds checking is changed to allow retrieval of the array despite it not really being there (since it has a size of 0)? Looks sensible.

I’ve changed the code on my end to not rely on the type of my variable length field too much, and instead just implemented #each on my struct that contains the necessary logic. It feels like a better solution than relying on non-documented behaviour of the ruby FFI gem.

Thank you!

@ghost

That is correct - I only added the #size method, and changed the slicing behaviour - so zero[:pointers] should work.

The types of array fields returned by Struct#[] were not intended for external use (just that they quacked sufficiently like arrays), so neither the ruby-ffi nor jruby ones were really specified.

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.