Core Concepts

Dave Worth edited this page Jun 18, 2013 · 4 revisions

FFI is a fantastic tool for easily interfacing your Ruby code with native libraries. To help you quickly ramp up and become a happier and more productive FFI master, the following are a few of the fundamental concepts you'll want to understand in order to get the most out of FFI.

Overall Architecture

Using FFI, you can use native libraries from Ruby without writing a single line of native code. Your good friend FFI takes care of all of the cross Ruby implementation (MRI, JRuby, Rubinius, MacRuby, etc) and platform specific issues so that you can focus on writing and testing your Ruby code.

As great as this sounds, you just know there's something missing. Your spidey sense is right again.

As FFI is effectively a bridge between the multiple worlds of Ruby implementations and multiple platform types, you might suffer a bit of cognitive dissonance trying to pull all the pieces together. When we develop in Ruby we tend to think in higher level terms, and don't so much concern ourselves with the lower level issues.

While FFI allows us to stay firmly rooted in Ruby, we also have to start thinking in lower level terms. To do so, it's good to have a high level picture of where FFI fits in.

Figure 1 - Overall FFI Architecture


Figure 2 - FFI Cross Ruby Distro Usage


Core Components

FFI has a number of useful components. Investing the time to understand FFI's components and capabilities will pay off as you begin using FFI. That said, it's nice to have an idea which components you should look at first. Understanding the following core modules and classes is a great way to start getting FFI's capabilities:

  • FFI::Library - along with require 'ffi', this module brings FFI's powerful native library interfacing capabilities into your Ruby code as a DSL. Typically you extend your custom module with this one, specify the native libraries and their calling conventions, prototype the native library's functions and structs in Ruby, and then start using the native library's API from Ruby.
  • FFI::Pointer - wraps native memory allocated by a third party library. It provides a number of methods for transferring data from unmanaged native memory to Ruby-managed native memory (FFI::MemoryPointer). The native memory wrapped by this class is not freed during garbage collection runs.
  • FFI::MemoryPointer - allows for Ruby code to allocate native memory and pass it to non-Ruby libraries. Lifecycle management (allocation and deallocation) are handled by this class, so when it gets garbage collected the native memory is also freed.
  • FFI::Struct and FFI::Union

Memory Management

As mentioned earlier, FFI plays multiple roles in your Ruby environment. It provides an API used by your Ruby code. It provides infrastructure code unique to each supported Ruby implementation, and at the lowest level, it provides a gateway to interfacing with your system's native capabilities. But with that combination of high-level and low-level power, you need to be aware of the implications to your Ruby code.

One such consideration is memory management. When you're writing Ruby code you usually don't think about memory management. It's just taken care of for most of your use cases. However, when you're leveraging FFI, even though you're still developing in Ruby, you have to begin thinking more about these low-level issues.

As a result of FFI supporting multiple Ruby implementations, you may also need to consider the memory management capabilities of a specific Ruby implementation. For example, in a Ruby implementation such as JRuby which uses a copying/moving garbage collector (GC) there are two types of heap memory - Ruby heap, and native heap with the Ruby heap being managed by the Ruby implementations GC.

TODO concrete rules-of-thumb to payoff the above intro

  • Every FFI instance lives on the heap; there is no concept of a C stack in FFI. Instances live either on the native heap (objects like FFI::MemoryPointer and other Pointer instances) or the Ruby heap (everything else).

String Memory Allocation

One often overlooked memory management consideration occurs when you try to integrate with C functions that keep a reference to a Ruby allocated string rather than making their own copy of the string's contents. Take the following rogueware snippet which occurs more often than we'd like to admit:

static char* my_name;

void bad_set_my_name(char* name) {
  my_name = name;

This code assumes that its my_name reference to the Ruby string will remain valid during the time it's needed. This poor assumption can complicate your Ruby code attempting to use this C function.

What happens if the Ruby string is garbage collected or otherwise moved in memory, resulting in the memory address held by my_name no longer pointing to the string?

In MRI, and Ruby implementations with similar memory management semantics, as long as you keep a hard reference to the Ruby allocated string (e.g. - use a class variable or constant), the string is kept alive, and you can call the C function like:

module Foo
  NAME = "Barney"

But the above will fail in JRuby because in JRuby, Ruby strings live in Java heap memory. During garbage collection these Ruby strings can be moved around making it impossible to guarantee that the memory address stored by the native C function still points to the Ruby string that the C function expects.

To solve this issue, your Ruby code should manually copy the Ruby string to native memory, pass the pointer to that string to the C function instead of passing a raw string, and finally, update your original attach_function signature to use the pointer rather than the original string. For example:

# proper use of the bad C function from Ruby
module Foo
  NAME = FFI::MemoryPointer.from_string("Barney")

# old Ruby integration code
module Bar
  attach_function :bad_set_my_name, [ :string ], :void

# new Ruby integration code 
module Bar
  attach_function :bad_set_my_name, [ :pointer ], :void

So should your Ruby code always manually copy string to native memory?

No, not all C code that you'll integrate with will behave this poorly. Stash this low-level consideration away for a rainy day and follow the sagely advice of that well known Rubyist Yogi Berra, "Don't do it until you need to do it."

Using malloc/free

The FFI library has a special library wrapper for libc across all major platforms.

module LibC
  extend FFI::Library
  ffi_lib FFI::Library::LIBC

  # call #attach_function to attach to malloc, free, memcpy, bcopy, etc.
  # For example
  # attach_function :malloc, [:size_t], :pointer
  # attach_function :free, [:pointer], :void

It is possible to use attach_function to wrap up malloc and free for doing memory management completely independently of the Ruby runtime. This is sometimes necessary for libraries that take a memory buffer as an argument and then expect to manage that buffer's lifecycle.

# the call to LibC.malloc will return an instance of FFI::Pointer
buffer = LibC.malloc 1000
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.