layout	title	previous	next
default	C Extensions	/run-your-own-gem-server	/resources

This guide explains how to package a C extension as a rubygem.

To get started, we'll add a C method to the hola gem from the make your own gem guide. Then there's some general advice and some references for further reading.

Tutorial: Add a C extension to `hola`

At the end of the make your own gem guide, hola looked like this:

% tree
.
|-- bin
|   `-- hola
|-- hola.gemspec
|-- lib
|   |-- hola
|   |   `-- translator.rb
|   `-- hola.rb
|-- Rakefile
`-- test
    `-- test_hola.rb

Now we'll add in a simple C extension.

Create an `ext` folder

Just as we put the ruby source files in the lib/hola folder, we put the C extension source files in ext/hola. This folder contains two files:

% tree ext
ext
`-- hola
    |-- extconf.rb
    `-- hola.c

extconf.rb tells rubygems how to build the extension. For hola, it reads:

% cat ext/hola/extconf.rb 
require 'mkmf'

create_makefile('hola/hola')

mkmf is part of ruby's standard library; it automates the process of creating a Makefile which, in turn, automates the process of building the extension. So, when rubygems installs a gem, it first runs the gem's extconf.rb, and then it runs make on the resulting Makefile.

The output from make is a dynamically linked library. If we build hola on Linux, the output is a file called hola/hola.so, where .so stands for 'shared object'. The extension is platform-dependent; it would be .dll on Windows, or .dylib on Mac OS X. The name is what we pass to create_makefile.

In general, a good convention for the name is <gem_name>/<gem_name>. While it is somewhat annoying to repeat the gem name in so many places, it will be seen that it helps to avoid naming conflicts, which are even more annoying.

The C source code for the extension is in hola.c, which reads:

% cat ext/hola/hola.c 
#include <ruby.h>

/* our new native method; it just returns
 * the string "bonjour!" */
static VALUE hola_bonjour(VALUE self) {
  return rb_str_new2("bonjour!");
}

/* ruby calls this to load the extension */
void Init_hola(void) {
  /* assume we haven't yet defined Hola */
  VALUE klass = rb_define_class("Hola",
      rb_cObject);

  /* the hola_bonjour function can be called
   * from ruby as "Hola.bonjour" */
  rb_define_singleton_method(klass,
      "bonjour", hola_bonjour, 0);
}

ruby.h provides declarations for ruby's C API, which is where the rb_... methods come from. The Makefile generated by mkmf ensures that the compiler knows where this header file is. Some references for ruby's C API are given at the end of this guide.

Again, the name of the file is important, because it matches the second hola in the hola/hola that we passed to create_makefile in extconf.rb. Similarly, the name of the Init_hola function is important, because ruby looks for a function with this name when it loads hola/hola.so. A summary of these naming rules is provided at the end of the guide.

Modify the `gemspec`

For rubygems to know that a gem contains a C extension, we have to tell it about extconf.rb, and we have to include the C source files in the files list.

% cat hola.gemspec 
Gem::Specification.new do |s|
  ... (other stuff) ...

  s.files = Dir.glob('lib/**/*.rb') +
            Dir.glob('ext/**/*{.c,.h}')
  s.extensions = ['ext/hola/extconf.rb']
  s.executables = ['hola']

  ... (other stuff) ...
end

Here we're computing the files list dynamically; when we run gem build, rubygems will include all matching files in the gem. Rubygems automatically adds the extensions and executables to the gem, so you don't have to list them under files.

Load the extension

The final step is to require the shared object so that ruby will load the extension with the rest of the gem. To do this, we simply require hola/hola at the top of lib/hola.rb:

% cat lib/hola.rb
require 'hola/hola'

... (rest of file unchanged) ...

This works because rubygems copies the shared object from ext to lib when the gem is installed. It seems a bit like magic, but we can now build and install the gem to see what's going on:

% gem build hola.gemspec 
Successfully built RubyGem
Name: hola
Version: 0.0.1
File: hola-0.0.1.gem

The gem build command creates a .gem file from the .gemspec. It includes all of the source files, but it doesn't compile the extension; that happens when the gem is installed:

% gem install hola-0.0.1.gem 
Building native extensions.  This could take a while...
Successfully installed hola-0.0.1
1 gem installed

Where exactly the gem is installed to depends on how ruby is set up. On Linux with rvm, the ruby version manager, the gem is installed into the following folder:

% tree ~/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/
/home/john/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/
|-- bin
|   `-- hola
|-- ext
|   `-- hola
|       |-- extconf.rb
|       |-- hola.c
|       |-- hola.o
|       |-- hola.so
|       `-- Makefile
|-- lib
|   |-- hola
|   |   |-- hola.so        # <----
|   |   `-- translator.rb
|   `-- hola.rb
`-- test
    `-- test_hola.rb

We can see that the ext/hola folder contains our extconf.rb and hola.c files, in addition to the Makefile generated during the install, the hola.o object file generated by the compiler, and the finished product, hola.so. We can also see that there is a copy of hola.so in lib/hola.

When we require the gem, rubygems puts hola's lib folder on the $LOAD_PATH, which is where require looks for files:

% irb -rubygems
ruby-1.8.7-p352 :001 > require 'hola'
 => true 
ruby-1.8.7-p352 :002 > puts $LOAD_PATH
/home/john/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/lib
... (lots of standard paths) ...
.
=> nil

When it sees require 'hola/hola' at the top of lib/hola.rb, it finds ~/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/lib/hola/hola.so. (Note that we don't have to write the .so extension -- ruby figures this out for itself, which is fortunate, because it is platform-dependent.)

Finally, we can call our C extension's bonjour method:

ruby-1.8.7-p352 :003 > Hola.bonjour
=> "bonjour!"

The string "bonjour!" came from our C extension. Hooray!

Of course, we should also add a test to our test suite:

% cat test/test_hola.rb 
require 'test/unit'
require 'hola'

class HolaTest < Test::Unit::TestCase

  ... (other tests) ...

  def test_bonjour
    assert_equal "bonjour!", Hola.bonjour
  end
end

Add helpful tasks to the `Rakefile`

Building and installing the gem every time you make a change quickly gets tedious. To speed things up, it helps to add some extra tasks to your Rakefile that automate the build process. The following should work on any unix, but it would need some tweaking to run on Windows.

require 'rake/testtask'
require 'rake/clean'

NAME = 'hola'

# rule to build the extension: this says 
# that the extension should be rebuilt
# after any change to the files in ext
file "lib/#{NAME}/#{NAME}.so" =>
    Dir.glob("ext/#{NAME}/*{.rb,.c}") do
  Dir.chdir("ext/#{NAME}") do
    # this does essentially the same thing
    # as what rubygems does
    ruby "extconf.rb"
    sh "make"
  end
  cp "ext/#{NAME}/#{NAME}.so", "lib/#{NAME}"
end

# make the :test task depend on the shared
# object, so it will be built automatically
# before running the tests
task :test => "lib/#{NAME}/#{NAME}.so"

# use 'rake clean' and 'rake clobber' to
# easily delete generated files
CLEAN.include('ext/**/*{.o,.log,.so}')
CLEAN.include('ext/**/Makefile')
CLOBBER.include('lib/**/*.so')

# the same as before
Rake::TestTask.new do |t|
  t.libs << 'test'
end

desc "Run tests"
task :default => :test

Now typing rake will run the tests after building (or rebuilding) the extension, as necessary:

% rake
(in /home/john/rubygems_hola)
/home/john/.rvm/rubies/ruby-1.8.7-p352/bin/ruby extconf.rb
creating Makefile
make
gcc -I. -I/home/john/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/1.8/i686-linux -I/home/john/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/1.8/i686-linux -I. -D_FILE_OFFSET_BITS=64  -fPIC -g -O2  -fPIC   -c hola.c
gcc -shared -o hola.so hola.o -L. -L/home/john/.rvm/rubies/ruby-1.8.7-p352/lib -Wl,-R/home/john/.rvm/rubies/ruby-1.8.7-p352/lib -L.  -rdynamic -Wl,-export-dynamic    -Wl,-R -Wl,/home/john/.rvm/rubies/ruby-1.8.7-p352/lib -L/home/john/.rvm/rubies/ruby-1.8.7-p352/lib -lruby  -lrt -ldl -lcrypt -lm   -lc
cp ext/hola/hola.so lib/hola
Loaded suite /home/john/.rvm/gems/ruby-1.8.7-p352/gems/rake-0.8.7/lib/rake/rake_test_loader
Started
....
Finished in 0.00182 seconds.

4 tests, 4 assertions, 0 failures, 0 errors

If the C source and extconf.rb build script have not changed, then running rake a second time runs only the test suite.

The code for this tutorial is available on github.

A good alternative to writing your own Rakefile like the one above is to use the rake compiler, which does the same stuff -- and much more -- out of the box. It also lets you more easily target multiple platforms and interpreters.

Advice

Extension Naming

To avoid unintended interactions between gems, it's a good idea for each gem to keep all of its files in a single folder. This is the motivation behind the naming conventions used in the tutorial. To summarize, the suggested conventions for a gem with name $g are:

ext/$g is the folder that contains the source files and extconf.rb
ext/$g/$g.c is the main source file (there may be others)
ext/$g/$g.c contains a function Init_$g
ext/$g/extconf.rb calls create_makefile('$g/$g')
the gemspec sets extensions = ['ext/$g/extconf.rb'] and sets files to list any C source or header files in ext/$g
the first require in lib/$g.rb is require '$g/$g'

An alternative is to name the extension like <gem_name>_ext instead of <gem_name>/<gem_name>. The result is that the <gem_name>_ext.so file is installed into the gem's lib folder, and it can be required from lib/<gem_name>.rb as require '<gem_name>_ext'. This also works, though it is perhaps not as clean as the first convention.

Wrapping Existing Libraries

A common reason for writing a C extension is to wrap an existing C or C++ library. This can be done manually (see this tutorial -- part 1 and part 2), but several tools also exist:

SWIG, the Simplified Wrapper Interface Generator, is mature and probably the most popular
rb++ is nicer in several ways but is less stable

Multi-Platform Extensions

The focus of this guide has been on writing extensions for Linux, but it is also possible to write extensions that work on multiple operating systems.

This section needs help! Please contribute.

Multi-Implementation Extensions

There are several ruby implementations. C extensions that use the ruby C API can be loaded by the standard ruby interpreter (the MRI -- Matz's Ruby Interpreter) and other C-based interpreters, but they cannot be loaded into JRuby (ruby on the Java Virtual machine) or IronRuby (ruby on the Common Language Runtime (.NET)), for example.

See ruby-ffi for a way to build extensions that work with other Ruby implementations.

This section needs help! Please contribute.

References

This guide is based largely on this excellent two-part tutorial:

part 1
part 2

The main references for ruby's C API are:

a chapter in the Pickaxe book
the README.EXT file

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

c-extensions.md

c-extensions.md

Tutorial: Add a C extension to `hola`

Create an `ext` folder

Modify the `gemspec`

Load the extension

Add helpful tasks to the `Rakefile`

Advice

Extension Naming

Wrapping Existing Libraries

Multi-Platform Extensions

Multi-Implementation Extensions

References

Files

c-extensions.md

Latest commit

History

c-extensions.md

File metadata and controls

Tutorial: Add a C extension to hola

Create an ext folder

Modify the gemspec

Load the extension

Add helpful tasks to the Rakefile

Advice

Extension Naming

Wrapping Existing Libraries

Multi-Platform Extensions

Multi-Implementation Extensions

References

Tutorial: Add a C extension to `hola`

Create an `ext` folder

Modify the `gemspec`

Add helpful tasks to the `Rakefile`