Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse files

I've filled out the C extension guide. It follows on from the 'make y…

…our own gem' guide.

The conventions are drawn from this tutorial
http://tenderlovemaking.com/2009/12/18/writing-ruby-c-extensions-part-1/
http://tenderlovemaking.com/2010/12/11/writing-ruby-c-extensions-part-2/
which I've used to create a couple of C extensions that seem to work well.
  • Loading branch information...
commit 23d65e2acf534ed4d4b39935611991c47fa0eaa0 1 parent 5477aa9
@jdleesmiller jdleesmiller authored cldwalker committed
Showing with 370 additions and 2 deletions.
  1. +370 −2 c-extensions.md
View
372 c-extensions.md
@@ -5,6 +5,374 @@ previous: /run-your-own-gem-server
next: /resources
---
-How to make a gem with C extensions.
+This guide explains how to package a C extension as a rubygem.
+
+To get started, we'll add a C method to the `hola` gem from the
+[make your own gem](/make-your-own-gem) guide. Then there's some general
+[advice](#advice) and some [references](#references) for further reading.
+
+<a id="tutorial"> </a>
+Tutorial: Add a C extension to `hola`
+========================================
+
+At the end of the [make your own gem](/make-your-own-gem) guide, `hola` looked
+like this:
+
+ % tree
+ .
+ |-- bin
+ | `-- hola
+ |-- hola.gemspec
+ |-- lib
+ | |-- hola
+ | | `-- translator.rb
+ | `-- hola.rb
+ |-- Rakefile
+ `-- test
+ `-- test_hola.rb
+
+Now we'll add in a simple C extension.
+
+<a id="tutorial-ext"> </a>
+Create an `ext` folder
+----------------------
+
+Just as we put the ruby source files in the `lib/hola` folder, we put the C
+extension source files in `ext/hola`. This folder contains two files:
+
+ % tree ext
+ ext
+ `-- hola
+ |-- extconf.rb
+ `-- hola.c
+
+`extconf.rb` tells rubygems how to build the extension. For `hola`, it reads:
+
+ % cat ext/hola/extconf.rb
+ require 'mkmf'
+
+ create_makefile('hola/hola')
+
+`mkmf` is part of ruby's standard library; it automates the process of creating
+a [Makefile](http://en.wikipedia.org/wiki/Makefile) which, in turn, automates
+the process of building the extension. So, when rubygems installs a gem, it
+first runs the gem's `extconf.rb`, and then it runs `make` on the resulting
+`Makefile`.
+
+The output from `make` is a [dynamically linked
+library](http://en.wikipedia.org/wiki/Shared_object). If we build `hola` on
+Linux, the output is a file called `hola/hola.so`, where `.so` stands for
+'shared object'. The extension is platform-dependent; it would be `.dll` on
+Windows, or `.dylib` on Mac OS X. The name is what we pass to `create_makefile`.
+
+In general, a good convention for the name is `<gem_name>/<gem_name>`. While it
+is somewhat annoying to repeat the gem name in so many places, it will be seen
+that it helps to avoid [naming conflicts](#naming), which are even more
+annoying.
+
+The C source code for the extension is in `hola.c`, which reads:
+
+ % cat ext/hola/hola.c
+ #include <ruby.h>
+
+ /* our new native method; it just returns
+ * the string "bonjour!" */
+ static VALUE hola_bonjour(VALUE self) {
+ return rb_str_new2("bonjour!");
+ }
+
+ /* ruby calls this to load the extension */
+ void Init_hola(void) {
+ /* assume we haven't yet defined Hola */
+ VALUE klass = rb_define_class("Hola",
+ rb_cObject);
+
+ /* the hola_bonjour function can be called
+ * from ruby as "Hola.bonjour" */
+ rb_define_singleton_method(klass,
+ "bonjour", hola_bonjour, 0);
+ }
+
+`ruby.h` provides declarations for ruby's C API, which is where the `rb_...`
+methods come from. The `Makefile` generated by `mkmf` ensures that the compiler
+knows where this header file is. Some [references](#references) for ruby's C API
+are given at the end of this guide.
+
+Again, the name of the file is important, because it matches the second `hola`
+in the `hola/hola` that we passed to `create_makefile` in `extconf.rb`.
+Similarly, the name of the `Init_hola` function is important, because ruby looks
+for a function with this name when it loads `hola/hola.so`. A [summary of these
+naming rules](#naming) is provided at the end of the guide.
+
+<a id="tutorial-gemspec"> </a>
+Modify the `gemspec`
+--------------------
+
+For rubygems to know that a gem contains a C extension, we have to tell it about
+`extconf.rb`, and we have to include the C source files in the `files` list.
+
+ % cat hola.gemspec
+ Gem::Specification.new do |s|
+ ... (other stuff) ...
+
+ s.files = Dir.glob('lib/**/*.rb') +
+ Dir.glob('ext/**/*{.c,.h}')
+ s.extensions = ['ext/hola/extconf.rb']
+ s.executables = ['hola']
+
+ ... (other stuff) ...
+ end
+
+Here we're computing the `files` list dynamically; when we run `gem build`,
+rubygems will include all matching files in the gem. Rubygems automatically
+adds the `extensions` and `executables` to the gem, so you don't have to list
+them under `files`.
+
+<a id="tutorial-load"> </a>
+Load the extension
+------------------
+
+The final step is to require the shared object so that ruby will load the
+extension with the rest of the gem. To do this, we simply require `hola/hola` at
+the top of `lib/hola.rb`:
+
+ % cat lib/hola.rb
+ require 'hola/hola'
+
+ ... (rest of file unchanged) ...
+
+This works because rubygems copies the shared object from `ext` to `lib` when
+the gem is installed. It seems a bit like magic, but we can now build and
+install the gem to see what's going on:
+
+ % gem build hola.gemspec
+ Successfully built RubyGem
+ Name: hola
+ Version: 0.0.1
+ File: hola-0.0.1.gem
+
+The `gem build` command creates a `.gem` file from the `.gemspec`. It includes
+all of the source files, but it doesn't compile the extension; that happens when
+the gem is installed:
+
+ % gem install hola-0.0.1.gem
+ Building native extensions. This could take a while...
+ Successfully installed hola-0.0.1
+ 1 gem installed
+
+Where exactly the gem is installed to depends on how ruby is set up. On Linux
+with [rvm](http://beginrescueend.com/), the ruby version manager, the gem is
+installed into the following folder:
+
+ % tree ~/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/
+ /home/john/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/
+ |-- bin
+ | `-- hola
+ |-- ext
+ | `-- hola
+ | |-- extconf.rb
+ | |-- hola.c
+ | |-- hola.o
+ | |-- hola.so
+ | `-- Makefile
+ |-- lib
+ | |-- hola
+ | | |-- hola.so # <----
+ | | `-- translator.rb
+ | `-- hola.rb
+ `-- test
+ `-- test_hola.rb
+
+We can see that the `ext/hola` folder contains our `extconf.rb` and `hola.c`
+files, in addition to the `Makefile` generated during the install, the `hola.o`
+object file generated by the compiler, and the finished product, `hola.so`. We
+can also see that there is a copy of `hola.so` in `lib/hola`.
+
+When we require the gem, rubygems puts `hola`'s `lib` folder on the
+`$LOAD_PATH`, which is where `require` looks for files:
+
+ % irb -rubygems
+ ruby-1.8.7-p352 :001 > require 'hola'
+ => true
+ ruby-1.8.7-p352 :002 > puts $LOAD_PATH
+ /home/john/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/lib
+ ... (lots of standard paths) ...
+ .
+ => nil
+
+When it sees `require 'hola/hola'` at the top of `lib/hola.rb`, it finds
+`~/.rvm/gems/ruby-1.8.7-p352/gems/hola-0.0.1/lib/hola/hola.so`.
+(Note that we don't have to write the `.so` extension -- ruby figures this out
+for itself, which is fortunate, because it is platform-dependent.)
+
+Finally, we can call our C extension's `bonjour` method:
+
+ ruby-1.8.7-p352 :003 > Hola.bonjour
+ => "bonjour!"
+
+The string `"bonjour!"` came from our C extension. Hooray!
+
+Of course, we should also add a test to our test suite:
+
+ % cat test/test_hola.rb
+ require 'test/unit'
+ require 'hola'
+
+ class HolaTest < Test::Unit::TestCase
+
+ ... (other tests) ...
+
+ def test_bonjour
+ assert_equal "bonjour!", Hola.bonjour
+ end
+ end
+
+<a id="tutorial-rakefile"> </a>
+Add helpful tasks to the `Rakefile`
+-----------------------------------
+
+Building and installing the gem every time you make a change quickly gets
+tedious. To speed things up, it helps to add some extra tasks to your Rakefile
+that automate the build process. The following should work on any unix, but
+it would need some tweaking to run on Windows.
+
+ require 'rake/testtask'
+ require 'rake/clean'
+
+ NAME = 'hola'
+
+ # rule to build the extension: this says
+ # that the extension should be rebuilt
+ # after any change to the files in ext
+ file "lib/#{NAME}/#{NAME}.so" =>
+ Dir.glob("ext/#{NAME}/*{.rb,.c}") do
+ Dir.chdir("ext/#{NAME}") do
+ # this does essentially the same thing
+ # as what rubygems does
+ ruby "extconf.rb"
+ sh "make"
+ end
+ cp "ext/#{NAME}/#{NAME}.so", "lib/#{NAME}"
+ end
+
+ # make the :test task depend on the shared
+ # object, so it will be built automatically
+ # before running the tests
+ task :test => "lib/#{NAME}/#{NAME}.so"
+
+ # use 'rake clean' and 'rake clobber' to
+ # easily delete generated files
+ CLEAN.include('ext/**/*{.o,.log,.so}')
+ CLEAN.include('ext/**/Makefile')
+ CLOBBER.include('lib/**/*.so')
+
+ # the same as before
+ Rake::TestTask.new do |t|
+ t.libs << 'test'
+ end
+
+ desc "Run tests"
+ task :default => :test
+
+Now typing `rake` will run the tests after building (or rebuilding) the
+extension, as necessary:
+
+ % rake
+ (in /home/john/rubygems_hola)
+ /home/john/.rvm/rubies/ruby-1.8.7-p352/bin/ruby extconf.rb
+ creating Makefile
+ make
+ gcc -I. -I/home/john/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/1.8/i686-linux -I/home/john/.rvm/rubies/ruby-1.8.7-p352/lib/ruby/1.8/i686-linux -I. -D_FILE_OFFSET_BITS=64 -fPIC -g -O2 -fPIC -c hola.c
+ gcc -shared -o hola.so hola.o -L. -L/home/john/.rvm/rubies/ruby-1.8.7-p352/lib -Wl,-R/home/john/.rvm/rubies/ruby-1.8.7-p352/lib -L. -rdynamic -Wl,-export-dynamic -Wl,-R -Wl,/home/john/.rvm/rubies/ruby-1.8.7-p352/lib -L/home/john/.rvm/rubies/ruby-1.8.7-p352/lib -lruby -lrt -ldl -lcrypt -lm -lc
+ cp ext/hola/hola.so lib/hola
+ Loaded suite /home/john/.rvm/gems/ruby-1.8.7-p352/gems/rake-0.8.7/lib/rake/rake_test_loader
+ Started
+ ....
+ Finished in 0.00182 seconds.
+
+ 4 tests, 4 assertions, 0 failures, 0 errors
+
+If the C source and `extconf.rb` build script have not changed, then running
+`rake` a second time runs only the test suite.
+
+The code for this tutorial is [available on
+github](https://github.com/jdleesmiller/hola).
+
+Advice
+======
+
+<a id="naming"> </a>
+Extension Naming
+----------------
+
+To avoid unintended interactions between gems, it's a good idea for each gem to
+keep all of its files in a single folder. This is the motivation behind the
+naming conventions used in the [tutorial](#tutorial). To summarize, the
+suggested conventions for a gem with name `$g` are:
+
+1. `ext/$g` is the folder that contains the source files and `extconf.rb`
+1. `ext/$g/$g.c` is the main source file (there may be others)
+1. `ext/$g/$g.c` contains a function `Init_$g`
+1. `ext/$g/extconf.rb` calls `create_makefile('$g/$g')`
+1. the gemspec sets `extensions = ['ext/$g/extconf.rb']` and sets `files` to
+ list any C source or header files in `ext/$g`
+1. the first require in `lib/$g.rb` is `require '$g/$g'`
+
+An alternative is to name the extension like `<gem_name>_ext` instead of
+`<gem_name>/<gem_name>`. The result is that the `<gem_name>_ext.so` file is
+installed into the gem's `lib` folder, and it can be required from
+`lib/<gem_name>.rb` as `require '<gem_name>_ext'`. This also works, though it is
+perhaps not as clean as the first convention.
+
+Wrapping Existing Libraries
+---------------------------
+
+A common reason for writing a C extension is to wrap an existing C or C++
+library. This can be done manually (see this tutorial --
+[part
+1](http://tenderlovemaking.com/2009/12/18/writing-ruby-c-extensions-part-1)
+and [part
+2](http://tenderlovemaking.com/2010/12/11/writing-ruby-c-extensions-part-2)),
+but several tools also exist:
+
+* [SWIG](http://www.swig.org/), the Simplified Wrapper Interface Generator, is
+ mature and probably the most popular
+* [rb++](http://rbplusplus.rubyforge.org/) is nicer in several ways but is
+ less stable
+
+Multi-Platform Extensions
+-------------------------
+
+The focus of this guide has been on writing extensions for Linux, but it is also
+possible to write extensions that work on multiple operating systems.
+
+This section needs help! [Please contribute.](http://github.com/rubygems/guides)
+
+Multi-Implementation Extensions
+-------------------------------
+
+There are several ruby implementations. C extensions that use the ruby C API can
+be loaded by the standard ruby interpreter (the MRI -- Matz's Ruby
+Interpreter) and other C-based interpreters, but they cannot be loaded into
+[JRuby](http://jruby.org/) (ruby on the Java Virtual machine) or
+[IronRuby](http://ironruby.net/) (ruby on the Common Language Runtime (.NET)),
+for example.
+
+See [ruby-ffi](http://github.com/ffi/ffi) for a way to build extensions that
+work with other Ruby implementations.
+
+This section needs help! [Please contribute.](http://github.com/rubygems/guides)
+
+References
+==========
+
+This guide is based largely on this excellent two-part tutorial:
+
+* [part 1](http://tenderlovemaking.com/2009/12/18/writing-ruby-c-extensions-part-1)
+* [part 2](http://tenderlovemaking.com/2010/12/11/writing-ruby-c-extensions-part-2)
+
+The main references for ruby's C API are:
+
+* [a chapter in the Pickaxe book](http://www.ruby-doc.org/docs/ProgrammingRuby/html/ext_ruby.html)
+* [the README.EXT file](https://github.com/ruby/ruby/blob/trunk/README.EXT)
-This guide needs help! [Contribute here.](http://github.com/rubygems/guides)
Please sign in to comment.
Something went wrong with that request. Please try again.