rufus-tokyo

ruby-ffi based interface to Tokyo Cabinet and Tokyo Tyrant.

The ‘abstract’ and the ‘table’ API are covered for now.

installation

  sudo gem install rufus-tokyo --source http://gemcutter.org

  (see after 'usage' for how to install Tokyo Cabinet (and Tyrant) if required)

Rufus::Edo

Note : Rufus::Tokyo focuses on leveraging Hirabayashi-san’s C libraries via ruby-ffi, but the gem rufus-tokyo also contains Rufus::Edo which wraps the Tokyo Cabinet/Tyrant author’s [native] C bindings :

  http://github.com/jmettraux/rufus-tokyo/tree/master/lib/rufus/edo

usage

hereafter TC references Tokyo Cabinet, while TT references Tokyo Tyrant.

the rdoc is at rufus.rubyforge.org/rufus-tokyo/

version [mis]match

this ‘master’ branch/version of rufus-tokyo works with

  ffi 0.3.1, 0.3.5, 0.4.0, 0.5.0, 0.5.1

  ruby 1.8.6
  ruby 1.9.1p129
  jruby 1.1.6, 1.2.0, 1.3.1
  ree 1.8.7 2009.10 (thanks to http://rvm.beginrescueend.com)

  TC 1.4.27 / 29 and TT 1.1.29 / 33

  tc-ruby 1.26 / 29 and tt-ruby 1.10 / 12 (Rufus::Edo)

Rufus::Tokyo : pointing to the right library

By default Rufus::Tokyo (the FFI side) searches for Tokyo Cabinet and Tyrant dynamic libraries under /usr/lib/, /usr/lib64/, /opt/local/lib/ and /usr/local/lib/

If you had to install Tokyo Cabinet (and Tokyo Tyrant) in another location, you can set the environment variables TOKYO_CABINET_LIB and TOKYO_TYRANT_LIB to point to the absolute path of the dynamic library, like in :

  export TOKYO_CABINET_LIB = /home/toto/tc/libtokyocabinet.so
  export TOKYO_TYRANT_LIB = /home/toto/tc/libtokyotyrant.so

So that your rufus-tokyo enabled Ruby application can see the libraries.

TC Abstract API

tokyocabinet.sourceforge.net/spex-en.html#tcadbapi

to create a hash (file named ‘data.tch’)

  require 'rubygems'
  require 'rufus/tokyo'

  db = Rufus::Tokyo::Cabinet.new('data.tch')

  db['nada'] = 'surf'

  p db['nada'] # => 'surf'
  p db['lost'] # => nil

  5000.times { |i| db[i.to_s] = "x" }

  p db.inject { |r, (k, v)| k } # => 4999

  db.close

TC Table API

tokyocabinet.sourceforge.net/spex-en.html#tctdbapi

  require 'rubygems'
  require 'rufus/tokyo'

  t = Rufus::Tokyo::Table.new('table.tct')

  t['pk0'] = { 'name' => 'alfred', 'age' => '22' }
  t['pk1'] = { 'name' => 'bob', 'age' => '18' }
  t['pk2'] = { 'name' => 'charly', 'age' => '45' }
  t['pk3'] = { 'name' => 'doug', 'age' => '77' }
  t['pk4'] = { 'name' => 'ephrem', 'age' => '32' }

  p t.query { |q|
    q.add_condition 'age', :numge, '32'
    q.order_by 'age'
  }
    # => [ {"name"=>"ephrem", :pk=>"pk4", "age"=>"32"},
    #      {"name"=>"charly", :pk=>"pk2", "age"=>"45"} ]

  t.close

Note that the Tokyo Cabinet Table API does support transactions :

  p t.size
    # => 0

  t.transaction do
    t['pk0'] = { 'name' => 'alfred', 'age' => '22' }
    t['pk1'] = { 'name' => 'bob', 'age' => '18' }
    t.abort
  end

  p t.size
    # => 0

TT remote db

tokyocabinet.sourceforge.net/tyrantdoc/

to start a ttserver (backed by a hash), on the command line

  ttserver -port 45001 data.tch

then, in Ruby :

  require 'rubygems'
  require 'rufus/tokyo/tyrant'

  db = Rufus::Tokyo::Tyrant.new('localhost', 45001)

  db['nada'] = 'surf'

  p db['nada'] # => 'surf'
  p db['lost'] # => nil

  db.close

Rufus::Tokyo::Tyrant instances have a #stat method :

  puts db.stat.inject('') { |s, (k, v)| s << "#{k} => #{v}\n" }
    # =>
    #   pid => 7566
    #   loadavg => 0.398438
    #   size => 528736
    #   rnum => 0
    #   time => 1234764065.305923
    #   sid => 898521513
    #   type => hash
    #   bigend => 0
    #   ru_sys => 3.398698
    #   version => 1.1.15
    #   ru_user => 2.155215
    #   ru_real => 3218.451152
    #   fd => 7

Note that it’s also OK to make a Tokyo Tyrant server listen on a unix socket :

  ttserver -host /tmp/ttsocket -port 0 data.tch

and then :

  require 'rubygems'
  require 'rufus/tokyo/tyrant'
  db = Rufus::Tokyo::Tyrant.new('/tmp/ttsocket')
  db['a'] = 'alpha'
  db.close

TT remote table

to start a ttserver (backed by a table), on the command line :

  ttserver -port 45002 data.tct

then, in Ruby, much like a local table :

  require 'rubygems'
  require 'rufus/tokyo/tyrant'

  t = Rufus::Tokyo::TyrantTable.new('localhost', 45002)

  t['pk0'] = { 'name' => 'alfred', 'age' => '22' }
  t['pk1'] = { 'name' => 'bob', 'age' => '18' }
  t['pk2'] = { 'name' => 'charly', 'age' => '45' }
  t['pk3'] = { 'name' => 'doug', 'age' => '77' }
  t['pk4'] = { 'name' => 'ephrem', 'age' => '32' }

  p t.query { |q|
    q.add_condition 'age', :numge, '32'
    q.order_by 'age'
  }
    # => [ {"name"=>"ephrem", :pk=>"pk4", "age"=>"32"},
    #      {"name"=>"charly", :pk=>"pk2", "age"=>"45"} ]

  t.close

Rufus::Tokyo::TyrantTable instances have a #stat method :

  puts t.stat.inject('') { |s, (k, v)| s << "#{k} => #{v}\n" }
    # =>
    #   pid => 7569
    #   loadavg => 0.295410
    #   size => 935792
    #   rnum => 0
    #   time => 1234764228.942014
    #   sid => 1027604232
    #   type => table
    #   bigend => 0
    #   ru_sys => 5.966750
    #   version => 1.1.15
    #   ru_user => 2.601947
    #   ru_real => 3382.084479
    #   fd => 10

Note that it’s also OK to make a Tokyo Tyrant server listen on a unix socket :

  ttserver -host /tmp/tttsocket -port 0 data.tct

and then :

  require 'rubygems'
  require 'rufus/tokyo/tyrant'
  t = Rufus::Tokyo::TyrantTable.new('/tmp/tttsocket')
  t['customer0'] = { 'name' => 'Heike no Kyomori', 'age' => '75' }
  t.close

rdoc

more in the rdoc

• rufus.rubyforge.org/rufus-tokyo/
• rufus.rubyforge.org/rufus-tokyo/classes/Rufus/Tokyo/Cabinet.html
• rufus.rubyforge.org/rufus-tokyo/classes/Rufus/Tokyo/Table.html
• rufus.rubyforge.org/rufus-tokyo/classes/Rufus/Tokyo/Tyrant.html
• rufus.rubyforge.org/rufus-tokyo/classes/Rufus/Tokyo/TyrantTable.html

don’t hesitate to "man ttserver" on the command line.

or directly in the source

• github.com/jmettraux/rufus-tokyo/blob/master/lib/rufus/tokyo/cabinet/abstract.rb
• github.com/jmettraux/rufus-tokyo/blob/master/lib/rufus/tokyo/cabinet/table.rb
• github.com/jmettraux/rufus-tokyo/blob/master/lib/rufus/tokyo/tyrant/abstract.rb
• github.com/jmettraux/rufus-tokyo/blob/master/lib/rufus/tokyo/tyrant/table.rb

Tokyo Cabinet / Tyrant install

a compilation of notes is available at :

  http://openwferu.rubyforge.org/tokyo.html

dependencies

the ruby gem ‘ffi’

mailing list

On the rufus-ruby list :

  http://groups.google.com/group/rufus-ruby

Note : Flinn Mueller started an english speaking mailing list for TC / TT / TD

  http://groups.google.com/group/tokyocabinet-users

issue tracker

  http://github.com/jmettraux/rufus-tokyo/issues/

irc

irc.freenode.net #ruote

source

github.com/jmettraux/rufus-tokyo

  git clone git://github.com/jmettraux/rufus-tokyo.git

credits

many thanks to the author of Tokyo Cabinet, Mikio Hirabayashi, and to the authors of ruby-ffi

• 1978th.net/tokyocabinet/
• 1978th.net/tokyotyrant/
• 1978th.net/tokyodystopia/
• wiki.github.com/ffi/ffi

dystopia interface developed with and sponsored by Seven Scale <sevenscale.com/>, creators of Open Syslog.

authors

• John Mettraux, jmettraux@gmail.com, jmettraux.wordpress.com
• Zev Blut, www.iknow.co.jp/users/zev
• Jeremy Hinegardner github.com/copiousfreetime (Tokyo Dystopia)
• James Edward Gray II blog.grayproductions.net/

the rest of Rufus

rufus.rubyforge.org

license

MIT