github github
  • Home
  • Pricing and Signup
  • Training
  • Gist
  • Blog
  • Login

igrigorik / tokyo-recipes

  • Admin
  • Watch Unwatch
  • Fork
  • Your Fork
  • Pull Request
  • Download Source
    • 137
    • 9
  • Source
  • Commits
  • Network (9)
  • Issues (0)
  • Downloads (0)
  • Graphs
  • Branch: master

click here to add a description

click here to add a homepage

  • Switch Branches (2)
    • 38f57d8ffc3bc7538b842f336ec4ca9baac6fa5e
    • master ✓
  • Switch Tags (0)
  • Branch List
Sending Request…

Lean & mean Tokyo Cabinet recipes (with Lua) — Read more

  Cancel

  Cancel
  • HTTP
  • Git Read-Only

This URL has Read+Write access

Added update_new_task_2 
makoto (author)
Sun Mar 07 15:09:05 -0800 2010
commit  b2b34fe52ca7163d9649
tree    614848a6f8ee3a8ffe78
parent  fe7577e51e5378f1e9bf
tokyo-recipes /
name age
history
message
file README.rdoc Thu Nov 26 15:16:46 -0800 2009 fix configuration readme [igrigorik]
directory dynamic-extension/ Thu Jul 30 21:15:34 -0700 2009 recipe for dynamically loading extensions at ru... [igrigorik]
directory echo-command/ Fri Jul 10 14:03:18 -0700 2009 fix echo example [igrigorik]
directory expire/ Tue Jul 07 21:58:33 -0700 2009 [igrigorik]
directory high-low-game/ Tue Jul 07 16:25:29 -0700 2009 link to mikio's post [igrigorik]
directory increment-command/ Mon Jul 06 21:00:24 -0700 2009 simple echo extension with Lua (nothign is stor... [igrigorik]
directory inverted-index/ Tue Jul 07 21:20:17 -0700 2009 simple inverted index via TC's key-value store [igrigorik]
directory map-reduce/ Wed Jul 08 22:09:26 -0700 2009 simple wordcount map-reduce job within Tokyo Ca... [igrigorik]
directory session-trail/ Tue Jul 07 16:25:29 -0700 2009 link to mikio's post [igrigorik]
directory sets/ Tue Jul 07 21:48:25 -0700 2009 Set operations with Lua + TC [igrigorik]
directory wait-queue/ Sun Mar 07 05:22:19 -0800 2010 Added wait-queue [makoto]
directory wait-table/ Sun Mar 07 15:09:05 -0800 2010 Added update_new_task_2 [makoto]
README.rdoc

Video of FutureRuby talk ("Lean & Mean Tokyo Cabinet Recipes") on InfoQ:

  • www.infoq.com/presentations/grigorik-tokyo-cabinet-recipes

Tokyo Cabinet is a library of routines for managing a database. The database is a simple data file containing records, each is a pair of a key and a value. Every key and value is serial bytes with variable length. Both binary data and character string can be used as a key and a value. There is neither concept of data tables nor data types. Records are organized in hash table, B+ tree, or fixed-length array.

User Defined Functions (UDF’s) via Lua

As of mid ‘08, Hirabayashi-san embedded Lua into Tokyo Cabinet’s runtime to allow for easy extensibility of the database. There is a common set of Lua API’s built in, which allow you to interact with the underlying data, execute your extensions remotely, or even on a timer. This allows us to build additional functionality on top of the Tokyo Cabinet database with Lua!

A few examples you will find in this repository:

  • Simple echo application
  • Implemention of sets operations (like Redis) on top of TC
  • Example of executing Map-Reduce jobs on data within TC
  • TTL / Expire functionality (like memcached)
  • and many others…

Please do fork the repo and add your recipes to the list!

  • Lua API: tokyocabinet.sourceforge.net/tyrantdoc/#luaext
  • Announcement blog post: alpha.mixi.co.jp/blog/?p=236

Installing Tokyo Cabinet with Lua

  yum install lua lua-devel

  git clone git://github.com/etrepum/tokyo-cabinet.git
  git clone git://github.com/etrepum/tokyo-tyrant.git

  (or get latest files from sourceforge: http://sourceforge.net/projects/tokyocabinet/files/)

  cd tokyo-cabinet
  ./configure
  make && make install

  cd tokyo-tyrant
  ./configure --enable-lua
  make && make install

  gem install rufus-tokyo

Source: openwferu.rubyforge.org/tokyo.html

Invoking Lua extensions via the HTTP interface

It is possible to invoke custom Lua functions via direct HTTP calls. For example, using the echo-command example, we can invoke the function via:

  $ curl -i -X POST -H “X-TT-XNAME: echo” http://localhost:1978/foo -d “bar”

  HTTP/1.1 200 OK
  Content-Type: application/octet-stream
  Content-Length: 7

  foo:bar
Dedicated Server Powered by the Dedicated Servers and
Cloud Computing of Rackspace Hosting®
  • Blog
  • Support
  • Training
  • Job Board
  • Shop
  • Contact
  • API
  • Status
  • © 2010 GitHub Inc. All rights reserved.
  • Terms of Service
  • Privacy
  • Security
  • English
  • Deutsch
  • Français
  • 日本語
  • Português (BR)
  • 中文
  • See all available languages

Your current locale selection: English. Choose another?

  • English
  • Afrikaans
  • Català
  • Čeština
  • Deutsch
  • Español
  • Français
  • Hrvatski
  • Indonesia
  • Italiano
  • 日本語
  • Nederlands
  • Norsk
  • Polski
  • Português (BR)
  • Српски
  • Svenska
  • 中文