Permalink
Browse files

rufus-tokyo for interfacing with tokyo cabinet instead of tc ruby bin…

…dings
  • Loading branch information...
1 parent 541c346 commit eae535e15486b1e1a420775feaa8e915d3c334fe Elise Huard committed Jul 7, 2009
View
59 README
@@ -4,44 +4,87 @@ Tokyo-cabinet4r
Tokyo cabinet hash plugin for Rails.
Subclassing TokyoCabinet4r allows the easy creation and usage of some Tokyo Cabinet functionalities.
- Tokyo Cabinet B+ tree (http://en.wikipedia.org/wiki/B%2B_tree)
-- Tokyo Cabinet table database API (arbitrary columns, usual key-value stores)
+- Tokyo Cabinet table database API (freely structured documents, like other key-value stores)
(see http://tokyocabinet.sourceforge.net/index.html for more, but not implemented yet)
Install
========
Prerequisites:
Install Tokyo Cabinet (source available at http://tokyocabinet.sourceforge.net/index.html).
-Install Tokyo Cabinet Ruby bindings (http://tokyocabinet.sourceforge.net/rubypkg/ - doc http://tokyocabinet.sourceforge.net/rubydoc/).
-(had to use build options --with-tokyocabinet-include= --with-tokyocabinet-lib= pointing to the source directory of tokyocabinet)
+UPDATE: this seems to be available as a gem on github:
+sudo gem install tokyocabinet-ruby
install as a plugin for Rails. Tested with Rails 2.3.2
Example
=======
Btree:
+------
use generator
script/generate tc_bdb phone_number
-this creates a subclass of TokyoCabinet4r::BDB, which can be used in rails. The db file is created in
-RAILS_ROOT/db/tokyo/RAILS_ENV/
-and has the underscored name of the given class.
+Usage:
+ new key-value
+ janes = PhoneNumber.new("jane","+32444888333")
+
+ retrieve
+ janes = PhoneNumber.get("jane")
+
+ on the instance, to update:
+ janes.update("+223222333444")
+
+ delete
+ janes.delete
+ PhoneNumber.delete("jane")
+
+ delete the whole table
+ PhoneNumber.drop
+
+Table in these examples is stored in RAILS_ROOT/db/tokyo/RAILS_ENV/phone_number.tcb
Table:
+------
use generator
script/generate tc_tdb shortest_path
+Usage:
+ john = Description.new({"hair" => "black", "size" => "tall"})
+
+ retrieve
+ john = Description.get("john")
-Why i developed this (example of use case): for scalable shortest path calculation.
+ on the instance, to update the whole record:
+ john.update({"hair" => "red", "eyes" => "squinty"})
+
+ add or update one value:
+ john.add_data("eyes","blue") # record now {"hair" => "red", "eyes" => "blue"}
+ john.add_data("shoe_size","42") # record now {"hair" => "red","eyes" => "blue", "shoe_size" => "42"}
+
+ delete the record
+ john.delete
+ Description.delete("john")
+
+ delete table
+ Description.drop
+
+Table in these examples is stored in RAILS_ROOT/db/tokyo/RAILS_ENV/description.tct
Todo
=====
-* test with Ruby 1.9
* more elegant handling of opening and closing of file, without resorting to too much metaprogramming.
+* add get_by_* name for the tables, similarly to find_by - use queries
+* getters and setters for attributes of tables ?
* investigate use of Tokyo Tyrant.
* transactions
* cursors
* implement other tokyo database structures (if demand)
+* explore and add extra features of the Tokyo Cabinet tables and hashes
+
+Changes:
+=======
+now using rufus-tokyo (http://github.com/jmettraux/rufus-tokyo/tree/master) to allow compatibility with several Ruby implementations (thanks jmettraux !)
Copyright (c) 2009 [Elise Huard], released under the MIT license
+Made in Belgium ;)
View
@@ -1,5 +1,5 @@
# Install hook code here
require 'fileutils'
-# create directory to store hashes in
+# create directory to store data stores in
FileUtils.mkpath_p("#{RAILS_ROOT}/db/tokyo/#{RAILS_ENV}")
@@ -1,40 +0,0 @@
-require 'rubygems'
-require 'tokyocabinet'
-
-module TokyoCabinet4r
-
- class UnknownType < StandardError
- end
-
- class FileNotOpened < StandardError
- end
-
- class Adapter
-
- STORAGE = "#{RAILS_ROOT}/db/tokyo/#{RAILS_ENV}/"
- attr_accessor :type
- attr_accessor :db
-
- def initialize(tokyo_klas)
- raise UnknownType if tokyo_klas != TokyoCabinet::TDB && tokyo_klas != TokyoCabinet::BDB
- @type = tokyo_klas
- end
-
- def close
- closed = @db.close
- closed
- end
-
- def open(klas)
- # create database file with name = name of class in lowercase ".hdb" - and then set @@storage
- # activesupport underscore
- name = klas.to_s.underscore
- storage = STORAGE + klas.to_s.underscore + "." + @type.to_s.split('::')[-1].downcase
- @db = @type::new
- if !@db.open(storage, TokyoCabinet::TDB::OWRITER | TokyoCabinet::TDB::OCREAT)
- STDERR.printf("open error: %s\n", @db.errmsg(@db.ecode))
- raise FileNotOpened.new(@db.errmsg(@db.ecode))
- end
- end
- end
-end
View
@@ -1,7 +1,6 @@
require 'rubygems'
-require 'tokyocabinet'
require 'activesupport'
-require 'adapter'
+require 'cabinet_adapter'
require 'connection'
require 'errors'
@@ -15,10 +14,10 @@ module TokyoCabinet4r
# It is forbidden for multible database objects in a process to open the same database at the same time."
class Bdb
- include TokyoCabinet
+ #include TokyoCabinet
cattr_accessor :connection, :instance_writer => false
- @@connection = Adapter.new(BDB)
+ @@connection = CabinetAdapter.new(:btree)
include Connection
attr_accessor :key, :value
@@ -33,26 +32,11 @@ def initialize(key,value)
# get record based on key
def self.get(key)
open
- value = db.get(key)
- close
- value
+ returning db[key] do |result|
+ close
+ end
end
-# ponder interface for duplicate keys
-# def self.all(key)
-# open
-# result = []
-# cur = BDBCUR::new(db)
-# cur.first
-# while key = cur.key
-# value = cur.val
-# result << value unless value.nil?
-# cur.next
-# end
-# close
-# result
-# end
-
# completely update existing record
# raises error if not found
# returns true if success, false if not
@@ -66,34 +50,31 @@ def update(value)
# delete an instance. Returns true if successful.
def delete
open
- result = db.out(self.key)
- close
- result
+ returning db.delete(self.key) do |result|
+ close
+ end
end
- # delete an instance from the class. Returns true if successful
+ # delete an instance from the class
def self.delete(key)
open
- result = db.out(key)
- close
- result
+ returning db.delete(key) do |result|
+ close
+ end
end
# drop the entire table - delete the file
def self.drop
open
- File.unlink(db.path)
+ delete_file # !!! TODO this is also abstracted away to adapter
close
end
protected
- # returns false if unsuccessful
+ # returns true if successful
def put(key,value)
open
- unless db.put(key,value)
- STDERR.printf("put error: %s\n", db.errmsg(db.ecode))
- raise TCPutError.new(db.errmsg(db.ecode))
- end
- close
+ db[key] = value
+ close
end
end
@@ -0,0 +1,37 @@
+require 'rubygems'
+require 'rufus/tokyo'
+require 'errors'
+
+module TokyoCabinet4r
+
+ class CabinetAdapter
+
+ STORAGE = "#{RAILS_ROOT}/db/tokyo/#{RAILS_ENV}/"
+ EXTENSIONS = {:btree => '.tcb',:hash => ".tch", :fixed => ".tcf"}
+ attr_accessor :type
+ attr_accessor :db
+ attr_accessor :path
+
+ def initialize(type)
+ raise UnknownType if type != :btree && type != :hash && type != :fixed
+ @type = type
+ end
+
+ def close
+ closed = @db.close
+ closed
+ end
+
+ def open(klas)
+ # create database file with name = name of class in lowercase ".hdb" - and then set @@storage
+ # activesupport underscore
+ name = STORAGE + klas.to_s.underscore
+ @path = name + EXTENSIONS[@type]
+ @db = Rufus::Tokyo::Cabinet.new(name,:type => @type)
+ end
+
+ def delete_file
+ File.unlink(@path)
+ end
+ end
+end
@@ -1,8 +1,7 @@
require 'activesupport'
-require 'tokyocabinet'
+#require 'tokyocabinet'
module TokyoCabinet4r
module Connection
-
def self.included(base)
base.extend(ClassMethods)
base.send :include, InstanceMethods
@@ -15,6 +14,10 @@ def open
connection.open(self.class)
end
+ def access(&block)
+ connection.access(self.class,&block)
+ end
+
def close
connection.close
end
@@ -28,22 +31,26 @@ def db
end
end
module ClassMethods
-
+ protected
def open
connection.open(self)
end
+ def access(&block)
+ connection.access(self,&block)
+ end
+
def close
connection.close
end
- #
- #def connection
- # @@connection
- #end
def db
connection.db
end
+
+ def delete_file
+ connection.delete_file
+ end
end
end
end
@@ -1,13 +1,19 @@
module TokyoCabinet4r
- class TCKeyNotFound < Exception
+ class TCKeyNotFound < StandardError
end
- class TCdbIssue < Exception
+ class TCdbIssue < StandardError
end
- class TCBDBIssue < Exception
+ class TCBDBIssue < StandardError
end
- class TCPutError < Exception
+ class TCPutError < StandardError
end
+
+ class UnknownType < StandardError
+ end
+
+ class FileNotOpened < StandardError
+ end
end
@@ -0,0 +1,35 @@
+require 'rubygems'
+require 'rufus/tokyo'
+require 'errors'
+
+module TokyoCabinet4r
+
+ class TableAdapter
+
+ STORAGE = "#{RAILS_ROOT}/db/tokyo/#{RAILS_ENV}/"
+ attr_accessor :db
+
+ def close
+ @db.close
+ end
+
+ def open(klas)
+ @db = Rufus::Tokyo::Table.new(path_name(klas))
+ end
+
+ def access(klas,&block)
+ Rufus::Tokyo::Cabinet.new(path_name(klas)) do |db|
+ yield db
+ end
+ end
+
+ def delete_file
+ File.unlink(@db.path)
+ end
+
+ private
+ def path_name(klas)
+ STORAGE + klas.to_s.underscore + ".tct"
+ end
+ end
+end
Oops, something went wrong.

0 comments on commit eae535e

Please sign in to comment.