Skip to content
This repository

Rails plugin for Tokyo Dystopia

branch: master

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 benchmarks
Octocat-spinner-32 lib
Octocat-spinner-32 tasks
Octocat-spinner-32 test
Octocat-spinner-32 .gitignore
Octocat-spinner-32 MIT-LICENSE
Octocat-spinner-32 README.textile
Octocat-spinner-32 Rakefile
Octocat-spinner-32 init.rb
Octocat-spinner-32 install.rb
Octocat-spinner-32 uninstall.rb
README.textile

DystopianIndex

This is a Rails plugin that uses Tokyo Dystopia to index models. It’s ideal for small projects that need a fast indexer because it’s easy to setup and understand, and doesn’t use too many resources or daemon processes.

Requirements

Gems:

  • faker
  • rufus-tokyo

Installation

Build and compile Tokyo Cabinet and Tokyo Dystopia from tokyocabinet.sourceforge.net — they’re clean and simple C projects and should build on your Mac or Linux machine.

Then install the rufus-tokyo gem.

Usage

To use the plugin with a model, define an index like this:

  dystopian_index do
    indexes :content, :name
    order_by :created_at
  end

Rake Tasks

rake dystopia:benchmarks                  # Runs benchmarks
rake dystopia:index                       # Indexes all models

Database files are stored in db/indexes. You can change the path where the database is stored by setting DystopianIndex.config.db_path.

Plugin Design

  • Each model has its own Dystopia index
  • Fields are concatenated into one big index record — this means you can’t currently search according to field
  • When an update is performed, the index is opened and closed on demand to help prevent data loss. I’ve indexed this scheme (see the benchmarks) and it’s not too slow
  • Data information is stored in the first few bytes of each index record to help sort values
  • Indexes are stored in db/indexes/ — the plugin will make this directory if required

TODO

  • Tests only run from within a Rails app, and by executing each test file with ruby
  • It might be better to create separate indexes for each field.
  • order_by needs desc
  • Sorting fetches every matching ID from dystopia and sorts according to date integers. There might be a better way to do this, but it’s still pretty fast

Acknowledgements

The Rake tasks need to load all models. This code was based on ThinkingSphinx’s code that does the same thing (which is a nice piece of work by the way). I hope to refactor this out later.

Copyright © 2009 Alex R. Young, released under the MIT license

Something went wrong with that request. Please try again.