Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Rack server for interacting with a Xapian database remotely through Xapit.

branch: master

Fetching latest commit…

Octocat-spinner-32-eaf2f5

Cannot retrieve the latest commit at this time

Octocat-spinner-32 lib
Octocat-spinner-32 spec
Octocat-spinner-32 .autotest
Octocat-spinner-32 .gitignore
Octocat-spinner-32 LICENSE
Octocat-spinner-32 README.rdoc
Octocat-spinner-32 Rakefile
Octocat-spinner-32 config.ru
README.rdoc

Xapit Server

A central Rack application to interact with a Xapian database. This way your data stays in sync without needing to update the readable databases on each instance.

This is intended to be used through the Xapit gem.

Pros & Cons

Compared to Xapit Sync

Pros

  • all records will be updated instantly so you do not have to worry about syncing the database.

  • the database is only loaded into memory once, not separately for each Rails instance. This is very important if you have a large database.

  • allows the Xapian database to be stored on a separate machine or network than the application.

  • does not require Rails.

Cons

  • additional security issues (if Rack server is exposed).

  • requires a separate daemon process to constantly be running (and monitored).

  • more overall traffic to/from the external process.

  • more difficult to test in staging/development.

Setup

To use Xapit Server you will need to set up a Rackup file like this.

# config.ru
require "xapit_server"
run XapitServer::Application.new(:database_path => "/path/to/xapiandb")

There are a number of webservers which can host this Rack application: Phusion Passenger, Mongrel, Thin, Unicorn, etc. I'll leave it up to you to set that up.

Once it is running, just change the Xapit database path to point to the URL of the Xapit Server.

# in config/initializers/setup_xapit.rb
Xapit::Config.setup(:database_path => "http://locahost:12345")

There is no authorization performed by Xapit Server, so be certain to restrict the port it is running on.

Initial Data

If you already have a large set of data which needs to be indexed, running that all through Xapit Server will create a lot of overhead. Instead I recommend having Xapit go straight to the file system to populate the initial database.

# in config/initializers/setup_xapit.rb
Xapit::Config.setup(:database_path => "/path/to/xapiandb")

Once this is done, use that database through Xapit Server.

Commands

Xapit Server is a REST API, here are the commands which it responds to.

POST /documents

Create a new document index using the POST attributes.

  • id: any unique string value to identify this record.

  • terms: a comma separated list of terms. Weight for each will be “1”.

  • data: custom data to fetch through the show action

  • weights: a comma separated list of weights which should parallel terms.

  • values: a comma separated list of values.

  • value_indexes: a comma separated list of value indexes, defaults to incrementing numbers starting with zero.

  • spellings: a comma separated list of spellings to add along with the document. There is another interface to add spellings but this is for convenience.

GET /documents/:id

Returns the custom data stored in the document at the given id.

PUT /documents/:id

Update an existing document index. The attributes passed are the same as creating a document except the id attribute is not allowed.

DELETE /documents/:id

Deletes the document matching that id.

POST /queries

Pass in a query and get a list of matching document ids and percentages in response. While this technically should be a GET request since nothing changes on the server, the query parameter may be very long, so a POST request is necessary.

  • query: serialized Xapian::Query

  • sort: a comma separated list of values to sort by

  • descending: to sort in descending order (true/false)

  • offset: where to start the fetching

  • limit: how many documents to fetch

  • collapse_key: the key to collapse

POST /spellings

  • spellings: a comma separated list of terms to add to the spelling dictionary.

GET /spellings

Returns spellings suggestions given a term.

Something went wrong with that request. Please try again.