Generic Persistence API for Ruby
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.

Hyperion Build Status

1 API, multiple database backends.

Hyperion provides you with a simple API for data persistence allowing you to delay the choice of database without delaying your development.

There are a few guiding principles for Hyperion.

  1. key/value store. All Hyperion implementations, even for relational databases, conform to the simple key/value store API.
  2. values are maps. Every 'value' the goes in or out of a Hyperion datastore is a hash.
  3. :key and :kind. Every 'value' must have a :kind entry; a short string like "user" or "product". Persisted 'value's will have a :key entry; strings generated by the datastore.
  4. Search with data. All searches are described by data. See find_by_kind below.

Hyperion Implementations:


To use just the in-memory datastore...

gem 'hyperion-api'

To use a specific implementation...

gem 'hyperion-postgres'
# or
gem 'hyperion-mysql'
# or
gem 'hyperion-sqlite'


Instantiating datastores

Hyperion provides a convenient factory function for instantiating any datastore implementation

require 'hyperion'
Hyperion.new_datastore(:postgres, options)
Hyperion.new_datastore(:sqlite, options)

This will require the file "hyperion/<impl>" and instantiate the class Hyperion::<impl camelcased>

Each implementation must accept a hash of options in it's initializer

module Hyperion
  class Memory
    def initialize(opts={})

Installing a datastore

Before you can use the Hyperion API, you must tell Hyperion which backend datastore you would like to use. There a couple of ways to go about this.

With elegance

This installs the datastore thread locally, and only within the given block.

# datastore not installed
Hyperion.with_datastore(:postgres, options) do
  # datastore is installed
  # some persistence code here
# datastore not installed

This method is useful for applications that have control of the main thread, such as webapps. The call to with_datastore can be placed inside of a middleware, such that the datastore is installed at the beginning of the request and uninstalled after the request. For example, this is a Rack middleware which instantiates a postgres datastore, opens a pooled connection, starts a transaction and calls the next middleware.

require 'hyperion'
require 'hyperion/sql'

class DatastoreMiddleware

  def initialize(app)
    @app = app

  def call(env)
    connection_url = 'postgres://'
    Hyperion.with_datastore(:postgres, :connection_url => connection_url) do
      Hyperion::Sql.with_connection(connection_url) do
        Hyperion::Sql.transaction do


Each of these calls could, of course, be their own separate middlewares.

With brute force

This installs the datastore across all threads.

Hyperion.datastore = Hyperion.new_datastore(:postgres, options)

To uninstall

Hyperion.datastore = nil

This method is useful for when you do not have control of the main thread, such as desktop GUI applications. If you can bind the datastore once at high level in your application, that's ideal. Otherwise use the brute force technique.

Saving a value:{:kind => :foo}) #=> {:kind=>"foo", :key=>"<generated key>"}{:kind => :foo}, :value => :bar) #=> {:kind=>"foo", :value=>:bar, :key=>"<generated key>"}

Updating a value:

record ={:kind => :foo, :name => 'Sam'}), :name => 'John') #=> {:kind=>"foo", :name=>'John', :key=>"<generated key>"}

Loading a value:

# if you have a key...

# otherwise
Hyperion.find_by_kind(:dog) # returns all records with :kind of \"dog\"
Hyperion.find_by_kind(:dog, :filters => [[:name, '=', "Fido"]]) # returns all dogs whose name is Fido
Hyperion.find_by_kind(:dog, :filters => [[:age, '>', 2], [:age, '<', 5]]) # returns all dogs between the age of 2 and 5 (exclusive)
Hyperion.find_by_kind(:dog, :sorts => [[:name, :asc]]) # returns all dogs in alphebetical order of their name
Hyperion.find_by_kind(:dog, :sorts => [[:age, :desc], [:name, :asc]]) # returns all dogs ordered from oldest to youngest, and gos of the same age ordered by name
Hyperion.find_by_kind(:dog, :limit => 10) # returns upto 10 dogs in undefined order
Hyperion.find_by_kind(:dog, :sorts => [[:name, :asc]], :limit => 10) # returns upto the first 10 dogs in alphebetical order of their name
Hyperion.find_by_kind(:dog, :sorts => [[:name, :asc]], :limit => 10, :offset => 10) # returns the second set of 10 dogs in alphebetical order of their name

Filter operations and acceptable syntax:

"=" "eq"
"<" "lt"
"<=" "lte"
">" "gt"
">=" "gte"
"!=" "not"
"contains?" "contains" "in?" "in"

Sort orders and acceptable syntax:

:asc "asc" :ascending "ascending"
:desc "desc" :descending "descending"

Deleting a value:

# if you have a key...

# otherwise
Hyperion.delete_by_kind(:dog) # deletes all records with :kind of "dog"
Hyperion.delete_by_kind(:dog, :filters => [[:name, "=", "Fido"]]) # deletes all dogs whose name is Fido
Hyperion.delete_by_kind(:dog, :filters => [[:age, ">", 2], [:age, "<", 5]]) # deletes all dogs between the age of 2 and 5 (exclusive)


An entity is, in essence, configuration. By defining an entity, you are telling Hyperion how to save ('pack') and load ('unpack') a specific kind. Defining entites is not required or necessary in some cases. Hyperion will work just fine without them. However, they offer some very clear advantages.


Only fields defined in the entity are allowed into the datastore.

Hyperion.defentity(:whitelisted) do |kind|
end :whitelisted, age: 23, hair_color: 'brown')
#=> {:kind=>"whitelisted", :key=><generated_key>, :age=>23}


Default values can be assigned to fields.

Hyperion.defentity(:defaulted) do |kind|
  kind.field(:age, :default => 25)
end :defaulted, name: 'Myles')
#=> {:kind=>"whitelisted", :key=><generated_key>, :name=>"Myles", :age=>25}


Packers tell Hyperion how to treat certain fields when they are being saved.

Hyperion.defentity(:packed) do |kind|
  kind.field(:age, :packer => lambda {|value| value.to_i})
end :packed, age: '25')
#=> {:kind=>"packed", :key=><generated_key>, :age=>25}

A packer must be callable, i.e. respond to 'call'. The current value of a field will be passed to the packer, and the packer returns a potentially transformed value.


Unpackers tell Hyperion how to treat certain fields when they are being loaded.

Hyperion.defentity(:unpacked) do |kind|
  kind.field(:age, :packer => lambda {|age| age.to_i}, :unpacker => lambda {|age| age.to_s})
end :unpacked, age: '25')
#=> {:kind=>"unpacked", :key=><generated_key>, :age=>"25"}

An unpacker must be callable, i.e. respond to 'call'. The current value of a field will be passed to the unpacker, and the unpacker returns a potentially transformed value.


Types are an easy way to share packers and unpackers between fields.

def to_int(value)
  if value

def to_str(value)
  if value

Hyperion.pack(Integer) {|value| to_int(value)}
Hyperion.unpack(Integer) {|value| to_int(value)}
Hyperion.pack(String) {|value| to_str(value)}
Hyperion.unpack(String) {|value| to_str(value)}

Hyperion.defentity(:typed) do |kind|
  kind.field(:age, :type => Integer)
  kind.field(:name, :type => String)
end :typed, age: '25', name: :a_symbol)
 => {:kind=>"typed", :key=><generated_key>, :age=>25, :name=>"a_symbol"}

If a type is specified as well as a custom packer or unpacker, the custom packer has priority.

Hyperion.defentity(:typed_2) do |kind|
  kind.field(:age, :type => Integer, :packer => lambda {|value| 2})
end :typed_2, age: '25')
#=> {:kind=>"typed_2", :key=><generated_key>, :age=>2}

Foreign Keys

In a traditional SQL database, you may have a schema that looks like this:


  • id
  • first_name
  • created_at
  • updated_at


  • id
  • user_id
  • created_at
  • updated_at

Since Hyperion presents every underlying datastore as a key-value store, configuring Hyperion to use this schema is a little tricky, but certainly possible.

This is what the corresponding defentity notation would be:

Hyperion.defentity(:users) do |kind|

Hyperion.defentity(:profiles) do |kind|
  kind.field(:user_key, :type => Hyperion::Types.foreign_key(:users), :db_name => :user_id)

myles = :users, :first_name => "Myles")
#=> {:kind=>"users", :key=>"46e5ebef8554422db90f7eb4d01be4c6", :email=>nil, :first_name=>"Myles", :last_name=>nil, :created_at=>2012-12-02 17:30:41 -0600, :updated_at=>nil}

# myles is stored in the users table as:
# | id | first_name | created_at | updated_at |
# | 1  | Myles      | <time>     | <time>     |

myles_profile = => :profiles, :user_key => myles[:key])
#=> {:kind=>"profiles", :user_key=>"46e5ebef8554422db90f7eb4d01be4c6", :key=>"356c89696b0d40f6a5754f225c01f7a9"}

# myles' profile is stored in the profiles table as:
# | id | user_id | created_at | updated_at |
# | 1  | 1       | <time>     | <time>     |

Using the Hyperion::Types.foreign_key helper, our foreign key references are stored in whatever way is conventional to the underlying datastore. In this example, the user_key field will be packed as an integer id, as stored in the user_id column.

If your schema requires foreign keys, ALWAYS USE THE FOREIGN KEY TYPE. If you do not, you will be storing generated keys instead of actual database ids. DO NOT DO THIS. If Hyperion changes the way it generates keys, all of your foreign key data will be useless.


Clone the master branch, and run all the tests:

git clone
cd hyperion-ruby

Make patches and submit them along with an issue (see below).


Post issues on the hyperion-ruby github project:


Copyright (C) 2012 8th Light, All Rights Reserved.

Distributed under the Eclipse Public License