Genghis - a MongoDB configuration and resilience
What is Genghis?
A configuration framework for mongoDB
A resilience framework when using MongoDB in replica sets
Genghis 1.2 is for use with MongoDB 1.6 or above; for MongoDB < 1.6, use Genghis 1.0.4
When invoked from rails, Genghis looks for a file called mongodb.yml in the RAILS_ROOT/config directory. the format of this file closely mimics that of the database.yml file you know well.
development: server: mongodb://localhost databases: paperclip : 'paperclip_files' mongo_mapper : 'mongo_mapper' connection_options: pool_size: 7 timeout: 2
In this case, we have a single mongodb server at localhost:27017. The database defines an alias for each actual mongo database in your mongodb server. You can then look up the databases by the alias later. The connection_options hash is passed directly to the mongo connection constructor.
If you are using replica sets, the configuration varies somewhat.
development: replica_set: - mongodb://replica1 - mongodb://replica2 databases: paperclip : 'paperclip_files' ... authorization: paperclip: username: pc password: secretpassword ... connection_options: pool_size: 5 slave_ok: true ... resilience_options: max_retries: 7 sleep_between_retries: 0.5
The servers are specified in an array, and most importantly for resilience the max_retries entry is specified in connection options. This specifies how many times Genghis will try to establish a connection to one of the servers if it detects a connection error. Anything listed under connection options is passed to ReplSetConnection or Connection.multi as options after the servers.
In rails, it's extremely simple to set up Genghis, simply include the gem and then require it. Then set it up in environmnet.rb
Genghis.environment = RAILS_ENV
Genghis manages connections so you don't have to. They are available through Genghis, so you can do the following:
MongoMapper.connection = Genghis.connection
Database names can then be used to configure mongo mapper or any other frameworks you have
MongoMapper.database = Genghis.databases['mongo_mapper'] or MongoMapper.database = Genghis.database[:mongo_mapper]
Similarly you can retrieve the actual mongo database
db = Genghis.database('paperclip') db.collections ...
While MongoDB provides impressive levels of stability and failover, its driver design leaves handling failover up to the implementer. This leaves your application subject to connection exceptions that can happen at any time, possibly littering your code with ugly and difficult to maintain reconnect logic. Genghis's resilience framework solves this for you.
To make an object resilient, you must first have a replica set of MongoDB servers. After you have that set up, you're ready to make your objects robust.
The following examples will assume you have a mongo_mapper object with the following definition:
class Foo < MongoMapper::Document key :bar, String key :baz, Integer end
First you must re-define your old object so that the application can't see it under the original name. I find the namespace Unsafe communicates my intention pretty well:
module Unsafe class Foo < MongoMapper::Document key :bar, String key :baz, Integer end end
Then you need to need to enlist Genghis's guardian class, which is a protective object proxy.
module Unsafe class Foo < MongoMapper::Document key :bar, String key :baz, Integer end end class Foo < Genghis::Guardian protects Unsafe::Foo end
That's it. You are now free to use the Foo class as you did before but now it has Genghis's guardian watching over its shoulder, protecting you from any connection related problems.
What happens on error
Let's say that while you are executing an update and a connection error occurs. Genghis's guardian realizes something has gone wrong and invalidates the current connection. It then waits sleep_between_retries seconds and then tries to make a new connection to the other server in the replica set. If that succeeds, it then re-tries the code that was executing when the failure occurs. It then keeps using this connection.
If the second connection fails, it then continues to the next server in the list. Genghis will cycle between the servers until it reaches the max_retries threshold, at which point it will raise a Mongo::ConnectionFailure exception (the same type that was originally thrown).