Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Newer
Older
100644 269 lines (195 sloc) 9.551 kb
281dcfd @schoefmax multi_db is a gem now.
authored
1 = multi_db
2
7558143 @schoefmax doc fixes
authored
3 =====-- This GEM was inspired by Rick Olson's "masochism"-Plugin
281dcfd @schoefmax multi_db is a gem now.
authored
4
5 multi_db uses a connection proxy, which sends read queries to slave databases,
6 and all write queries to the master database (Read/Write Split).
7 Within transactions, while executing ActiveRecord Observers and
8 within "with_master" blocks (see below), even read queries are sent to the
9 master database.
10
8874a15 @schoefmax some refactoring and first stab at thread-safety. See the "Changes in…
authored
11 === Important changes in 0.2.0
12
13 * As of this version, <tt>ActiveRecord::Base.connection</tt> does not return the
14 connection proxy by default anymore (therefore the jump to 0.2.0). Only models
15 inheriting from AR::B return the proxy, unless they are defined as master_models
16 (see below). If you want to access the connection proxy from AR::B directly,
17 use <tt>ActiveRecord::Base.connection_proxy</tt>.
18
19 * This version is the first attempt for thread-safety of this gem. <em>There might
20 still be some threading issues left!</em>. So please test your apps thoroughly
21 and report any issues you might encounter.
22
23 * <tt>CGI::Session::ActiveRecordStore::Session</tt> is now automatically registered
24 as a master model.
25
281dcfd @schoefmax multi_db is a gem now.
authored
26 === Caveats
27
0fa36a4 @lukeludwig modifications to work with Rails 3: reordered include of QueryCacheCo…
lukeludwig authored
28 * works with activerecord 2.1, 2.2, 2.3, and 3.0
281dcfd @schoefmax multi_db is a gem now.
authored
29
30 === Install
1fcf515 @KensoDev Some reformatting, to make it clearer
KensoDev authored
31 ==== Rails 2
9df0100 @KensoDev Fixed the documentation, the gem is hosted on RubyGems now, so you do…
KensoDev authored
32 gem install multi_db
281dcfd @schoefmax multi_db is a gem now.
authored
33
9df0100 @KensoDev Fixed the documentation, the gem is hosted on RubyGems now, so you do…
KensoDev authored
34 Then add this to your environment.rb:
35 config.gem 'multi_db', :lib => 'multi_db'
281dcfd @schoefmax multi_db is a gem now.
authored
36
1fcf515 @KensoDev Some reformatting, to make it clearer
KensoDev authored
37 ==== Rails 3
9df0100 @KensoDev Fixed the documentation, the gem is hosted on RubyGems now, so you do…
KensoDev authored
38 put this in your Gemfile
39 gem 'multi_db'
281dcfd @schoefmax multi_db is a gem now.
authored
40
41 === Setup
42
43 In your database.yml, add sections for the slaves, e.g.:
44
45 production: # that would be the master
46 adapter: mysql
47 database: myapp_production
48 username: root
49 password:
50 host: localhost
51
52 production_slave_database: # that would be a slave
53 adapter: mysql
54 database: myapp_production
55 username: root
56 password:
57 host: 10.0.0.2
58
95bef06 @schoefmax made it _really_ work outside of Rails
authored
59 production_slave_database_2: # another slave
281dcfd @schoefmax multi_db is a gem now.
authored
60 ...
95bef06 @schoefmax made it _really_ work outside of Rails
authored
61 production_slave_database_in_india: # yet another one
281dcfd @schoefmax multi_db is a gem now.
authored
62 ...
63
95bef06 @schoefmax made it _really_ work outside of Rails
authored
64 *NOTE*: multi_db identifies slave databases by looking for entries of the form
65 "<tt><environment>_slave_database<_optional_name></tt>". As a (useless) side effect you
66 get abstract classes named <tt>MultiDb::SlaveDatabaseInIndia</tt> etc.
67 The advantage of specifying the slaves explicitly, instead of the master, is that
68 you can use the same configuration file for scripts that don't use multi_db.
69 Also, when you decide to disable multi_db for some reason, you don't have to
70 swap hosts in your <tt>database.yml</tt> from master to slave (which is easy to forget...).
281dcfd @schoefmax multi_db is a gem now.
authored
71
72 To enable the proxy globally, add this to your environment.rb, or some file in
73 config/initializers:
74
75 MultiDb::ConnectionProxy.setup!
76
77 If you only want to enable it for specific environments, add this to
78 the corresponding file in config/environments:
79
80 config.after_initialize do
81 MultiDb::ConnectionProxy.setup!
82 end
83
95bef06 @schoefmax made it _really_ work outside of Rails
authored
84 In the development and test environments, you can use identical configurations
85 for master and slave connections. This can help you finding (some of the) issues
86 your application might have with a replicated database setup without actually having
87 one on your development machine.
88
2ac8233 @schoefmax added docs how to use with Phusion Passenger
authored
89 === Using with Phusion Passenger
90
91 With Passengers smart spawning method, child processes forked by the ApplicationSpawner
92 won't have the connection proxy set up properly.
93
94 To make it work, add this to your <tt>environment.rb</tt> or an initializer script
95 (e.g. <tt>config/initializers/connection_proxy.rb</tt>):
96
97 if defined?(PhusionPassenger)
98 PhusionPassenger.on_event(:starting_worker_process) do |forked|
99 if forked
100 # ... set MultiDb configuration options, if any ...
101 MultiDb::ConnectionProxy.setup!
102 end
103 end
104 else # not using passenger (e.g. development/testing)
105 # ... set MultiDb configuration options, if any ...
106 MultiDb::ConnectionProxy.setup!
107 end
108
109 Thanks to Nathan Esquenazi for testing this.
110
95bef06 @schoefmax made it _really_ work outside of Rails
authored
111 === Forcing the master for certain actions
112
113 Just add this to your controller:
114
8874a15 @schoefmax some refactoring and first stab at thread-safety. See the "Changes in…
authored
115 around_filter(:only => :foo_action) { |c,a| ActiveRecord::Base.connection_proxy.with_master { a.call } }
95bef06 @schoefmax made it _really_ work outside of Rails
authored
116
117 === Forcing the master for certain models
118
8874a15 @schoefmax some refactoring and first stab at thread-safety. See the "Changes in…
authored
119 In your environment.rb or an initializer, add this *before* the call to <tt>setup!</tt>:
120
121 MultiDb::ConnectionProxy.master_models = ['CGI::Session::ActiveRecordStore::Session', 'PaymentTransaction', ...]
122 MultiDb::ConnectionProxy.setup!
95bef06 @schoefmax made it _really_ work outside of Rails
authored
123
8874a15 @schoefmax some refactoring and first stab at thread-safety. See the "Changes in…
authored
124 *NOTE*: You cannot safely add more master_models after calling <tt>setup!</tt>.
95bef06 @schoefmax made it _really_ work outside of Rails
authored
125
59329d3 @schoefmax implemented 'sticky master' mode
authored
126 === Making one slave database sticky during a request
127
128 This can be useful to leverage database level query caching as all queries will
129 be sent to the same slave database during one web request.
130
131 To enable, add this to your environment.rb just before <tt>MultiDb::ConnectionProxy.setup!</tt>:
132
133 MultiDb::ConnectionProxy.sticky_slave = true
134
135 And add this to your ApplicationController:
136
137 after_filter { ActiveRecord::Base.connection_proxy.next_reader! }
138
a0ea45d @schoefmax fixes a typo, clearifies how not to use sticky_slave
authored
139 *NOTE*: It's not possible to toggle this mode in a running process, as the dynamically
140 generated methods will have the initially defined "stickyness" built in.
141
8698951 Adding docs to README
David Palm authored
142 === Using the weighted scheduler
143 The standard scheduler roundrobins queries to evenly to all slaves. This means that if you're using servers with different capacity (slower machines, some slaves receiving traffic from other apps etc) you might run into problems. The weighted scheduler tries to address this by assigning a weight attribute to each slave and distribute queries evenly among the server pool.
144
145 In your database.yml file add your weights like so:
146 test_slave_database_1:
147 <<: *creds
148 host: my.slavedb_1
149 weight: 1
150
151 test_slave_database_2:
152 <<: *creds
153 host: my.slavedb_2
154 weight: 10
155
156 The above configuration will lead to slavedb_2 to receive 9 times more queries than slavedb_1. Adding in a new slave with:
157 test_slave_database_3:
158 <<: *creds
159 host: my.slavedb_3
160 weight: 5
161
162 leads to a distribution of 1:10:5. For 100k queries the numbers could look like this:
163 Slave 1, with weight 1: 6302 queries
164 Slave 2, with weight 10: 62764 queries
165 Slave 3, with weight 5: 30934 queries
166
167 The weighted scheduler does not guarantee that the same slave will not receive two queries in a row. We feel this is not an issue, or rather, that such a guarantee doesn't help much as it's the complexity of the queries rather than the number that creates problems.
168
169 If no weight param is given for a slave, a weight of 1 is assumed. A weight of 0 is caught and silently transformed into a weight of 1.
170
95bef06 @schoefmax made it _really_ work outside of Rails
authored
171 === Usage outside of Rails
172
173 You can use multi_db together with other framworks or in standalone scripts.
174 Example:
175
176 require 'rubygems'
177 require 'active_record'
178 require 'multi_db'
179
180 ActiveRecord::Base.logger = Logger.new(STDOUT)
181 ActiveRecord::Base.configurations = {
182 'development' => {
183 'adapter' => 'mysql',
184 'host' => 'localhost',
185 'username' => 'root',
186 'database' => 'multi_db_test'
187 },
188 'development_slave_database' => {
189 'adapter' => 'mysql',
190 'host' => 'localhost',
191 'username' => 'root',
192 'database' => 'multi_db_test'
193 }
194 }
195 ActiveRecord::Base.establish_connection :development
196 MultiDb::ConnectionProxy.setup!
197
198 class MyModel < ActiveRecord::Base
199 # ...
200 end
201
202 # ...
203
204 Note that the configurations hash should contain strings as keys instead of symbols.
281dcfd @schoefmax multi_db is a gem now.
authored
205
206 === Differences to "masochism":
207
95bef06 @schoefmax made it _really_ work outside of Rails
authored
208 * Supports multiple slave databases (round robin)
209 * It sends everything except "select ..." queries to the master, instead of
210 sending only specific things to the master and anything "else" to the slave.
211 This avoids accidential writes to the master when there are API changes in
212 ActiveRecord which haven't been picked up by multi_db yet.
213 Note that this behaviour will also always send helper methods like "+quote+" or
214 "<tt>add_limit!</tt>" to the master connection object, which doesn't add any
215 more load on the master, as these methods don't communicate with the db server
216 itself.
217 * It uses its own query cache as the slave's cache isn't emptied when there are
218 changes on the master
281dcfd @schoefmax multi_db is a gem now.
authored
219 * It supports immediate failover for slave connections
220 * It will wait some time before trying to query a failed slave database again
95bef06 @schoefmax made it _really_ work outside of Rails
authored
221 * It supports nesting "with_master"-blocks, without unexpectedly switching you
222 back to the slave again
223 * It schedules a reconnect of the master connection if statements fail there.
224 This might help with HA setups using virtual IPs (a test setup would be nice
225 to verify this)
226 * You specify slave databases in the configuration instead of specifying an extra
8874a15 @schoefmax some refactoring and first stab at thread-safety. See the "Changes in…
authored
227 master database. This makes disabling or removing multi_db less dangerous
228 (Update: Recent versions of masochism support this, too).
95bef06 @schoefmax made it _really_ work outside of Rails
authored
229 * There are no <tt>set_to_master!</tt> and <tt>set_to_slave!</tt> methods, just
230 <tt>with_master(&block)</tt>
231 * All proxied methods are dynamically generated for better performance
281dcfd @schoefmax multi_db is a gem now.
authored
232
f0528ec @schoefmax see also section in readme
authored
233 === See also
234
235 ==== Masochism
236
237 The original plugin:
238
239 * http://github.com/technoweenie/masochism
240
241 ==== DataFabric
242
243 A solution by FiveRuns, also based on masochism but without the "nested with_master"-issue,
244 threadsafe and allows sharding of data.
245
246 * http://github.com/fiveruns/data_fabric
247
320a305 @schoefmax allow_concurrency doesn't seem to cause problems with the rails 2.2 p…
authored
248 === Contributors
249
4e7218f @schoefmax bump version
authored
250 * David Palm http://github.com/dvdplm
95bef06 @schoefmax made it _really_ work outside of Rails
authored
251 * Matt Conway http://github.com/wr0ngway
252 * Matthias Marshall http://github.com/webops
320a305 @schoefmax allow_concurrency doesn't seem to cause problems with the rails 2.2 p…
authored
253
f0528ec @schoefmax see also section in readme
authored
254 === Ideas
255
256 See: http://github.com/schoefmax/multi_db/wikis/home
257
281dcfd @schoefmax multi_db is a gem now.
authored
258 === Running specs
259
260 If you haven't already, install the rspec gem, then create an empty database
261 called "multi_db_test" (you might want to tweak the spec/config/database.yml).
262 From the plugin directory, run:
263
f366534 @lukeludwig bringin rspec tests up to date with rails3 and the latest rspec
lukeludwig authored
264 rspec spec
281dcfd @schoefmax multi_db is a gem now.
authored
265
266
320a305 @schoefmax allow_concurrency doesn't seem to cause problems with the rails 2.2 p…
authored
267 Copyright (c) 2008, Max Schoefmann <max (a) pragmatic-it de>
281dcfd @schoefmax multi_db is a gem now.
authored
268 Released under the MIT license
Something went wrong with that request. Please try again.