Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 267 lines (192 sloc) 9.639 kb
281dcfd @schoefmax multi_db is a gem now.
authored
1 = multi_db
2
7558143 @schoefmax doc fixes
authored
3 =====-- This GEM was inspired by Rick Olson's "masochism"-Plugin
281dcfd @schoefmax multi_db is a gem now.
authored
4
5 multi_db uses a connection proxy, which sends read queries to slave databases,
6 and all write queries to the master database (Read/Write Split).
7 Within transactions, while executing ActiveRecord Observers and
8 within "with_master" blocks (see below), even read queries are sent to the
9 master database.
10
8874a15 @schoefmax some refactoring and first stab at thread-safety. See the "Changes in…
authored
11 === Important changes in 0.2.0
12
13 * As of this version, <tt>ActiveRecord::Base.connection</tt> does not return the
14 connection proxy by default anymore (therefore the jump to 0.2.0). Only models
15 inheriting from AR::B return the proxy, unless they are defined as master_models
16 (see below). If you want to access the connection proxy from AR::B directly,
17 use <tt>ActiveRecord::Base.connection_proxy</tt>.
18
19 * This version is the first attempt for thread-safety of this gem. <em>There might
20 still be some threading issues left!</em>. So please test your apps thoroughly
21 and report any issues you might encounter.
22
23 * <tt>CGI::Session::ActiveRecordStore::Session</tt> is now automatically registered
24 as a master model.
25
281dcfd @schoefmax multi_db is a gem now.
authored
26 === Caveats
27
0fa36a4 @lukeludwig modifications to work with Rails 3: reordered include of QueryCacheCo…
lukeludwig authored
28 * works with activerecord 2.1, 2.2, 2.3, and 3.0
281dcfd @schoefmax multi_db is a gem now.
authored
29
30 === Install
31
bf01a3e @schoefmax now using gem version of tlattr_accessors
authored
32 gem sources --add http://gems.github.com # only if you haven't already added github
33 gem install schoefmax-multi_db
281dcfd @schoefmax multi_db is a gem now.
authored
34
95bef06 @schoefmax made it _really_ work outside of Rails
authored
35 When using Rails, add this to your environment.rb:
281dcfd @schoefmax multi_db is a gem now.
authored
36
37 config.gem 'schoefmax-multi_db', :lib => 'multi_db', :source => 'http://gems.github.com'
38
39 === Setup
40
41 In your database.yml, add sections for the slaves, e.g.:
42
43 production: # that would be the master
44 adapter: mysql
45 database: myapp_production
46 username: root
47 password:
48 host: localhost
49
50 production_slave_database: # that would be a slave
51 adapter: mysql
52 database: myapp_production
53 username: root
54 password:
55 host: 10.0.0.2
56
95bef06 @schoefmax made it _really_ work outside of Rails
authored
57 production_slave_database_2: # another slave
281dcfd @schoefmax multi_db is a gem now.
authored
58 ...
95bef06 @schoefmax made it _really_ work outside of Rails
authored
59 production_slave_database_in_india: # yet another one
281dcfd @schoefmax multi_db is a gem now.
authored
60 ...
61
95bef06 @schoefmax made it _really_ work outside of Rails
authored
62 *NOTE*: multi_db identifies slave databases by looking for entries of the form
63 "<tt><environment>_slave_database<_optional_name></tt>". As a (useless) side effect you
64 get abstract classes named <tt>MultiDb::SlaveDatabaseInIndia</tt> etc.
65 The advantage of specifying the slaves explicitly, instead of the master, is that
66 you can use the same configuration file for scripts that don't use multi_db.
67 Also, when you decide to disable multi_db for some reason, you don't have to
68 swap hosts in your <tt>database.yml</tt> from master to slave (which is easy to forget...).
281dcfd @schoefmax multi_db is a gem now.
authored
69
70 To enable the proxy globally, add this to your environment.rb, or some file in
71 config/initializers:
72
73 MultiDb::ConnectionProxy.setup!
74
75 If you only want to enable it for specific environments, add this to
76 the corresponding file in config/environments:
77
78 config.after_initialize do
79 MultiDb::ConnectionProxy.setup!
80 end
81
95bef06 @schoefmax made it _really_ work outside of Rails
authored
82 In the development and test environments, you can use identical configurations
83 for master and slave connections. This can help you finding (some of the) issues
84 your application might have with a replicated database setup without actually having
85 one on your development machine.
86
2ac8233 @schoefmax added docs how to use with Phusion Passenger
authored
87 === Using with Phusion Passenger
88
89 With Passengers smart spawning method, child processes forked by the ApplicationSpawner
90 won't have the connection proxy set up properly.
91
92 To make it work, add this to your <tt>environment.rb</tt> or an initializer script
93 (e.g. <tt>config/initializers/connection_proxy.rb</tt>):
94
95 if defined?(PhusionPassenger)
96 PhusionPassenger.on_event(:starting_worker_process) do |forked|
97 if forked
98 # ... set MultiDb configuration options, if any ...
99 MultiDb::ConnectionProxy.setup!
100 end
101 end
102 else # not using passenger (e.g. development/testing)
103 # ... set MultiDb configuration options, if any ...
104 MultiDb::ConnectionProxy.setup!
105 end
106
107 Thanks to Nathan Esquenazi for testing this.
108
95bef06 @schoefmax made it _really_ work outside of Rails
authored
109 === Forcing the master for certain actions
110
111 Just add this to your controller:
112
8874a15 @schoefmax some refactoring and first stab at thread-safety. See the "Changes in…
authored
113 around_filter(:only => :foo_action) { |c,a| ActiveRecord::Base.connection_proxy.with_master { a.call } }
95bef06 @schoefmax made it _really_ work outside of Rails
authored
114
115 === Forcing the master for certain models
116
8874a15 @schoefmax some refactoring and first stab at thread-safety. See the "Changes in…
authored
117 In your environment.rb or an initializer, add this *before* the call to <tt>setup!</tt>:
118
119 MultiDb::ConnectionProxy.master_models = ['CGI::Session::ActiveRecordStore::Session', 'PaymentTransaction', ...]
120 MultiDb::ConnectionProxy.setup!
95bef06 @schoefmax made it _really_ work outside of Rails
authored
121
8874a15 @schoefmax some refactoring and first stab at thread-safety. See the "Changes in…
authored
122 *NOTE*: You cannot safely add more master_models after calling <tt>setup!</tt>.
95bef06 @schoefmax made it _really_ work outside of Rails
authored
123
59329d3 @schoefmax implemented 'sticky master' mode
authored
124 === Making one slave database sticky during a request
125
126 This can be useful to leverage database level query caching as all queries will
127 be sent to the same slave database during one web request.
128
129 To enable, add this to your environment.rb just before <tt>MultiDb::ConnectionProxy.setup!</tt>:
130
131 MultiDb::ConnectionProxy.sticky_slave = true
132
133 And add this to your ApplicationController:
134
135 after_filter { ActiveRecord::Base.connection_proxy.next_reader! }
136
a0ea45d @schoefmax fixes a typo, clearifies how not to use sticky_slave
authored
137 *NOTE*: It's not possible to toggle this mode in a running process, as the dynamically
138 generated methods will have the initially defined "stickyness" built in.
139
8698951 Adding docs to README
David Palm authored
140 === Using the weighted scheduler
141 The standard scheduler roundrobins queries to evenly to all slaves. This means that if you're using servers with different capacity (slower machines, some slaves receiving traffic from other apps etc) you might run into problems. The weighted scheduler tries to address this by assigning a weight attribute to each slave and distribute queries evenly among the server pool.
142
143 In your database.yml file add your weights like so:
144 test_slave_database_1:
145 <<: *creds
146 host: my.slavedb_1
147 weight: 1
148
149 test_slave_database_2:
150 <<: *creds
151 host: my.slavedb_2
152 weight: 10
153
154 The above configuration will lead to slavedb_2 to receive 9 times more queries than slavedb_1. Adding in a new slave with:
155 test_slave_database_3:
156 <<: *creds
157 host: my.slavedb_3
158 weight: 5
159
160 leads to a distribution of 1:10:5. For 100k queries the numbers could look like this:
161 Slave 1, with weight 1: 6302 queries
162 Slave 2, with weight 10: 62764 queries
163 Slave 3, with weight 5: 30934 queries
164
165 The weighted scheduler does not guarantee that the same slave will not receive two queries in a row. We feel this is not an issue, or rather, that such a guarantee doesn't help much as it's the complexity of the queries rather than the number that creates problems.
166
167 If no weight param is given for a slave, a weight of 1 is assumed. A weight of 0 is caught and silently transformed into a weight of 1.
168
95bef06 @schoefmax made it _really_ work outside of Rails
authored
169 === Usage outside of Rails
170
171 You can use multi_db together with other framworks or in standalone scripts.
172 Example:
173
174 require 'rubygems'
175 require 'active_record'
176 require 'multi_db'
177
178 ActiveRecord::Base.logger = Logger.new(STDOUT)
179 ActiveRecord::Base.configurations = {
180 'development' => {
181 'adapter' => 'mysql',
182 'host' => 'localhost',
183 'username' => 'root',
184 'database' => 'multi_db_test'
185 },
186 'development_slave_database' => {
187 'adapter' => 'mysql',
188 'host' => 'localhost',
189 'username' => 'root',
190 'database' => 'multi_db_test'
191 }
192 }
193 ActiveRecord::Base.establish_connection :development
194 MultiDb::ConnectionProxy.setup!
195
196 class MyModel < ActiveRecord::Base
197 # ...
198 end
199
200 # ...
201
202 Note that the configurations hash should contain strings as keys instead of symbols.
281dcfd @schoefmax multi_db is a gem now.
authored
203
204 === Differences to "masochism":
205
95bef06 @schoefmax made it _really_ work outside of Rails
authored
206 * Supports multiple slave databases (round robin)
207 * It sends everything except "select ..." queries to the master, instead of
208 sending only specific things to the master and anything "else" to the slave.
209 This avoids accidential writes to the master when there are API changes in
210 ActiveRecord which haven't been picked up by multi_db yet.
211 Note that this behaviour will also always send helper methods like "+quote+" or
212 "<tt>add_limit!</tt>" to the master connection object, which doesn't add any
213 more load on the master, as these methods don't communicate with the db server
214 itself.
215 * It uses its own query cache as the slave's cache isn't emptied when there are
216 changes on the master
281dcfd @schoefmax multi_db is a gem now.
authored
217 * It supports immediate failover for slave connections
218 * It will wait some time before trying to query a failed slave database again
95bef06 @schoefmax made it _really_ work outside of Rails
authored
219 * It supports nesting "with_master"-blocks, without unexpectedly switching you
220 back to the slave again
221 * It schedules a reconnect of the master connection if statements fail there.
222 This might help with HA setups using virtual IPs (a test setup would be nice
223 to verify this)
224 * You specify slave databases in the configuration instead of specifying an extra
8874a15 @schoefmax some refactoring and first stab at thread-safety. See the "Changes in…
authored
225 master database. This makes disabling or removing multi_db less dangerous
226 (Update: Recent versions of masochism support this, too).
95bef06 @schoefmax made it _really_ work outside of Rails
authored
227 * There are no <tt>set_to_master!</tt> and <tt>set_to_slave!</tt> methods, just
228 <tt>with_master(&block)</tt>
229 * All proxied methods are dynamically generated for better performance
281dcfd @schoefmax multi_db is a gem now.
authored
230
f0528ec @schoefmax see also section in readme
authored
231 === See also
232
233 ==== Masochism
234
235 The original plugin:
236
237 * http://github.com/technoweenie/masochism
238
239 ==== DataFabric
240
241 A solution by FiveRuns, also based on masochism but without the "nested with_master"-issue,
242 threadsafe and allows sharding of data.
243
244 * http://github.com/fiveruns/data_fabric
245
320a305 @schoefmax allow_concurrency doesn't seem to cause problems with the rails 2.2 p…
authored
246 === Contributors
247
4e7218f @schoefmax bump version
authored
248 * David Palm http://github.com/dvdplm
95bef06 @schoefmax made it _really_ work outside of Rails
authored
249 * Matt Conway http://github.com/wr0ngway
250 * Matthias Marshall http://github.com/webops
320a305 @schoefmax allow_concurrency doesn't seem to cause problems with the rails 2.2 p…
authored
251
f0528ec @schoefmax see also section in readme
authored
252 === Ideas
253
254 See: http://github.com/schoefmax/multi_db/wikis/home
255
281dcfd @schoefmax multi_db is a gem now.
authored
256 === Running specs
257
258 If you haven't already, install the rspec gem, then create an empty database
259 called "multi_db_test" (you might want to tweak the spec/config/database.yml).
260 From the plugin directory, run:
261
262 spec spec
263
264
320a305 @schoefmax allow_concurrency doesn't seem to cause problems with the rails 2.2 p…
authored
265 Copyright (c) 2008, Max Schoefmann <max (a) pragmatic-it de>
281dcfd @schoefmax multi_db is a gem now.
authored
266 Released under the MIT license
Something went wrong with that request. Please try again.