Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Newer
Older
100644 326 lines (245 sloc) 11.519 kb
88249fa @tarcieri Update logo URL
tarcieri authored
1 ![DCell](https://github.com/celluloid/dcell/raw/master/logo.png)
7f42c76 Some basic project configuration
Tony Arcieri authored
2 =====
013fd28 @tarcieri Celluloidify ALL THE URLS
tarcieri authored
3 [![Build Status](http://travis-ci.org/celluloid/dcell.png)](http://travis-ci.org/celluloid/dcell)
40c2a7a @tarcieri Dependency status
tarcieri authored
4 [![Dependency Status](https://gemnasium.com/celluloid/dcell.png)](https://gemnasium.com/celluloid/dcell)
d831c17 @tarcieri Build status and Jobs quote
tarcieri authored
5
6 > "Objects can message objects transparently that live on other machines
7 > over the network, and you don't have to worry about the networking gunk,
8 > and you don't have to worry about finding them, and you don't have to
9 > worry about anything. It's just as if you messaged an object that's
10 > right next door."
11 > _--Steve Jobs describing the NeXT Portable Distributed Object system_
7f42c76 Some basic project configuration
Tony Arcieri authored
12
ca80dc7 @tarcieri A real README
tarcieri authored
13 DCell is a simple and easy way to build distributed applications in Ruby.
14 Somewhat similar to DRb, DCell lets you easily expose Ruby objects as network
15 services, and call them remotely just like you would any other Ruby object.
16 However, unlike DRb all objects in the system are concurrent. You can create
17 and register several available services on a given node, obtain handles to
18 them, and easily pass these handles around the network just like any other
19 objects.
20
8bef0d2 @tarcieri zomg README links
tarcieri authored
21 DCell is a distributed extension to [Celluloid][celluloid], which provides
22 concurrent objects for Ruby with many of the features of Erlang, such as the
23 ability to supervise objects and restart them when they crash, and also link to
ca80dc7 @tarcieri A real README
tarcieri authored
24 other objects and receive event notifications of when they crash. This makes
25 it easier to build robust, fault-tolerant distributed systems.
26
8bef0d2 @tarcieri zomg README links
tarcieri authored
27 DCell uses the [0MQ][zeromq] messaging protocol which provides a robust,
28 fault-tolerant brokerless transport for asynchronous messages sent between
29 nodes. DCell is built on top of the [Celluloid::ZMQ][celluloid-zmq] library,
30 which provides a Celluloid-oriented wrapper around the underlying
31 [ffi-rzmq][ffi-rzmq] library.
7f42c76 Some basic project configuration
Tony Arcieri authored
32
8bef0d2 @tarcieri zomg README links
tarcieri authored
33 Like DCell? [Join the Celluloid Google Group][googlegroup]
34
35 [celluloid]: http://celluloid.io/
36 [zeromq]: http://www.zeromq.org/
37 [celluloid-zmq]: https://github.com/celluloid/celluloid-zmq
38 [ffi-rzmq]: https://github.com/chuckremes/ffi-rzmq
39 [googlegroup]: http://groups.google.com/group/celluloid-ruby
536e9b6 @tarcieri Note about Celluloid google group
tarcieri authored
40
6c4cb9f @tarcieri Is it good? Yes.
tarcieri authored
41 ### Is It Good?
42
43 Yes.
44
45 ### Is It "Production Ready™"?
46
47 Not entirely, but eager early adopters are welcome!
48
347ddd4 @tarcieri Supported platforms
tarcieri authored
49 Supported Platforms
50 -------------------
51
5ae2c4d @danielhopkins Upgrade for jRuby 1.7
danielhopkins authored
52 DCell works on Ruby 1.9.2/1.9.3, JRuby 1.6 (in 1.9 mode), JRuby 1.7, and Rubinius 2.0.
347ddd4 @tarcieri Supported platforms
tarcieri authored
53
5ae2c4d @danielhopkins Upgrade for jRuby 1.7
danielhopkins authored
54 To use JRuby 1.6 in 1.9 mode, you'll need to pass the "--1.9" command line
347ddd4 @tarcieri Supported platforms
tarcieri authored
55 option to the JRuby executable, or set the "JRUBY_OPTS=--1.9" environment
56 variable:
57
58 export JRUBY_OPTS=--1.9
59
60 (Note: I'd recommend putting the above in your .bashrc/.zshrc/etc in
61 general. 1.9 is the future, time to embrace it)
62
5ae2c4d @danielhopkins Upgrade for jRuby 1.7
danielhopkins authored
63 To use JRuby 1.7 in 1.9 mode...just use it :)
64
347ddd4 @tarcieri Supported platforms
tarcieri authored
65 Celluloid works on Rubinius in either 1.8 or 1.9 mode.
66
67 All components, including the 0MQ bindings, Redis, and Zookeeper adapters
68 are all certified to work on the above platforms. The 0MQ binding is FFI.
69 The Redis adapter is pure Ruby. The Zookeeper adapter uses an MRI-style
70 native extension but also supplies a pure-Java backend for JRuby.
71
7f42c76 Some basic project configuration
Tony Arcieri authored
72 Prerequisites
73 -------------
74
75 DCell requires 0MQ. On OS X, this is available through Homebrew by running:
76
a392ca4 @tarcieri Zookeeper cleanups
tarcieri authored
77 brew install zeromq
f32c59e Download and install Zookeeper
Tony Arcieri authored
78
1801489 @jhosteny Doc update.
jhosteny authored
79 DCell keeps the state of all global configuration data
1725c5f @tarcieri Cassandra info fixups
tarcieri authored
80 in a service it calls the "registry". DCell supports any of the following for
81 use as registries:
f32c59e Download and install Zookeeper
Tony Arcieri authored
82
1801489 @jhosteny Doc update.
jhosteny authored
83 * **Gossip**: node connection info is tracked via a native gossip protocol.
84 This adapter uses the same protocol to distribute data lazily to all
85 connected nodes. It uses version vectors to determine update ordering.
86 Be forewarned: when updates are found to be concurrent, one is arbitrarily
87 dropped. Furthermore, this may not be a good choice if you have many nodes
88 or a lot of global data, since each piece of data has a version vector
89 (which contains a version for every peer node) at every node. **This adapter
90 is experimental**.
91
1725c5f @tarcieri Cassandra info fixups
tarcieri authored
92 * **Redis**: a persistent data structures server. It's simple and easy to use
93 for development and prototyping, but lacks a good distribution story.
56c91fb @tarcieri Better descriptions of Redis and Zookeeper
tarcieri authored
94
1725c5f @tarcieri Cassandra info fixups
tarcieri authored
95 * **Zookeeper**: Zookeeper is a high-performance coordination service for
96 distributed applications. It exposes common services such as naming,
97 configuration management, synchronization, and group management.
56c91fb @tarcieri Better descriptions of Redis and Zookeeper
tarcieri authored
98 Unfortunately, it has slightly more annoying client-side dependencies and is
99 more difficult to deploy than Redis.
f048bb4 @tarcieri Redis rationale
tarcieri authored
100
1725c5f @tarcieri Cassandra info fixups
tarcieri authored
101 * **Cassandra**: a distributed database with no single points of
eec34ce Add cassandra rake task & README entry.
Al Tobey authored
102 failure and can store huge amounts of data. Setup requires creating a
103 keyspace and defining a single column family before staring DCell. The
104 Cassandra backend defaults to a keyspace/CF both named "dcell". There
105 are two rows, "nodes" and "globals" each with one column per entry.
106
107 You may pick any of these services to use as DCell's registry. The
f048bb4 @tarcieri Redis rationale
tarcieri authored
108 default is Redis.
576d2d2 @tarcieri Use Redis as the default registry adapter
tarcieri authored
109
110 To install a local copy of Redis on OS X with Homebrew, run:
111
112 brew install redis
113
114 To install a local copy Zookeeper for testing purposes, run:
a392ca4 @tarcieri Zookeeper cleanups
tarcieri authored
115
116 rake zookeeper:install
117
118 and to start it run:
119
120 rake zookeeper:start
121
eec34ce Add cassandra rake task & README entry.
Al Tobey authored
122 To install a local copy Apache Cassandra for testing purposes, run:
123
124 rake cassandra:install
125 rake cassandra:start
126
f537b08 @tarcieri Split apart configuration and usage in the README
tarcieri authored
127 Configuration
128 -------------
ca80dc7 @tarcieri A real README
tarcieri authored
129
f537b08 @tarcieri Split apart configuration and usage in the README
tarcieri authored
130 The simplest way to configure and start DCell is with the following:
ca80dc7 @tarcieri A real README
tarcieri authored
131
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
132 ```ruby
133 require 'dcell'
ca80dc7 @tarcieri A real README
tarcieri authored
134
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
135 DCell.start
136 ```
ca80dc7 @tarcieri A real README
tarcieri authored
137
f537b08 @tarcieri Split apart configuration and usage in the README
tarcieri authored
138 This configures DCell with all the default options, however there are many
139 options you can override, e.g.:
ca80dc7 @tarcieri A real README
tarcieri authored
140
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
141 ```ruby
142 DCell.start :id => "node42", :addr => "tcp://127.0.0.1:2042"
143 ```
f537b08 @tarcieri Split apart configuration and usage in the README
tarcieri authored
144
145 DCell identifies each node with a unique node ID, that defaults to your
146 hostname. Each node needs to be reachable over 0MQ, and the addr option
147 specifies the 0MQ address where the host can be reached. When giving a tcp://
148 URL, you *must* specify an IP address and not a hostname.
ca80dc7 @tarcieri A real README
tarcieri authored
149
f9d385c @jhosteny Doc update.
jhosteny authored
150 To join a cluster you'll need to provide the location of the unique node id of
151 the directory server. This can be done through the "directory" configuration
152 key:
153
154 ```ruby
155 DCell.start :id => "node24", :addr => "tcp://127.0.0.1:2042",
156 :directory => {
157 :id => 'node42',
158 :addr => 'tcp://127.0.0.1:2042'
159 }
160 ```
161
162 To use the registry for global data distribution, you'll need to provide the
163 location of the registry server. This can be done through the "registry"
164 configuration key:
f37aa0b @tarcieri Document registry configuration w\ Redis
tarcieri authored
165
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
166 ```ruby
167 DCell.start :id => "node24", :addr => "tcp://127.0.0.1:2042",
168 :registry => {
169 :adapter => 'redis',
170 :host => 'mycluster.example.org',
171 :port => 6379
172 }
173 ```
f37aa0b @tarcieri Document registry configuration w\ Redis
tarcieri authored
174
175 When configuring DCell to use Redis, use the following options:
176
5335a82 @jhosteny Doc update.
jhosteny authored
177 - **adapter**: "redis" (*optional, alternatively "zk", "moneta", "cassandra" or "gossip"*)
f37aa0b @tarcieri Document registry configuration w\ Redis
tarcieri authored
178 - **host**: hostname or IP address of the Redis server (*optional, default localhost*)
179 - **port**: port of the Redis server (*optional, default 6379*)
180 - **password**: password to the Redis server (*optional*)
181
f537b08 @tarcieri Split apart configuration and usage in the README
tarcieri authored
182 Usage
183 -----
ca80dc7 @tarcieri A real README
tarcieri authored
184
f537b08 @tarcieri Split apart configuration and usage in the README
tarcieri authored
185 You've now configured a single node in a DCell cluster. You can obtain the
186 DCell::Node object representing the local node by calling DCell.me:
187
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
188 ```ruby
189 >> DCell.start
190 => #<Celluloid::Supervisor(DCell::Application):0xed6>
191 >> DCell.me
192 => #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:7777">
193 ```
ca80dc7 @tarcieri A real README
tarcieri authored
194
195 DCell::Node objects are the entry point for locating actors on the system.
196 DCell.me returns the local node. Other nodes can be obtained by their
197 node IDs:
198
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
199 ```ruby
200 >> node = DCell::Node["cryptosphere.local"]
201 => #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:7777">
202 ```
ca80dc7 @tarcieri A real README
tarcieri authored
203
f26cd9e @tarcieri DCell::Node.all
tarcieri authored
204 DCell::Node.all returns all connected nodes in the cluster:
205
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
206 ```ruby
207 >> DCell::Node.all
208 => [#<DCell::Node[test_node] @addr="tcp://127.0.0.1:21264">, #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:7777">]
209 ```
f26cd9e @tarcieri DCell::Node.all
tarcieri authored
210
f537b08 @tarcieri Split apart configuration and usage in the README
tarcieri authored
211 DCell::Node is a Ruby Enumerable. You can iterate across all nodes with
212 DCell::Node.each.
213
f26cd9e @tarcieri DCell::Node.all
tarcieri authored
214 Once you've obtained a node, you can look up services it exports and call them
215 just like you'd invoke methods on any other Ruby object:
ca80dc7 @tarcieri A real README
tarcieri authored
216
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
217 ```ruby
218 >> node = DCell::Node["cryptosphere.local"]
219 => #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:7777">
220 >> time_server = node[:time_server]
221 => #<Celluloid::Actor(TimeServer:0xee8)>
222 >> time_server.time
223 => "The time is: 2011-11-10 20:23:47 -0800"
224 ```
ca80dc7 @tarcieri A real README
tarcieri authored
225
519cd31 @tarcieri Add info on DCell::Node#all to the README
tarcieri authored
226 You can also find all available services on a node with DCell::Node#all:
227
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
228 ```ruby
229 >> node = DCell::Node["cryptosphere.local"]
230 => #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:7777">
231 >> node.all
232 => [:time_server]
233 ```
519cd31 @tarcieri Add info on DCell::Node#all to the README
tarcieri authored
234
ca80dc7 @tarcieri A real README
tarcieri authored
235 Registering Actors
236 ------------------
237
f048bb4 @tarcieri Redis rationale
tarcieri authored
238 All services exposed by DCell must take the form of registered Celluloid actors.
239 What follows is an extremely brief introduction to creating and registering
240 actors, but for more information, you should definitely [read the Celluloid
241 documentation](http://celluloid.github.com).
ca80dc7 @tarcieri A real README
tarcieri authored
242
243 DCell exposes all Celluloid actors you've registered directly onto the network.
244 The best way to register an actor is by supervising it. Below is an example of
245 how to create an actor and register it on the network:
246
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
247 ```ruby
248 class TimeServer
249 include Celluloid
ca80dc7 @tarcieri A real README
tarcieri authored
250
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
251 def time
252 "The time is: #{Time.now}"
253 end
254 end
255 ```
ca80dc7 @tarcieri A real README
tarcieri authored
256
257 Now that we've defined the TimeServer, we're going to supervise it and register
258 it in the local registry:
259
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
260 ```ruby
261 >> TimeServer.supervise_as :time_server
262 => #<Celluloid::Supervisor(TimeServer):0xee4>
263 ```
347ddd4 @tarcieri Supported platforms
tarcieri authored
264
ca80dc7 @tarcieri A real README
tarcieri authored
265 Supervising actors means that if they crash, they're automatically restarted
266 and registered under the same name. We can access registered actors by using
267 Celluloid::Actor#[]:
268
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
269 ```ruby
270 >> Celluloid::Actor[:time_server]
271 => #<Celluloid::Actor(TimeServer:0xee8)>
272 >> Celluloid::Actor[:time_server].time
273 => "The time is: 2011-11-10 20:17:48 -0800"
274 ```
ca80dc7 @tarcieri A real README
tarcieri authored
275
276 This same actor is now available using the DCell::Node#[] syntax:
277
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
278 ```ruby
279 >> node = DCell.me
280 => #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:1870">
281 >> node[:time_server].time
282 => "The time is: 2011-11-10 20:28:27 -0800"
283 ```
8481f99 @tarcieri Initial DCell::Global implementation
tarcieri authored
284
285 Globals
286 -------
287
288 DCell provides a registry global for storing configuration data and actors you
289 wish to publish globally to the entire cluster:
290
e30051b @tarcieri Syntax highlight all examples
tarcieri authored
291 ```ruby
292 >> actor = Celluloid::Actor[:dcell_server]
293 => #<Celluloid::Actor(DCell::Server:0xf2e) @addr="tcp://127.0.0.1:7777">
294 >> DCell::Global[:sweet_server] = actor
295 => #<Celluloid::Actor(DCell::Server:0xf2e) @addr="tcp://127.0.0.1:7777">
296 >> DCell::Global[:sweet_server]
297 => #<Celluloid::Actor(DCell::Server:0xf2e) @addr="tcp://127.0.0.1:7777">
298 ```
f393b9e @tarcieri What about DRb?
tarcieri authored
299
300 What about DRb?
301 ---------------
302
303 Ruby already has a distributed object system as part of its standard library:
304 DRb, which stands for Distributed Ruby. What's wrong with DRb? Why do we need
305 a new system? DRb has one major drawback: it's inherently synchronous. The
306 only thing you can do to an object is to make a method call, which sends a
307 remote object a message, executes the method, and returns a response.
308
309 Under the covers, DCell uses an asynchronous message protocol. As noted in the
310 last section, asynchronous messaging allows many more modes of messaging than
311 the standard reqeust/response pattern afforded by DRb. DCell also supports the
312 Erlang-style approach to fault-tolerance, advocating that actors shouldn't handle
313 errors but should crash and restart in a clean state. Linking to actors on remote
314 nodes can be used to detect these sorts of errors and have dependent actors
315 restart in a clean state.
316
317 By far the biggest difference between DCell and DRb is how the underlying
318 Celluloid framework has you think about the problem. Celluloid provides a useful
319 concurrent in-process messaging system in its own right without the distributed
5fb26f1 @tarcieri Reconfigure DCell to use the celluloid-zmq gem
tarcieri authored
320 components.
321
322 Copyright
323 ---------
324
5b7dbc7 @tarcieri 2012
tarcieri authored
325 Copyright (c) 2012 Tony Arcieri. See LICENSE.txt for further details.
Something went wrong with that request. Please try again.