Skip to content

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also .

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also .
...
  • 14 commits
  • 15 files changed
  • 0 commit comments
  • 2 contributors
View
12 CHANGES.md
@@ -1,11 +1,13 @@
-HEAD
-----
+0.10.0
+------
* DCell::Explorer provides a web UI with Reel
-* Info service
-* Use Celluloid.uuid for mailbox and call IDs
+* Info service at DCell::Node#[:info]
+* Distributed gossip protocol, now default adapter
+* Support for marshaling Celluloid::Futures
* Cassandra registry
-* celluloid-zmq split out into a separate gem
* Initial DCell::NodeManager
+* celluloid-zmq split out into a separate gem
+* Use Celluloid.uuid for mailbox and call IDs
0.9.0
-----
View
4 Gemfile
@@ -3,7 +3,9 @@ source "http://rubygems.org"
gem 'celluloid', :git => 'git://github.com/celluloid/celluloid'
gem 'celluloid-io', :git => 'git://github.com/celluloid/celluloid-io'
gem 'celluloid-zmq', :git => 'git://github.com/celluloid/celluloid-zmq'
-gem 'reel', :git => 'git://github.com/celluloid/reel'
+#gem 'reel', :git => 'git://github.com/celluloid/reel'
+
+gem 'jruby-openssl', :platform => :jruby
# Specify your gem's dependencies in dcell.gemspec
gemspec
View
275 README.md
@@ -30,6 +30,9 @@ nodes. DCell is built on top of the [Celluloid::ZMQ][celluloid-zmq] library,
which provides a Celluloid-oriented wrapper around the underlying
[ffi-rzmq][ffi-rzmq] library.
+[Please see the DCell Wiki](https://github.com/celluloid/dcell/wiki)
+for more detailed documentation and usage notes.
+
Like DCell? [Join the Celluloid Google Group][googlegroup]
[celluloid]: http://celluloid.io/
@@ -49,275 +52,13 @@ Not entirely, but eager early adopters are welcome!
Supported Platforms
-------------------
-DCell works on Ruby 1.9.2/1.9.3, JRuby 1.6 (in 1.9 mode), JRuby 1.7, and Rubinius 2.0.
-
-To use JRuby 1.6 in 1.9 mode, you'll need to pass the "--1.9" command line
-option to the JRuby executable, or set the "JRUBY_OPTS=--1.9" environment
-variable:
-
- export JRUBY_OPTS=--1.9
-
-(Note: I'd recommend putting the above in your .bashrc/.zshrc/etc in
-general. 1.9 is the future, time to embrace it)
-
-To use JRuby 1.7 in 1.9 mode...just use it :)
-
-Celluloid works on Rubinius in either 1.8 or 1.9 mode.
-
-All components, including the 0MQ bindings, Redis, and Zookeeper adapters
-are all certified to work on the above platforms. The 0MQ binding is FFI.
-The Redis adapter is pure Ruby. The Zookeeper adapter uses an MRI-style
-native extension but also supplies a pure-Java backend for JRuby.
-
-Prerequisites
--------------
-
-DCell requires 0MQ. On OS X, this is available through Homebrew by running:
-
- brew install zeromq
-
-DCell keeps the state of all global configuration data
-in a service it calls the "registry". DCell supports any of the following for
-use as registries:
-
-* **Gossip**: node connection info is tracked via a native gossip protocol.
- This adapter uses the same protocol to distribute data lazily to all
- connected nodes. It uses version vectors to determine update ordering.
- Be forewarned: when updates are found to be concurrent, one is arbitrarily
- dropped. Furthermore, this may not be a good choice if you have many nodes
- or a lot of global data, since each piece of data has a version vector
- (which contains a version for every peer node) at every node. **This adapter
- is experimental**.
-
-* **Redis**: a persistent data structures server. It's simple and easy to use
- for development and prototyping, but lacks a good distribution story.
-
-* **Zookeeper**: Zookeeper is a high-performance coordination service for
- distributed applications. It exposes common services such as naming,
- configuration management, synchronization, and group management.
- Unfortunately, it has slightly more annoying client-side dependencies and is
- more difficult to deploy than Redis.
-
-* **Cassandra**: a distributed database with no single points of
- failure and can store huge amounts of data. Setup requires creating a
- keyspace and defining a single column family before staring DCell. The
- Cassandra backend defaults to a keyspace/CF both named "dcell". There
- are two rows, "nodes" and "globals" each with one column per entry.
-
-You may pick any of these services to use as DCell's registry. The
-default is Redis.
-
-To install a local copy of Redis on OS X with Homebrew, run:
-
- brew install redis
-
-To install a local copy Zookeeper for testing purposes, run:
-
- rake zookeeper:install
-
-and to start it run:
-
- rake zookeeper:start
-
-To install a local copy Apache Cassandra for testing purposes, run:
-
- rake cassandra:install
- rake cassandra:start
-
-Configuration
--------------
-
-The simplest way to configure and start DCell is with the following:
-
-```ruby
-require 'dcell'
-
-DCell.start
-```
-
-This configures DCell with all the default options, however there are many
-options you can override, e.g.:
-
-```ruby
-DCell.start :id => "node42", :addr => "tcp://127.0.0.1:2042"
-```
-
-DCell identifies each node with a unique node ID, that defaults to your
-hostname. Each node needs to be reachable over 0MQ, and the addr option
-specifies the 0MQ address where the host can be reached. When giving a tcp://
-URL, you *must* specify an IP address and not a hostname.
-
-To join a cluster you'll need to provide the location of the unique node id of
-the directory server. This can be done through the "directory" configuration
-key:
-
-```ruby
-DCell.start :id => "node24", :addr => "tcp://127.0.0.1:2042",
- :directory => {
- :id => 'node42',
- :addr => 'tcp://127.0.0.1:2042'
- }
-```
-
-To use the registry for global data distribution, you'll need to provide the
-location of the registry server. This can be done through the "registry"
-configuration key:
-
-```ruby
-DCell.start :id => "node24", :addr => "tcp://127.0.0.1:2042",
- :registry => {
- :adapter => 'redis',
- :host => 'mycluster.example.org',
- :port => 6379
- }
-```
-
-When configuring DCell to use Redis, use the following options:
-
-- **adapter**: "redis" (*optional, alternatively "zk", "moneta", "cassandra" or "gossip"*)
-- **host**: hostname or IP address of the Redis server (*optional, default localhost*)
-- **port**: port of the Redis server (*optional, default 6379*)
-- **password**: password to the Redis server (*optional*)
-
-Usage
------
-
-You've now configured a single node in a DCell cluster. You can obtain the
-DCell::Node object representing the local node by calling DCell.me:
-
-```ruby
->> DCell.start
- => #<Celluloid::Supervisor(DCell::Application):0xed6>
->> DCell.me
- => #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:7777">
-```
-
-DCell::Node objects are the entry point for locating actors on the system.
-DCell.me returns the local node. Other nodes can be obtained by their
-node IDs:
-
-```ruby
->> node = DCell::Node["cryptosphere.local"]
- => #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:7777">
-```
-
-DCell::Node.all returns all connected nodes in the cluster:
-
-```ruby
->> DCell::Node.all
- => [#<DCell::Node[test_node] @addr="tcp://127.0.0.1:21264">, #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:7777">]
-```
-
-DCell::Node is a Ruby Enumerable. You can iterate across all nodes with
-DCell::Node.each.
-
-Once you've obtained a node, you can look up services it exports and call them
-just like you'd invoke methods on any other Ruby object:
-
-```ruby
->> node = DCell::Node["cryptosphere.local"]
- => #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:7777">
->> time_server = node[:time_server]
- => #<Celluloid::Actor(TimeServer:0xee8)>
->> time_server.time
- => "The time is: 2011-11-10 20:23:47 -0800"
-```
-
-You can also find all available services on a node with DCell::Node#all:
-
-```ruby
->> node = DCell::Node["cryptosphere.local"]
- => #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:7777">
->> node.all
- => [:time_server]
-```
-
-Registering Actors
-------------------
-
-All services exposed by DCell must take the form of registered Celluloid actors.
-What follows is an extremely brief introduction to creating and registering
-actors, but for more information, you should definitely [read the Celluloid
-documentation](http://celluloid.github.com).
-
-DCell exposes all Celluloid actors you've registered directly onto the network.
-The best way to register an actor is by supervising it. Below is an example of
-how to create an actor and register it on the network:
-
-```ruby
-class TimeServer
- include Celluloid
-
- def time
- "The time is: #{Time.now}"
- end
-end
-```
-
-Now that we've defined the TimeServer, we're going to supervise it and register
-it in the local registry:
-
-```ruby
->> TimeServer.supervise_as :time_server
- => #<Celluloid::Supervisor(TimeServer):0xee4>
-```
-
-Supervising actors means that if they crash, they're automatically restarted
-and registered under the same name. We can access registered actors by using
-Celluloid::Actor#[]:
-
-```ruby
->> Celluloid::Actor[:time_server]
- => #<Celluloid::Actor(TimeServer:0xee8)>
->> Celluloid::Actor[:time_server].time
- => "The time is: 2011-11-10 20:17:48 -0800"
-```
-
-This same actor is now available using the DCell::Node#[] syntax:
-
-```ruby
->> node = DCell.me
- => #<DCell::Node[cryptosphere.local] @addr="tcp://127.0.0.1:1870">
->> node[:time_server].time
- => "The time is: 2011-11-10 20:28:27 -0800"
-```
-
-Globals
--------
-
-DCell provides a registry global for storing configuration data and actors you
-wish to publish globally to the entire cluster:
-
-```ruby
->> actor = Celluloid::Actor[:dcell_server]
- => #<Celluloid::Actor(DCell::Server:0xf2e) @addr="tcp://127.0.0.1:7777">
->> DCell::Global[:sweet_server] = actor
- => #<Celluloid::Actor(DCell::Server:0xf2e) @addr="tcp://127.0.0.1:7777">
->> DCell::Global[:sweet_server]
- => #<Celluloid::Actor(DCell::Server:0xf2e) @addr="tcp://127.0.0.1:7777">
-```
-
-What about DRb?
----------------
-
-Ruby already has a distributed object system as part of its standard library:
-DRb, which stands for Distributed Ruby. What's wrong with DRb? Why do we need
-a new system? DRb has one major drawback: it's inherently synchronous. The
-only thing you can do to an object is to make a method call, which sends a
-remote object a message, executes the method, and returns a response.
+DCell works on Ruby 1.9.3, JRuby 1.6, and Rubinius 2.0.
-Under the covers, DCell uses an asynchronous message protocol. As noted in the
-last section, asynchronous messaging allows many more modes of messaging than
-the standard reqeust/response pattern afforded by DRb. DCell also supports the
-Erlang-style approach to fault-tolerance, advocating that actors shouldn't handle
-errors but should crash and restart in a clean state. Linking to actors on remote
-nodes can be used to detect these sorts of errors and have dependent actors
-restart in a clean state.
+DCell requires Ruby 1.9 mode on all interpreters. This works out of the
+box on MRI/YARV, and requires the following flags elsewhere:
-By far the biggest difference between DCell and DRb is how the underlying
-Celluloid framework has you think about the problem. Celluloid provides a useful
-concurrent in-process messaging system in its own right without the distributed
-components.
+* JRuby: --1.9 command line option, or JRUBY_OPTS=--1.9 environment variable
+* rbx: -X19 command line option
Copyright
---------
View
12 dcell.gemspec
@@ -16,13 +16,13 @@ Gem::Specification.new do |gem|
gem.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
gem.require_paths = ["lib"]
- gem.add_dependency "celluloid", "~> 0.10.0"
- gem.add_dependency "celluloid-zmq", "~> 0.10.0"
- gem.add_dependency "redis"
- gem.add_dependency "redis-namespace"
- gem.add_dependency "moneta"
+ gem.add_runtime_dependency "celluloid", "~> 0.10.0"
+ gem.add_runtime_dependency "celluloid-zmq", "~> 0.10.0"
+ gem.add_runtime_dependency "reel", "~> 0.0.1"
+ gem.add_runtime_dependency "redis"
+ gem.add_runtime_dependency "redis-namespace"
+ gem.add_runtime_dependency "moneta"
gem.add_development_dependency "rake"
gem.add_development_dependency "rspec"
- gem.add_development_dependency "reel"
end
View
6 explorer/css/explorer.css
@@ -16,15 +16,13 @@ body {
.status-small {
width: 16px;
height: 16px;
- border: 1px solid black;
}
.status-large {
width: 48px;
height: 48px;
- border: 3px solid black;
}
.status-ok {
- background: #468847;
-}
+ background: #90E000;
+}
View
23 explorer/index.html.erb
@@ -43,35 +43,34 @@
<ul class="nav nav-list">
<li class="nav-header">Nodes</li>
<% DCell::Node.each do |node| %>
- <li<%= ' class="active"' if node.id == DCell.id %>>
- <a href="<%= node_path(node) %>"><span class="status status-small status-ok"></span> <%= node.id %></a>
- </li>
+ <li<%= ' class="active"' if node == @node %>>
+ <a href="<%= node_path(node) %>"><span class="status status-small status-ok"></span> <%= node.id %></a>
+ </li>
<% end %>
</ul>
</div><!--/.well -->
</div><!--/span-->
<div class="span9">
- <% node_info = DCell.me[:info].to_hash %>
<div class="hero-unit">
- <h1><div class="status status-large status-ok"></div> <%= node_info[:hostname] %></h1>
+ <h1><div class="status status-large status-ok"></div> <%= @info[:hostname] %></h1>
<br>
- <h2>Uptime: <%= node_info[:uptime] %> days</h2>
- <h2>Load average: <%= node_info[:load_averages].join(", ") %></h2>
+ <h2>Uptime: <%= @info[:uptime] %> days</h2>
+ <h2>Load average: <%= @info[:load_averages].join(", ") %></h2>
</div>
<div class="row-fluid">
<div class="span4">
<h2>System Info</h2>
<ul>
- <li><%= node_info[:cpu][:type] %> @ <%= node_info[:cpu][:speed] %>Ghz (<%= node_info[:cpu][:count] %> cores)</li>
- <li><%= node_info[:cpu][:arch] %> CPU architecture</li>
- <li><%= node_info[:distribution] %></li>
+ <li><%= @info[:cpu][:type] %> @ <%= @info[:cpu][:speed] %>Ghz (<%= @info[:cpu][:count] %> cores)</li>
+ <li><%= @info[:cpu][:arch] %> CPU architecture</li>
+ <li><%= @info[:distribution] %></li>
</ul>
</div><!--/span-->
<div class="span4">
<h2>Ruby Interpreter</h2>
<ul>
- <li><%= node_info[:ruby_engine] == "ruby" ? "MRI/YARV" : node_info[:ruby_platform] %></li>
- <li>Ruby <%= node_info[:ruby_version] %></li>
+ <li><%= @info[:ruby_engine] == "ruby" ? "MRI/YARV" : @info[:ruby_platform] %></li>
+ <li>Ruby <%= @info[:ruby_version] %></li>
</ul>
</div><!--/span-->
</div><!--/row-->
View
3 lib/dcell.rb
@@ -14,6 +14,7 @@
require 'dcell/responses'
require 'dcell/router'
require 'dcell/rpc'
+require 'dcell/future_proxy'
require 'dcell/server'
require 'dcell/info_service'
@@ -117,9 +118,9 @@ def start(options = {})
# DCell's actor dependencies
class Group < Celluloid::Group
+ supervise NodeManager, :as => :node_manager
supervise Server, :as => :dcell_server
supervise InfoService, :as => :info
- supervise NodeManager, :as => :node_manager
end
Logger = Celluloid::Logger
View
11 lib/dcell/celluloid_ext.rb
@@ -66,4 +66,15 @@ def self._load(string)
DCell::RPC._load(string)
end
end
+
+ class Future
+ def _dump(level)
+ mailbox_id = DCell::Router.register self
+ "#{mailbox_id}@#{DCell.id}@#{DCell.addr}"
+ end
+
+ def self._load(string)
+ DCell::FutureProxy._load(string)
+ end
+ end
end
View
37 lib/dcell/explorer.rb
@@ -4,14 +4,17 @@
require 'erb'
module DCell
- class Explorer
+ # Web UI for DCell
+ # TODO: rewrite this entire thing with less hax
+ class Explorer < Reel::Server
+ include Celluloid::IO # FIXME: this should really be unnecessary
ASSET_ROOT = Pathname.new File.expand_path("../../../explorer", __FILE__)
def initialize(host = "127.0.0.1", port = 7778)
- @server = Reel::Server.new(host, port, &method(:handle_connection))
+ super(host, port, &method(:on_connection))
end
- def handle_connection(connection)
+ def on_connection(connection)
request = connection.request
return unless request
route connection, request
@@ -34,24 +37,38 @@ def route(connection, request)
end
def render_resource(connection, path)
+ if node_id = path[%r{^nodes/(.*)$}, 1]
+ p node_id
+ node = DCell::Node[node_id]
+ path = "index.html"
+ else
+ node = DCell.me
+ end
+
asset_path = ASSET_ROOT.join(path)
if asset_path.exist?
asset_path.open("r") do |file|
connection.respond :ok, file
end
- Logger.info "200 OK: #{path}"
- elsif File.exist?(asset_path.to_s + ".erb")
- template = ERB.new File.read("#{asset_path.to_s}.erb", :mode => 'rb')
- connection.respond :ok, template.result(binding)
-
- Logger.info "200 OK: #{path}"
+ Logger.info "200 OK: /#{path}"
+ elsif File.exist?(asset_path.to_s + ".erb") and node
+ connection.respond :ok, render_template(asset_path.to_s + ".erb", node)
+ Logger.info "200 OK: /#{path}"
else
connection.respond :not_found, "Not found"
- Logger.info "404 Not Found: #{path}"
+ Logger.info "404 Not Found: /#{path}"
end
end
+ def render_template(template, node)
+ @node = node
+ @info = @node[:info].to_hash
+
+ template = ERB.new File.read(template, :mode => 'rb')
+ template.result(binding)
+ end
+
def node_path(node)
"/nodes/#{node.id}"
end
View
32 lib/dcell/future_proxy.rb
@@ -0,0 +1,32 @@
+module DCell
+ class FutureProxy
+ def initialize(mailbox_id,node_id,node_addr)
+ @mailbox_id = mailbox_id
+ @node_id = node_id
+ @node_addr = node_addr
+ end
+
+ def <<(message)
+ node = Node[@node_id]
+ node = Node.new(@node_id, @node_addr) unless node
+ node.send_message! Message::Relay.new(self, message)
+ end
+
+ def _dump(level)
+ "#{@mailbox_id}@#{@node_id}@#{@node_addr}"
+ end
+
+ # Loader for custom marshal format
+ def self._load(string)
+ mailbox_id, node_id, node_addr = string.split("@")
+
+ if node_id == DCell.id
+ future = Router.find(mailbox_id)
+ raise "tried to unmarshal dead Celluloid::Future: #{mailbox_id}" unless future
+ future
+ else
+ new(mailbox_id, node_id, node_addr)
+ end
+ end
+ end
+end
View
4 lib/dcell/router.rb
@@ -44,8 +44,8 @@ def find(mailbox_address)
end
# Route a message to a given mailbox ID
- def route(mailbox_address, message)
- recipient = find mailbox_address
+ def route(recipient, message)
+ recipient = find(recipient) if recipient.is_a? String
if recipient
recipient << message
View
3 lib/dcell/server.rb
@@ -5,6 +5,9 @@ class Server
# Bind to the given 0MQ address (in URL form ala tcp://host:port)
def initialize
+ # The gossip protocol is dependent on the node manager
+ link Actor[:node_manager]
+
@addr = DCell.addr
@socket = PullSocket.new
View
4 spec/dcell/actor_proxy_spec.rb
@@ -26,6 +26,10 @@ def exit_handler(actor, reason)
@remote_actor.value.should == 42
end
+ it "makes future calls to remote actors" do
+ @remote_actor.future(:value).value.should == 42
+ end
+
context :linking do
before :each do
@local_actor = LocalActor.new
View
11 spec/dcell/celluloid_ext_spec.rb
@@ -4,6 +4,10 @@
before do
class WillKane
include Celluloid
+
+ def speak
+ "Don't shove me Harv."
+ end
end
@marshal = WillKane.new
end
@@ -18,4 +22,11 @@ class WillKane
string = Marshal.dump(@marshal.mailbox)
Marshal.load(string).should be_alive
end
+
+ it "marshals Celluloid::Future objects" do
+ future = @marshal.future(:speak)
+ future.should be_a(Celluloid::Future)
+ string = Marshal.dump(future)
+ Marshal.load(string).value.should == "Don't shove me Harv."
+ end
end
View
19 spec/spec_helper.rb
@@ -5,12 +5,19 @@
require 'dcell'
Dir['./spec/support/*.rb'].map { |f| require f }
-DCell.setup :directory => { :id => 'test_node', :addr => "tcp://127.0.0.1:#{TestNode::PORT}" }
-DCell.run!
+RSpec.configure do |config|
+ config.before(:suite) do
+ DCell.setup :directory => { :id => 'test_node', :addr => "tcp://127.0.0.1:#{TestNode::PORT}" }
+ @supervisor = DCell.run!
-TestNode.start
-TestNode.wait_until_ready
+ TestNode.start
+ TestNode.wait_until_ready
+ end
-at_exit do
- TestNode.stop
+ config.after(:suite) do
+ TestNode.stop
+ end
end
+
+# FIXME: this is hax to bypass the other at_exit handlers
+at_exit { exit! $!.status }

No commit comments for this range

Something went wrong with that request. Please try again.