Skip to content

Commit

Permalink
more work on README, a little tidy up on Client class
Browse files Browse the repository at this point in the history
  • Loading branch information
jakesgordon committed Aug 23, 2014
1 parent f87de8f commit 5d0c1b1
Show file tree
Hide file tree
Showing 11 changed files with 151 additions and 152 deletions.
125 changes: 63 additions & 62 deletions README.md
@@ -1,58 +1,49 @@
Rack Rabbit (v0.0.1)
====================
Rack Rabbit (v0.1.0) - ALPHA
============================

**WARNING**: This library is in very, very early development

A preforking server for hosting RabbitMQ consumer processes as load balanced rack applications.
A forking server for hosting rabbitMQ consumer processes as load balanced rack applications.

$ rack-rabbit --queue myqueue --workers 4 app/config.ru

Description
-----------

Building an SOA with RabbitMQ ? Want an easy way to host and load balance your consumer processes ?
Building an SOA with rabbitMQ? Want an easy way to host and load balance your consumer processes?

RackRabbit will...

* Create, and manage, a cluster of worker processes that will each...
* Subscribe to a queue
* Subscribe to a queue (or an exchange)
* Convert incoming messages into a suitable Rack environment
* Call your Rack app to handle the message
* Publish a reply back to the original caller (if `reply_to` queue was provided)
* Publish a reply back to the original caller (if `reply_to` was provided)

The goal is to support a RabbitMQ-based SOA with multiple message passing patterns:
RackRabbit will support a rabbitMQ-based SOA with multiple message passing patterns:

* Synchronous Request/Response (e.g. GET/POST/PUT/DELETE)
* Asynchronous Worker queue
* Asynchronous PubSub
* Asynchronous Worker queue (e.g. ENQUEUE)
* Asynchronous PubSub (e.g. PUBLISH)

Installation
------------

Eventually, installation will be via rubygems:

$ gem install bunny # or an alternative rabbitMQ cient library (e.g. AMQP)
$ gem install rack-rabbit
Install a rabbitMQ server if necessary:

... but since the gem has not been officially published yet, for now you need to build it yourself:
$ sudo apt-get install rabbitmq-server

$ git clone https://github.com/jakesgordon/rack-rabbit
$ cd rack-rabbit
$ bundle install
$ build rack-rabbit.gemspec
$ gem install rack-rabbit.gem
Update your Gemfile to include RackRabbit and your preferred rabbitMQ client library

Don't forget to install a rabbitMQ server:
gem bunny, "~> 1.4" # or an alternative such as AMQP or march-hare
gem rack-rabbit, "~> 0.1"

$ sudo apt-get install rabbitmq-server

Getting Started
---------------

You can use RackRabbit + Sinatra (or any rack app) to easily host an AMQP-based SOA in the same way
that you might use Unicorn + Sinatra to host an HTTP-based SOA.

Imagine a simple sinatra application in `app.rb`:
Imagine a simple sinatra application in `config.ru`:

require 'sinatra/base'

Expand All @@ -68,9 +59,6 @@ Imagine a simple sinatra application in `app.rb`:

end

... and a rack configuration file `config.ru`:

require_relative 'app'
run MyApp

You can now host and load balance this application using `rack-rabbit`:
Expand All @@ -86,24 +74,23 @@ Ensure the worker processes are running:
15721 pts/4 Sl+ 0:00 | \_ rack-rabbit -- waiting for request
15723 pts/4 Sl+ 0:00 | \_ rack-rabbit -- waiting for request

You can connect to the worker from your client applications using the `RackRabbit::Client`:

require 'rack-rabbit/client'
You can connect to the worker from the command line using the provided client excutables:

client = RackRabbit::Client.new(:queue => :myqueue)
$ request -q myqueue /hello # synchronous GET request/response
Hello World

foo = client.get "/hello" # -> "Hello World"
bar = client.post "/submit", "some data" # -> "Submitted some data"
$ request -q myqueue POST /submit "data" # synchronous POST request/response
Submitted some data

client.disconnect
$ enqueue -q myqueue /do/work "data" # asynchronous POST to a worker queue

You can also connect to the worker from the command line using the `request` client binary:
You can also connect to the worker from your applications using the `RackRabbit::Client` class.

$ request -q myqueue GET /hello
Hello World
require 'rack-rabbit/client'

$ request -q myqueue POST /submit "some data"
Submitted some data
foo = RackRabbit::Client.get :myqueue, "/hello" # -> "Hello World"
bar = RackRabbit::Client.post :myqueue, "/sumbit", "data" # -> "Submitted data"
RackRabbit::Client.enqueue :myqueue, "/do/work", "data" # async worker queue


HTTP vs AMQP based SOA
Expand All @@ -120,7 +107,7 @@ subscribes either to a named queue or an exchange.

$ rack-rabbit --help

A load balanced rack server for hosting RabbitMQ consumer processes.
A load balanced rack server for hosting rabbitMQ consumer processes.

Usage: rack-rabbit [options] rack-file

Expand All @@ -138,8 +125,8 @@ subscribes either to a named queue or an exchange.
-t, --type TYPE subscribe to an exchange for incoming requests - type (e.g. :direct, :fanout, :topic)
-r, --route ROUTE subscribe to an exchange for incoming requests - routing key
-a, --app_id ID an app_id for this application server
--host HOST the RabbitMQ broker IP address (default: 127.0.0.1)
--port PORT the RabbitMQ broker port (default: 5672)
--host HOST the rabbitMQ broker IP address (default: 127.0.0.1)
--port PORT the rabbitMQ broker port (default: 5672)

Process options:
-w, --workers COUNT the number of worker processes (default: 1)
Expand Down Expand Up @@ -168,7 +155,7 @@ Detailed RackRabbit configuration can be provided by an external config file usi
# set the Rack application to be used to handle messages (default 'config.ru'):
rack_file 'app/config.ru'

# set the RabbitMQ connection:
# set the rabbitMQ connection:
rabbit :host => '10.0.0.42', # default '127.0.0.1'
:port => '1234' # default '5672'
:adapter => :amqp # default :bunny
Expand All @@ -181,6 +168,12 @@ Detailed RackRabbit configuration can be provided by an external config file usi
exchange_type :topic
routing_key 'my.topic'

# set the app_id used to identify your application in response messages
app_id 'my-application'

# enable rabbitMQ acknowledgements
acknowledge true

# set the initial number of worker processes (default: 1):
workers 8

Expand All @@ -193,14 +186,21 @@ Detailed RackRabbit configuration can be provided by an external config file usi
# preload the Rack app in the server for faster worker forking (default: false):
preload_app true

# daemonize the process
daemonize true

# set the path to the logfile
logfile "/var/log/my-application.log"

# set the path to the pidfile
pidfile "/var/run/my-application.pid"

# set the log level for the Rack Rabbit logger (default: info)
log_level 'debug'

# set the Logger to used by the Rack Rabbit server and the worker Rack applications (default: Logger)
logger MyLogger.new

# set the app_id used to identify your application in response messages
app_id 'my-application'

Signals
-------
Expand Down Expand Up @@ -235,10 +235,15 @@ This should NOT be needed when the `preload_app` directive is false.

>> _this is an issue with any preforking style server (e.g. Unicorn)_
RabbitMQ Acknowledgements
-------------------------

TODO: document :ack and :reject support

Client Library
--------------

Posting a message to a RackRabbit hosted server can be done using any RabbitMQ client library, but
Posting a message to a RackRabbit hosted server can be done using any rabbitMQ client library, but
is easiest using the built in `RackRabbit::Client`...

TODO: document RackRabbit::Client
Expand All @@ -248,29 +253,25 @@ Supported Platforms

Nothing formal yet, development is happening on MRI 2.1.2p95

TODO: test on other platforms


TODO
----

* testing
- replace DEFAULT rack app with MIRROR rack app ?
* avoid infinite spawn worker loop if worker fails during startup (e.g. connection to rabbit fails)
* allow multiple synchronous req/response in parallel (block until all have replied)
* allow a single reply queue to be shared across client requests
* have exception callstacks sent back to client (in development mode only)
* automatically deserialize body into hash if content type is json ?

* more testing
- client
- worker
- server
- client
- adapter/bunny
- adapter/amqp

* better documentation
- client
- :ack and :reject support

* platform support
* MISC
- avoid infinite spawn worker loop if worker fails during startup (e.g. connection to rabbit fails)
- allow a single reply queue to be shared across client requests
- allow multiple synchronous req/response in parallel (block until all have replied)
- automatically deserialize body into hash if content type is json ?
- have exception callstacks sent back to client (in development mode only)

License
-------

Expand All @@ -285,7 +286,7 @@ Thanks to [Jesse Storimer](http://www.jstorimer.com/) for his book
Thanks to the [Unicorn Team](http://unicorn.bogomips.org/) for providing a great
example of a preforking server.

Thanks to the [Bunny Team](http://rubybunny.info/) for providing an easy RabbitMQ Ruby client.
Thanks to the [Bunny Team](http://rubybunny.info/) for providing an easy rabbitMQ Ruby client.

Contact
-------
Expand Down
4 changes: 2 additions & 2 deletions bin/enqueue
Expand Up @@ -18,7 +18,7 @@ debug_help = "set $DEBUG to true"
warn_help = "enable warnings"

op = OptionParser.new
op.banner = "Enqueue an asynchronous request to a RabbitMQ queue without waiting for a reply"
op.banner = "Enqueue an asynchronous request to a rabbitMQ queue without waiting for a reply"
op.separator ""
op.separator "Usage: enqueue [options] [METHOD] [PATH] [BODY]"
op.separator ""
Expand Down Expand Up @@ -66,7 +66,7 @@ case action
when :help then puts op.to_s
when :version then puts RackRabbit::VERSION
else
puts RackRabbit::Client.enqueue(options)
puts RackRabbit::Client.enqueue(options[:queue], options)
end

#==============================================================================
4 changes: 2 additions & 2 deletions bin/publish
Expand Up @@ -20,7 +20,7 @@ debug_help = "set $DEBUG to true"
warn_help = "enable warnings"

op = OptionParser.new
op.banner = "Publish an asynchronous request to a RabbitMQ exchange with a routing key"
op.banner = "Publish an asynchronous request to a rabbitMQ exchange with a routing key"
op.separator ""
op.separator "Usage: publish [options] [METHOD] [PATH] [BODY]"
op.separator ""
Expand Down Expand Up @@ -70,7 +70,7 @@ case action
when :help then puts op.to_s
when :version then puts RackRabbit::VERSION
else
puts RackRabbit::Client.publish(options)
puts RackRabbit::Client.publish(options[:exchange], options)
end

#==============================================================================
6 changes: 3 additions & 3 deletions bin/rack-rabbit
Expand Up @@ -19,8 +19,8 @@ exchange_help = "subscribe to an exchange for incoming requests"
exchange_type_help = "subscribe to an exchange for incoming requests - type (e.g. :direct, :fanout, :topic)"
routing_key_help = "subscribe to an exchange for incoming requests - routing key"
app_id_help = "an app_id for this application server"
host_help = "the RabbitMQ broker IP address (default: 127.0.0.1)"
port_help = "the RabbitMQ broker port (default: 5672)"
host_help = "the rabbitMQ broker IP address (default: 127.0.0.1)"
port_help = "the rabbitMQ broker port (default: 5672)"

workers_help = "the number of worker processes (default: 1)"
daemonize_help = "run daemonized in the background (default: false)"
Expand All @@ -33,7 +33,7 @@ debug_help = "set $DEBUG to true"
warn_help = "enable warnings"

op = OptionParser.new
op.banner = "A load balanced rack server for hosting RabbitMQ consumer processes."
op.banner = "A load balanced rack server for hosting rabbitMQ consumer processes."
op.separator ""
op.separator "Usage: rack-rabbit [options] rack-file"
op.separator ""
Expand Down
4 changes: 2 additions & 2 deletions bin/request
Expand Up @@ -18,7 +18,7 @@ debug_help = "set $DEBUG to true"
warn_help = "enable warnings"

op = OptionParser.new
op.banner = "Make a synchronous request to a RabbitMQ queue and wait for a reply"
op.banner = "Make a synchronous request to a rabbitMQ queue and wait for a reply"
op.separator ""
op.separator "Usage: request [options] [METHOD] [PATH] [BODY]"
op.separator ""
Expand Down Expand Up @@ -66,7 +66,7 @@ case action
when :help then puts op.to_s
when :version then puts RackRabbit::VERSION
else
puts RackRabbit::Client.request(options)
puts RackRabbit::Client.request(options[:queue], options)
end

#==============================================================================
28 changes: 0 additions & 28 deletions examples/rack/app.rb

This file was deleted.

29 changes: 28 additions & 1 deletion examples/rack/config.ru
@@ -1,3 +1,30 @@
require_relative 'app'
class MyApp

def self.call(env)

request = Rack::Request.new env
logger = request.logger
path = request.path_info

if path && path.include?("sleep")
duration = path.to_s.split("/").last.to_i
duration.times do |n|
logger.info "sleep #{n}"
sleep(1)
end
end

response = Rack::Response.new
response.write "Method: #{request.request_method}\n"
response.write "Path: #{path}\n"
response.write "Query: #{request.query_string}\n" unless request.query_string.empty?
response.write "Slept for: #{duration}\n" unless duration.nil?
response.write request.body.read
response.status = 200
response.finish

end

end

run MyApp

0 comments on commit 5d0c1b1

Please sign in to comment.