merge master

.gitignore

@@ -1,15 +1,19 @@
-Makefile
-mkmf.log
-*.o
-*.bundle
+Gemfile.lock
+
+.bundle/
 doc/
 pkg/
-examples/log
-*.swp
-Gemfile.lock
+_site
+
 .yardoc
 .livereload
+.rvmrc
+
+*.swp
 *.watchr
 *.rbc
-.rvmrc
-_site
+
+examples/log
+examples/goliath.log*
+examples/goliath.pid
+examples/rasterize/thumb

Gemfile

@@ -1,6 +1,3 @@
 source "http://rubygems.org"
 
-gem 'em-websocket', :git => 'http://github.com/dj2/em-websocket', :branch => 'factory_change'
-gem 'http_parser.rb', :git => 'http://github.com/dj2/http_parser.rb', :branch => 'upgrade_data'
-
 gemspec

HISTORY.md

@@ -0,0 +1,64 @@
+# HISTORY
+
+## v0.9.3 (Oct 16, 2011)
+
+  - new router DSL - much improved, see examples
+  - refactored async_aroundware
+  - make jruby friendlier (removed 1.9 req in gemspec)
+  - enable epoll
+  - SSL support
+  - unix socket support
+  - reload config on HUP
+  - and a number of small bugfixes + other improvements..
+  - See full list @ https://github.com/postrank-labs/goliath/compare/v0.9.2...v0.9.3
+
+## v0.9.2 (July 21, 2011)
+
+  - See full list @ https://github.com/postrank-labs/goliath/compare/v0.9.1...v0.9.2
+
+## v0.9.1 (Apr 12, 2011)
+
+  - Added extra messaging around the class not matching the file name (Carlos Brando)
+
+  - Fix issue with POST parameters not being parsed by Goliath::Rack::Params
+  - Added support for multipart encoded POST bodies
+  - Added support for parsing nested query string parameters (Nolan Evans)
+  - Added support for parsing application/json POST bodies
+  - Content-Types outside of multipart, urlencoded and application/json will not be parsed automatically.
+
+  - added 'run as user' option
+  - SERVER_NAME and SERVER_PORT are set to values in HOST header
+
+  - Cleaned up spec examples (Justin Ko)
+
+  - moved logger into 'rack.logger' key to be more Rack compliant (Env#logger added to
+    keep original API consistent)
+  - add command line option for specifying config file
+  - HTTP_CONTENT_LENGTH and HTTP_CONTENT_TYPE were changed to CONTENT_TYPE and CONTENT_LENGTH
+    to be more Rack compliant
+  - fix issue with loading config file in development mode
+
+  - Rack::Reloader will be loaded automatically by the framework in development mode.
+
+
+## v0.9.0 (Mar 9, 2011)
+
+(Initial Public Release)
+
+Goliath is an open source version of the non-blocking (asynchronous) Ruby web server framework
+powering PostRank. It is a lightweight framework designed to meet the following goals: bare
+metal performance, Rack API and middleware support, simple configuration, fully asynchronous
+processing, and readable and maintainable code (read: no callbacks).
+
+The framework is powered by an EventMachine reactor, a high-performance HTTP parser and Ruby 1.9
+runtime. One major advantage Goliath has over other asynchronous frameworks is the fact that by
+leveraging Ruby fibers, it can untangle the complicated callback-based code into a format we are
+all familiar and comfortable with: linear execution, which leads to more maintainable and readable code.
+
+While MRI is the recommend platform, Goliath has been tested to run on JRuby and Rubinius.
+
+Goliath has been in production at PostRank for over a year, serving a sustained 500 requests/s for
+internal and external applications. Many of the Goliath processes have been running for months at
+a time (read: no memory leaks) and have served hundreds of gigabytes of data without restarts. To
+scale up and provide failover and redundancy, our individual Goliath servers at PostRank are usually
+deployed behind a reverse proxy (such as HAProxy).

README.md

@@ -20,21 +20,23 @@ Each HTTP request within Goliath is executed in its own Ruby fiber and all async
 
 ## Getting Started: Hello World
 
-    require 'goliath'
+```ruby
+require 'goliath'
 
-    class Hello < Goliath::API
-      # reload code on every request in dev environment
-      use ::Rack::Reloader, 0 if Goliath.dev?
+class Hello < Goliath::API
+  def response(env)
+    [200, {}, "Hello World"]
+  end
+end
 
-      def response(env)
-        [200, {}, "Hello World"]
-      end
-    end
+> ruby hello.rb -sv
+> [97570:INFO] 2011-02-15 00:33:51 :: Starting server on 0.0.0.0:9000 in development mode. Watch out for stones.
+```
 
-    > ruby hello.rb -sv
-    > [97570:INFO] 2011-02-15 00:33:51 :: Starting server on 0.0.0.0:9000 in development mode. Watch out for stones.
+See examples directory for more, hands-on examples of building Goliath powered web-services. Are you new to EventMachine, or want a detailed walk-through of building a Goliath powered API? You're in luck, we have two super-awesome peepcode screencasts which will teach you all you need to know:
 
-See examples directory for more, hands-on examples of building Goliath powered web-services.
+* [Meet EventMachine: Part 1](http://peepcode.com/products/eventmachine) - introduction to EM, Fibers, etc.
+* [Meet EventMachine: Part 2](http://peepcode.com/products/eventmachine-ii) - building an API with Goliath
 
 ## Performance: MRI, JRuby, Rubinius
 
@@ -55,7 +57,7 @@ Goliath has been in production at PostRank for over a year, serving a sustained
     * Mongrel is a threaded web-server, and both Passenger and Unicorn fork an entire VM to isolate each request from each other. By contrast, Goliath builds a single instance of the Rack app and runs all requests in parallel through a single VM, which leads to a much smaller memory footprint and less overhead.
 
 * How do I deploy Goliath in production?
-    * We recommend deploying Goliath behind a reverse proxy such as HAProxy, Nginx or equivalent. Using one of the above, you can easily run multiple instances of the same application and load balance between them within the reverse proxy.
+    * We recommend deploying Goliath behind a reverse proxy such as HAProxy ([sample config](https://github.com/postrank-labs/goliath/wiki/HAProxy)), Nginx or equivalent. Using one of the above, you can easily run multiple instances of the same application and load balance between them within the reverse proxy.
 
 ## Guides
 
@@ -66,6 +68,7 @@ Goliath has been in production at PostRank for over a year, serving a sustained
 
 ### Hands-on applications:
 
+* [Peepcode](http://peepcode.com/products/eventmachine) [screencasts](http://peepcode.com/products/eventmachine-ii)
 * [Asynchronous HTTP, MySQL, etc](https://github.com/postrank-labs/goliath/wiki/Asynchronous-Processing)
 * [Response streaming with Goliath](https://github.com/postrank-labs/goliath/wiki/Streaming)
 * [Examples](https://github.com/postrank-labs/goliath/tree/master/examples)
@@ -74,6 +77,10 @@ Goliath has been in production at PostRank for over a year, serving a sustained
 
 * [Goliath: Non-blocking, Ruby 1.9 Web Server](http://www.igvita.com/2011/03/08/goliath-non-blocking-ruby-19-web-server)
 * [Stage left: Enter Goliath - HTTP Proxy + MongoDB](http://everburning.com/news/stage-left-enter-goliath/)
+* [InfoQ: Meet the Goliath of Ruby Application Servers](http://www.infoq.com/articles/meet-goliath)
+* [Node.jsはコールバック・スパゲティを招くか](http://el.jibun.atmarkit.co.jp/rails/2011/03/nodejs-d123.html)
+* [Goliath on LinuxFr.org (french)](http://linuxfr.org/news/en-vrac-spécial-ruby-jruby-sinatra-et-goliath)
+* [Goliath et ses amis (slides in french)](http://nono.github.com/Presentations/20110416_Goliath/)
 
 ## Discussion and Support
 
@@ -83,4 +90,5 @@ Goliath has been in production at PostRank for over a year, serving a sustained
 
 ## License & Acknowledgments
 
-Goliath is distributed under the MIT license, for full details please see the LICENSE file.
+Goliath is distributed under the MIT license, for full details please see the LICENSE file.
+Rock favicon CC-BY from [Douglas Feer](http://www.favicon.cc/?action=icon&file_id=375421)

Rakefile

@@ -5,6 +5,7 @@ require 'yard'
 require 'rspec/core/rake_task'
 
 task :default => [:spec]
+task :test => [:spec]
 
 desc "run spec tests"
 RSpec::Core::RakeTask.new('spec') do |t|

examples/activerecord/srv.rb

@@ -19,13 +19,9 @@ class User < ActiveRecord::Base
 end
 
 class Srv < Goliath::API
-  use ::Rack::Reloader, 0 if Goliath.dev?
-
   use Goliath::Rack::Params
   use Goliath::Rack::DefaultMimeType
-  use Goliath::Rack::Formatters::JSON
-  use Goliath::Rack::Render
-  use Goliath::Rack::ValidationError
+  use Goliath::Rack::Render, 'json'
 
   use Goliath::Rack::Validation::RequiredParam, {:key => 'id', :type => 'ID'}
   use Goliath::Rack::Validation::NumericRange, {:key => 'id', :min => 1}

examples/api_proxy.rb

@@ -0,0 +1,28 @@
+#!/usr/bin/env ruby
+
+# Rewrites and proxies requests to a third-party API, with HTTP basic authentication.
+
+require 'goliath'
+require 'em-synchrony/em-http'
+
+class TwilioResponse < Goliath::API
+  use Goliath::Rack::Params
+  use Goliath::Rack::JSONP
+
+  HEADERS = { authorization: ENV.values_at("TWILIO_SID","TWILIO_AUTH_TOKEN") }
+  BASE_URL = "https://api.twilio.com/2010-04-01/Accounts/#{ENV['TWILIO_SID']}/AvailablePhoneNumbers/US"
+
+  def response(env)
+    url = "#{BASE_URL}#{env['REQUEST_PATH']}?#{env['QUERY_STRING']}"
+    logger.debug "Proxying #{url}"
+
+    http = EM::HttpRequest.new(url).get head: HEADERS
+    logger.debug "Received #{http.response_header.status} from Twilio"
+
+    [200, {'X-Goliath' => 'Proxy','Content-Type' => 'application/javascript'}, http.response]
+  end
+end
+
+class Twilio < Goliath::API
+  get %r{^/(Local|TollFree)}, TwilioResponse
+end

examples/async_aroundware_demo.rb

@@ -0,0 +1,85 @@
+#!/usr/bin/env ruby
+$: << File.dirname(__FILE__)+'/../lib'
+
+require 'goliath'
+require 'em-synchrony/em-http'
+require 'yajl/json_gem'
+
+#
+# Here's a way to make an asynchronous request in the middleware, and only
+# proceed with the response when both the endpoint and our middleware's
+# responses have completed.
+#
+# To run this, start the 'test_rig.rb' server on port 9002:
+#
+#   bundle exec ./examples/test_rig.rb -sv -p 9002
+#
+# And then start this server on port 9000:
+#
+#   bundle exec ./examples/barrier_aroundware_demo.rb -sv -p 9000
+#
+# Now curl the async_aroundware_demo_multi:
+#
+#    $ time curl  'http://127.0.0.1:9000/?delay_1=1.0&delay_2=1.5'
+#         { "results": {
+#             "sleep_2": { "delay": 1.5, "actual": 1.5085558891296387 },
+#             "sleep_1": { "delay": 1.0, "actual": 1.0098700523376465 }
+#           } }
+#
+# The requests are run concurrently:
+#
+#   $ ./examples/async_aroundware_demo.rb -sv -p 9000 -e prod &
+#   [68463:INFO] 2011-05-03 23:13:03 :: Starting server on 0.0.0.0:9000 in production mode. Watch out for stones.
+#   $ ab -n10 -c10 'http://127.0.0.1:9000/?delay_1=1.5&delay_2=2.0'
+#   Connection Times (ms)
+#                 min  mean[+/-sd] median   max
+#   Connect:        0    0   0.1      0       0
+#   Processing:  2027 2111  61.6   2112    2204
+#   Waiting:     2027 2111  61.5   2112    2204
+#   Total:       2027 2112  61.5   2113    2204
+#
+#
+
+BASE_URL     = 'http://localhost:9002/'
+
+class RemoteRequestBarrier
+  include Goliath::Rack::BarrierAroundware
+  attr_accessor :sleep_1
+
+  def pre_process
+    # Request with delay_1 and drop_1 -- note: 'aget', because we want execution to continue
+    req = EM::HttpRequest.new(BASE_URL).aget(:query => { :delay => env.params['delay_1'], :drop => env.params['drop_1'] })
+    enqueue :sleep_1, req
+    return Goliath::Connection::AsyncResponse
+  end
+
+  def post_process
+    # unify the results with the results of the API call
+    if successes.include?(:sleep_1) then body[:results][:sleep_1] = JSON.parse(sleep_1.response)
+    else                                 body[:errors][:sleep_1]  = sleep_1.error     ; end
+    [status, headers, JSON.pretty_generate(body)]
+  end
+end
+
+class BarrierAroundwareDemo < Goliath::API
+  use Goliath::Rack::Params
+  use Goliath::Rack::Validation::NumericRange, {:key => 'delay_1', :default => 1.0, :max => 5.0, :min => 0.0, :as => Float}
+  use Goliath::Rack::Validation::NumericRange, {:key => 'delay_2', :default => 0.5, :max => 5.0, :min => 0.0, :as => Float}
+  #
+  use Goliath::Rack::BarrierAroundwareFactory, RemoteRequestBarrier
+
+  def response(env)
+    # Request with delay_2 and drop_2 -- note: 'get', because we want execution to proceed linearly
+    resp = EM::HttpRequest.new(BASE_URL).get(:query => { :delay => env.params['delay_2'], :drop => env.params['drop_2'] })
+
+    body = { :results => {}, :errors => {} }
+
+    if resp.response_header.status.to_i != 0
+      body[:results][:sleep_2] = JSON.parse(resp.response) rescue 'parsing failed'
+    else
+      body[:errors ][:sleep_2] = resp.error
+    end
+
+    [200, { }, body]
+  end
+end

examples/async_upload.rb

@@ -5,14 +5,9 @@
 require 'yajl'
 
 class AsyncUpload < Goliath::API
-
-  # reload code on every request in dev environment
-  use ::Rack::Reloader, 0 if Goliath.dev?
-
   use Goliath::Rack::Params             # parse & merge query and body parameters
   use Goliath::Rack::DefaultMimeType    # cleanup accepted media types
-  use Goliath::Rack::Formatters::JSON   # JSON output formatter
-  use Goliath::Rack::Render             # auto-negotiate response format
+  use Goliath::Rack::Render, 'json'     # auto-negotiate response format
 
   def on_headers(env, headers)
     env.logger.info 'received headers: ' + headers.inspect