Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

Utility middlewares #2

Open
wants to merge 8 commits into from

2 participants

@mrflip
Collaborator

Middlewares to do many useful forms of diagnosis and fault injection:

  • force_delay does a 'sleep for x +- y' seconds before responding
  • force_drop drops the connection, before or after its middleware descendents run
  • force_fault raises the requested type of error (eg _fault=404 will bail out of the rpocessing chain, returning a 404 Not Found response
  • force_response forcibly replaces the status, headers or body with what's given in the URL
  • handle_exceptions gives you response methods, same that the middleware provides, for your app
  • diagnostics recycles info about the request into the output

I don't seem to know how to do these as individual pull requests

Philip (flip... added some commits
Philip (flip) Kromer Made the support files (gemspec, Gemfile, etc) look like goliath's) 4280923
Philip (flip) Kromer Middlewares to do many useful forms of diagnosis and fault injection:
* force_delay       does a 'sleep for x +- y' seconds before responding
* force_drop        drops the connection, before or after its middleware descendents run
* force_fault       raises the requested type of error (eg _fault=404 will bail out of the rpocessing chain, returning a 404 Not Found response
* force_response    forcibly replaces the status, headers or body with what's given in the URL
* handle_exceptions gives you response methods, same that the middleware provides, for your app
* diagnostics recycles info about the request into the output
31d6072
Philip (flip) Kromer middleware to load white-listed params into the env; another to load …
…static values into the env
3bd3c95
Philip (flip) Kromer reworked injection middlewares to be configured by env, not params (b…
…ack to how it ics api) so that it's actually useful outside test rig
34e7f9e
Philip (flip) Kromer minor tweaks to utility middlewares while landing the last pieces ac9b329
Philip (flip) Kromer force timeout calls its response chain rather than raising an exception.
 it now formulates the response in the timeout condition directly, and calls its middleware predecessor with that error reposnse. This was always the intended behavior, but a too-helpful rescue block was making it skip the remaining middleware chain.
fedee64
Philip (flip) Kromer more clearer the documentation was made to be 3a24739
Philip (flip) Kromer Rakefile does yard & tests too. Dependency list more minimalist.
The rakefile is copied from goliath; it now has tasks for running tests, making yardoc, etc
b7c8a39
@igrigorik igrigorik commented on the diff
examples/test_rig.rb
((64 lines not shown))
+ use Goliath::Contrib::Rack::HandleExceptions # turn raised errors into HTTP responses
+ use Goliath::Rack::Params # parse & merge query and body parameters
+
+ # turn params like '_force_delay' into env vars :force_delay
+ use(Goliath::Contrib::Rack::ConfigurateFromParams,
+ [ :force_timeout, :force_drop, :force_drop_after, :force_fault, :force_fault_after,
+ :force_status, :force_headers, :force_body, :force_delay, :force_randelay, ],)
+
+ use Goliath::Contrib::Rack::ForceTimeout # raise an error if response takes longer than given time
+ use Goliath::Contrib::Rack::ForceDrop # drop connection immediately with no response
+ use Goliath::Contrib::Rack::ForceFault # raise an error of the given type (eg `_force_fault=400` causes a BadRequestError)
+ use Goliath::Contrib::Rack::ForceResponse # replace as given by '_force_status', '_force_headers' or '_force_body'
+ use Goliath::Contrib::Rack::ForceDelay # force response to take at least (_force_delay + rand*_force_randelay) seconds
+ use Goliath::Contrib::Rack::Diagnostics # summarize the request in the response headers
+ end
+ self.set_middleware!
@igrigorik Owner

what's the reason for class method + invocation vs plain use?

@mrflip Collaborator
mrflip added a note

so that other things can inherit from TestRig and get its middleware stack -- is there a cleaner way to do that?

@igrigorik Owner

Oh, hmm.. interesting. I guess in the router code we had something similar, where we automatically walked up the tree and injected parents middleware. I think it's a reasonable thing to do and support in Goliath in the subclass use case.. we should pull this out into a separate story/bug and tackle it there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@igrigorik igrigorik commented on the diff
lib/goliath/contrib/rack/configurator.rb
@@ -0,0 +1,48 @@
+module Goliath
+ module Contrib
+ module Rack
+
+ # Place static values fron initialize into the env on each request
@igrigorik Owner

fron > from :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@igrigorik igrigorik commented on the diff
lib/goliath/contrib/rack/configurator.rb
((14 lines not shown))
+ def call(env,*)
+ env.merge!(@extra_env_vars)
+ super
+ end
+ end
+
+ #
+ #
+ # @example imposes a timeout if 'rapid_timeout' param is present
+ # class RapidServiceOrYour408Back < Goliath::API
+ # use Goliath::Rack::Params
+ # use ConfigurateFromParams, [:timeout], 'rapid'
+ # use Goliath::Contrib::Rack::ForceTimeout
+ # end
+ #
+ class ConfigurateFromParams
@igrigorik Owner

ConfigureFromParams? :)

@igrigorik Owner

Hmm, so this middleware is just syntax sugar to turn "x_y" into "x" key in env?

@mrflip Collaborator
mrflip added a note

this lets me inject behavior deep into the stack -- I can put a param foo_force_drop, ensure intervening proxies and middlewares preserve that parameter, and only when it hits the foo_ layer does it go from param into the environment (and thus trigger the middleware in question).

@igrigorik Owner

So why is just the order of the middleware execution not sufficient to match this case? Still trying to wrap my head around the use case.. Is it parameter name collisions with other upstream proxies, so you're trying to work around that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@igrigorik igrigorik commented on the diff
lib/goliath/contrib/rack/diagnostics.rb
((24 lines not shown))
+ #
+ # Add headers showing the request's parameters, path, headers and method
+ #
+ # Please also include Goliath::Contrib::CaptureHeaders in your app class.
+ #
+ # @example
+ # class AwesomeApp < Goliath::API
+ # include Goliath::Contrib::CaptureHeaders
+ # use Goliath::Contrib::Rack::Diagnostics
+ # end
+ #
+ class Diagnostics
+ include Goliath::Rack::AsyncMiddleware
+
+ def request_diagnostics(env)
+ client_headers = env[:client_headers] or env.logger.info("Please 'include Goliath::Contrib::CaptureHeaders' in your API class")
@igrigorik Owner

Why can't this be done automatically in this code?

@mrflip Collaborator
mrflip added a note

Is there a way to capture headers in middleware? I only know how to in the Goliath::API app itself.

@igrigorik Owner

Ah, hmm.. Well, you should have all the "rackified" headers directly in the ENV hash (as per Rack spec). I guess if you wanted the raw headers, then that's a bit different.. /cc @dj2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@igrigorik igrigorik commented on the diff
lib/goliath/contrib/rack/force_delay.rb
((1 lines not shown))
+module Goliath
+ module Contrib
+
+ module Rack
+
+ # Delays response for `delay + (0 to randelay)` additional seconds after
+ # the app's response.
+ #
+ # This delay is non-blocking -- *other* requests may proceed in turn --
+ # though naturally the call chain for this response doesn't proceed until
+ # the delay is complete.
+ #
+ # ForceDelay ensures your response takes *at least* N seconds. Force
+ # Timeout ensures your response takes *at most* N seconds. To have a
+ # response take *as-close-as-reasonable-to* N seconds, use an N-second
+ # ForceTimeout with an (N+1)-second ForceDelay.
@igrigorik Owner

this line is talking about ForceTime + ForceDelay, but later we switch to force_delay and force_randelay - seems inconsistent. Am I missing something?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@igrigorik igrigorik commented on the diff
lib/goliath/contrib/rack/force_drop.rb
@@ -0,0 +1,42 @@
+module Goliath
+ module Contrib
+ module Rack
+
+ # Middleware to simulate dropping a connection.
@igrigorik Owner

Is this purely for testing? What's the use case? Seems of limited application. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@igrigorik igrigorik commented on the diff
lib/goliath/contrib/rack/force_timeout.rb
((48 lines not shown))
+ # timeout callback, executed by EM timer.
+ # This will always fire, we just don't do anything if already handled.
+ # If not handled elsewhere, mark as handled and raise an error
+ EM.add_timer(timeout) do
+ unless env[:force_timeout_complete]
+ env[:force_timeout_complete] = true
+ err = Goliath::Validation::RequestTimeoutError.new("Request exceeded #{timeout} seconds")
+ async_cb.call(error_response(err, 'X-Resp-Timeout' => timeout.to_s))
+ end
+ end
+ end
+
+ status, headers, body = @app.call(env)
+
+ if status == Goliath::Connection::AsyncResponse.first
+ env[:force_timeout_complete] = true
@igrigorik Owner

Instead of the checks above, should probably store the timer and simply cancel it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@igrigorik igrigorik commented on the diff
lib/goliath/contrib/rack/handle_exceptions.rb
@@ -0,0 +1,63 @@
+module Goliath
+ module Contrib
+ module Rack
+
+ # Rescue validation errors in the app just as you do
+ # in middleware
@igrigorik Owner

We recently added some logic to behave differently based on which environment you're running in (dev, prod). This should probably follow same patter, and.. should this be in core instead? What's the difference between this and current behavior in master?

@mrflip Collaborator
mrflip added a note

This was written prior to the changes you mentioned, but compared to the old error code:

  • lets you preserve headers, and puts some details in header,
  • works with the exception object rather than shipping around all its pieces
  • emits a richer hash
  • handles errors in middleware same as errors in app (this has been partially addressed I think, but see below)

I'd prefer it in core, and will fix up a pull request for review.

You'll still want the HandleExceptions class, because you want "normal" errors (database errors, bad requests, 404, etc) handled right before the render part of the stack. HandleException and everything below it supply a hash as the body; the render layer and everything above it supply a string as the body. If an error occurs, this will emit it a hash and be serialized same as anything else: CORS headers, correct serialization format, reporting, etc still occur.

@igrigorik Owner

sgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Jun 23, 2012
  1. Made the support files (gemspec, Gemfile, etc) look like goliath's)

    Philip (flip) Kromer authored
Commits on Jun 24, 2012
  1. Middlewares to do many useful forms of diagnosis and fault injection:

    Philip (flip) Kromer authored
    * force_delay       does a 'sleep for x +- y' seconds before responding
    * force_drop        drops the connection, before or after its middleware descendents run
    * force_fault       raises the requested type of error (eg _fault=404 will bail out of the rpocessing chain, returning a 404 Not Found response
    * force_response    forcibly replaces the status, headers or body with what's given in the URL
    * handle_exceptions gives you response methods, same that the middleware provides, for your app
    * diagnostics recycles info about the request into the output
  2. middleware to load white-listed params into the env; another to load …

    Philip (flip) Kromer authored
    …static values into the env
  3. reworked injection middlewares to be configured by env, not params (b…

    Philip (flip) Kromer authored
    …ack to how it ics api) so that it's actually useful outside test rig
  4. minor tweaks to utility middlewares while landing the last pieces

    Philip (flip) Kromer authored
Commits on Jul 6, 2012
  1. force timeout calls its response chain rather than raising an exception.

    Philip (flip) Kromer authored
     it now formulates the response in the timeout condition directly, and calls its middleware predecessor with that error reposnse. This was always the intended behavior, but a too-helpful rescue block was making it skip the remaining middleware chain.
  2. more clearer the documentation was made to be

    Philip (flip) Kromer authored
  3. Rakefile does yard & tests too. Dependency list more minimalist.

    Philip (flip) Kromer authored
    The rakefile is copied from goliath; it now has tasks for running tests, making yardoc, etc
This page is out of date. Refresh to see the latest.
View
15 Gemfile
@@ -2,3 +2,18 @@ source 'https://rubygems.org'
# Specify your gem's dependencies in goliath-contrib.gemspec
gemspec
+
+gem 'goliath', :path => '../goliath'
+
+group :development do
+ gem 'rake'
+end
+
+# Gems for testing and coverage
+group :test do
+ gem 'pry'
+ gem 'rspec'
+ gem 'guard'
+ gem 'guard-rspec'
+ gem 'guard-yard'
+end
View
20 Guardfile
@@ -0,0 +1,20 @@
+# -*- ruby -*-
+
+format = 'progress' # 'doc' for more verbose, 'progress' for less
+tags = %w[ ]
+guard 'rspec', :version => 2, :cli => "--format #{format} #{ tags.map{|tag| "--tag #{tag}"}.join(' ') }" do
+ watch(%r{^spec/.+_spec\.rb$})
+ watch(%r{^lib/(.+)\.rb$}) { |m| "spec/#{m[1]}_spec.rb" }
+ watch(%r{^examples/(.+)\.rb$}) { |m| "spec/integration/#{m[1]}_spec.rb" }
+
+ watch('spec/spec_helper.rb') { 'spec' }
+ watch(/spec\/support\/(.+)\.rb/) { 'spec' }
+end
+
+group :docs do
+ guard 'yard' do
+ watch(%r{app/.+\.rb})
+ watch(%r{lib/.+\.rb})
+ watch(%r{ext/.+\.c})
+ end
+end
View
29 Rakefile
@@ -1,2 +1,27 @@
-#!/usr/bin/env rake
-require "bundler/gem_tasks"
+require 'bundler'
+Bundler::GemHelper.install_tasks
+
+require 'yard'
+require 'rspec/core/rake_task'
+require 'rake/testtask'
+
+task :default => [:test]
+task :test => [:spec, :unit]
+
+desc "run the unit test"
+Rake::TestTask.new(:unit) do |t|
+ t.libs << "test"
+ t.test_files = FileList['test/**/*_test.rb']
+ t.verbose = true
+end
+
+desc "run spec tests"
+RSpec::Core::RakeTask.new('spec') do |t|
+ t.pattern = 'spec/**/*_spec.rb'
+end
+
+desc 'Generate documentation'
+YARD::Rake::YardocTask.new do |t|
+ t.files = ['lib/**/*.rb', '-', 'LICENSE']
+ t.options = ['--main', 'README.md', '--no-private']
+end
View
84 examples/test_rig.rb
@@ -0,0 +1,84 @@
+#!/usr/bin/env ruby
+# $:.unshift File.expand_path('../lib', File.dirname(__FILE__))
+
+require 'goliath'
+require 'goliath/contrib/rack/configurator'
+require 'goliath/contrib/rack/diagnostics'
+require 'goliath/contrib/rack/force_delay'
+require 'goliath/contrib/rack/force_drop'
+require 'goliath/contrib/rack/force_fault'
+require 'goliath/contrib/rack/force_response'
+require 'goliath/contrib/rack/force_timeout'
+require 'goliath/contrib/rack/handle_exceptions'
+
+#
+# A test endpoint allowing fault injection, variable delay, or a response forced
+# by the client. Besides being a nice demo of those middlewares, it's a useful
+# test dummy for seeing how your SOA apps handle downstream failures.
+#
+# Launch with
+#
+# bundle exec ./examples/test_rig.rb -s -p 9000 -e development &
+#
+# If using it as a test rig, launch with `-e production`. The test rig acts on
+# the following URL parameters:
+#
+# * `_force_timeout` -- raise an error if response takes longer than given time
+# * `_force_delay` -- delay the given length of time before responding
+# * `_force_drop`/`_force_drop_after` -- drop connection immediately with no response
+# * `_force_fail`/`_force_fail_after` -- raise an error of the given type (eg `_force_fail_pre=400` causes a BadRequestError)
+# * `_force_status`, `_force_headers`, or `_force_body' -- replace the given component directly.
+#
+# @example delay for 2 seconds:
+# curl -v 'http://127.0.0.1:9000/?_force_delay=2'
+# => Headers: X-Resp-Delay: 2.0 / X-Resp-Randelay: 0.0 / X-Resp-Actual: 2.003681182861328
+#
+# @example drop connection:
+# curl -v 'http://127.0.0.1:9000/?_force_drop=true'
+#
+# @example delay for 2 seconds, then drop the connection:
+# curl -v 'http://127.0.0.1:9000/?_force_delay=2&_force_drop_after=true'
+#
+# @example force timeout; first call is 200 OK, second will error with 408 RequestTimeoutError:
+# curl -v 'http://127.0.0.1:9000/?_force_timeout=1.0&_force_delay=0.5'
+# => Headers: X-Resp-Delay: 0.5 / X-Resp-Randelay: 0.0 / X-Resp-Actual: 0.513401985168457 / X-Resp-Timeout: 1.0
+# curl -v 'http://127.0.0.1:9000/?_force_timeout=1.0&_force_delay=2.0'
+# => {"status":408,"error":"RequestTimeoutError","message":"Request exceeded 1.0 seconds"}
+#
+# @example simulate a 503:
+# curl -v 'http://127.0.0.1:9000/?_force_fault=503'
+# => {"status":503,"error":"ServiceUnavailableError","message":"Injected middleware fault 503"}
+#
+# @example force-set headers and body:
+# curl -v -H "Content-Type: application/json" --data-ascii '{"_force_headers":{"X-Question":"What is brown and sticky"},"_force_body":{"answer":"a stick"}}' 'http://127.0.0.1:9001/'
+# => {"answer":"a stick"}
+#
+class TestRig < Goliath::API
+ include Goliath::Contrib::CaptureHeaders
+
+ def self.set_middleware!
+ use Goliath::Rack::Heartbeat # respond to /status with 200, OK (monitoring, etc)
+ use Goliath::Rack::Tracer # log trace statistics
+ use Goliath::Rack::DefaultMimeType # cleanup accepted media types
+ use Goliath::Rack::Render, 'json' # auto-negotiate response format
+ use Goliath::Contrib::Rack::HandleExceptions # turn raised errors into HTTP responses
+ use Goliath::Rack::Params # parse & merge query and body parameters
+
+ # turn params like '_force_delay' into env vars :force_delay
+ use(Goliath::Contrib::Rack::ConfigurateFromParams,
+ [ :force_timeout, :force_drop, :force_drop_after, :force_fault, :force_fault_after,
+ :force_status, :force_headers, :force_body, :force_delay, :force_randelay, ],)
+
+ use Goliath::Contrib::Rack::ForceTimeout # raise an error if response takes longer than given time
+ use Goliath::Contrib::Rack::ForceDrop # drop connection immediately with no response
+ use Goliath::Contrib::Rack::ForceFault # raise an error of the given type (eg `_force_fault=400` causes a BadRequestError)
+ use Goliath::Contrib::Rack::ForceResponse # replace as given by '_force_status', '_force_headers' or '_force_body'
+ use Goliath::Contrib::Rack::ForceDelay # force response to take at least (_force_delay + rand*_force_randelay) seconds
+ use Goliath::Contrib::Rack::Diagnostics # summarize the request in the response headers
+ end
+ self.set_middleware!
@igrigorik Owner

what's the reason for class method + invocation vs plain use?

@mrflip Collaborator
mrflip added a note

so that other things can inherit from TestRig and get its middleware stack -- is there a cleaner way to do that?

@igrigorik Owner

Oh, hmm.. interesting. I guess in the router code we had something similar, where we automatically walked up the tree and injected parents middleware. I think it's a reasonable thing to do and support in Goliath in the subclass use case.. we should pull this out into a separate story/bug and tackle it there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+ def response(env)
+ [200, { 'X-API' => self.class.name }, {}]
+ end
+end
View
59 goliath-contrib.gemspec
@@ -1,23 +1,52 @@
# -*- encoding: utf-8 -*-
+$:.push File.expand_path("../lib", __FILE__)
+require 'goliath/contrib/version'
-require File.expand_path('../lib/goliath/contrib', __FILE__)
+Gem::Specification.new do |s|
+ s.name = "goliath-contrib"
+ s.version = Goliath::Contrib::VERSION
-# require './lib/goliath/contrib'
+ s.authors = ["goliath-io"]
+ s.email = ["goliath-io@googlegroups.com"]
-Gem::Specification.new do |gem|
- gem.authors = ["goliath-io"]
- gem.email = ["goliath-io@googlegroups.com"]
+ s.homepage = "https://github.com/postrank-labs/goliath-contrib"
+ s.summary = "Contributed Goliath middleware, plugins, and utilities"
+ s.description = s.summary
- gem.homepage = "https://github.com/postrank-labs/goliath-contrib"
- gem.description = "Contributed Goliath middleware, plugins, and utilities"
- gem.summary = gem.description
+ s.required_ruby_version = '>=1.9.2'
- gem.files = `git ls-files`.split($\)
- gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
- gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
- gem.name = "goliath-contrib"
- gem.require_paths = ["lib"]
- gem.version = Goliath::Contrib::VERSION
+ s.add_dependency 'goliath'
- gem.add_dependency 'goliath'
+ s.add_development_dependency 'rspec', '>2.0'
+
+ s.add_development_dependency 'em-http-request', '>=1.0.0'
+ s.add_development_dependency 'postrank-uri'
+
+ s.add_development_dependency 'guard'
+ s.add_development_dependency 'guard-rspec'
+ if RUBY_PLATFORM.include?('darwin')
+ s.add_development_dependency 'growl', '~> 1.0.3'
+ s.add_development_dependency 'rb-fsevent'
+ end
+
+ if RUBY_PLATFORM != 'java'
+ s.add_development_dependency 'yajl-ruby'
+ s.add_development_dependency 'bluecloth'
+ s.add_development_dependency 'bson_ext'
+ else
+ s.add_development_dependency 'json-jruby'
+ s.add_development_dependency 'maruku'
+ end
+
+ ignores = File.readlines(".gitignore").grep(/\S+/).map {|i| i.chomp }
+ dotfiles = [".gemtest", ".gitignore", ".rspec", ".yardopts"]
+
+ # s.files = `git ls-files`.split($\)
+ # s.executables = s.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+ # s.test_files = s.files.grep(%r{^(test|spec|features)/})
+ # s.require_paths = ["lib"]
+
+ s.files = Dir["**/*"].reject {|f| File.directory?(f) || ignores.any? {|i| File.fnmatch(i, f) } } + dotfiles
+ s.test_files = s.files.grep(/^spec\//)
+ s.require_paths = ['lib']
end
View
6 lib/goliath/contrib.rb
@@ -1,11 +1,9 @@
+require 'goliath/contrib'
+
# TODO: tries to start server :-)
# require 'goliath'
module Goliath
- module Contrib
- VERSION = "1.0.0.beta1"
- end
-
# autoload :MiddlewareName, "goliath/contrib/middleware_name"
# ...
end
View
48 lib/goliath/contrib/rack/configurator.rb
@@ -0,0 +1,48 @@
+module Goliath
+ module Contrib
+ module Rack
+
+ # Place static values fron initialize into the env on each request
@igrigorik Owner

fron > from :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ class StaticConfigurator
+ include Goliath::Rack::AsyncMiddleware
+
+ def initialize(app, env_vars)
+ @extra_env_vars = env_vars
+ super(app)
+ end
+
+ def call(env,*)
+ env.merge!(@extra_env_vars)
+ super
+ end
+ end
+
+ #
+ #
+ # @example imposes a timeout if 'rapid_timeout' param is present
+ # class RapidServiceOrYour408Back < Goliath::API
+ # use Goliath::Rack::Params
+ # use ConfigurateFromParams, [:timeout], 'rapid'
+ # use Goliath::Contrib::Rack::ForceTimeout
+ # end
+ #
+ class ConfigurateFromParams
@igrigorik Owner

ConfigureFromParams? :)

@igrigorik Owner

Hmm, so this middleware is just syntax sugar to turn "x_y" into "x" key in env?

@mrflip Collaborator
mrflip added a note

this lets me inject behavior deep into the stack -- I can put a param foo_force_drop, ensure intervening proxies and middlewares preserve that parameter, and only when it hits the foo_ layer does it go from param into the environment (and thus trigger the middleware in question).

@igrigorik Owner

So why is just the order of the middleware execution not sufficient to match this case? Still trying to wrap my head around the use case.. Is it parameter name collisions with other upstream proxies, so you're trying to work around that?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ include Goliath::Rack::AsyncMiddleware
+
+ def initialize(app, param_keys, slug='')
+ @extra_env_vars = param_keys.inject({}){|acc,el| acc[el.to_sym] = [slug, el].join("_") ; acc }
+ super(app)
+ end
+
+ def call(env,*)
+ @extra_env_vars.each do |env_key, param_key|
+ # env.logger.info [env_key, param_key, env.params[param_key]]
+ env[env_key] ||= env.params.delete(param_key)
+ end
+ super
+ end
+ end
+
+ end
+ end
+end
View
58 lib/goliath/contrib/rack/diagnostics.rb
@@ -0,0 +1,58 @@
+module Goliath
+ module Contrib
+
+ # saves client headers into env[:client_headers]
+ #
+ # This is a module, not middleware: apps should `include` (not `use`) it.
+ # Also, please call 'super' if your app implements `on_headers`.
+ #
+ # @example
+ # class AwesomeApp < Goliath::API
+ # include Goliath::Contrib::CaptureHeaders
+ # end
+ #
+ module CaptureHeaders
+ # save client headers (only) into env[:client_headers]
+ def on_headers(env, headers)
+ env[:client_headers] = headers
+ super(env, headers) if defined?(super)
+ end
+ end
+
+ module Rack
+
+ #
+ # Add headers showing the request's parameters, path, headers and method
+ #
+ # Please also include Goliath::Contrib::CaptureHeaders in your app class.
+ #
+ # @example
+ # class AwesomeApp < Goliath::API
+ # include Goliath::Contrib::CaptureHeaders
+ # use Goliath::Contrib::Rack::Diagnostics
+ # end
+ #
+ class Diagnostics
+ include Goliath::Rack::AsyncMiddleware
+
+ def request_diagnostics(env)
+ client_headers = env[:client_headers] or env.logger.info("Please 'include Goliath::Contrib::CaptureHeaders' in your API class")
@igrigorik Owner

Why can't this be done automatically in this code?

@mrflip Collaborator
mrflip added a note

Is there a way to capture headers in middleware? I only know how to in the Goliath::API app itself.

@igrigorik Owner

Ah, hmm.. Well, you should have all the "rackified" headers directly in the ENV hash (as per Rack spec). I guess if you wanted the raw headers, then that's a bit different.. /cc @dj2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ req_params = env.params.collect{|param| param.join(": ") }
+ req_headers = (client_headers||{}).collect{|param| param.join(": ") }
+ {
+ "X-Next" => @app.class.name,
+ "X-Req-Params" => req_params.join("|"),
+ "X-Req-Path" => env[Goliath::Request::REQUEST_PATH],
+ "X-Req-Headers" => req_headers.join("|"),
+ "X-Req-Method" => env[Goliath::Request::REQUEST_METHOD] }
+ end
+
+ def post_process env, status, headers, body
+ headers.merge!(request_diagnostics(env))
+ [status, headers, body]
+ end
+ end
+
+ end
+ end
+end
View
51 lib/goliath/contrib/rack/force_delay.rb
@@ -0,0 +1,51 @@
+module Goliath
+ module Contrib
+
+ module Rack
+
+ # Delays response for `delay + (0 to randelay)` additional seconds after
+ # the app's response.
+ #
+ # This delay is non-blocking -- *other* requests may proceed in turn --
+ # though naturally the call chain for this response doesn't proceed until
+ # the delay is complete.
+ #
+ # ForceDelay ensures your response takes *at least* N seconds. Force
+ # Timeout ensures your response takes *at most* N seconds. To have a
+ # response take *as-close-as-reasonable-to* N seconds, use an N-second
+ # ForceTimeout with an (N+1)-second ForceDelay.
@igrigorik Owner

this line is talking about ForceTime + ForceDelay, but later we switch to force_delay and force_randelay - seems inconsistent. Am I missing something?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ #
+ # The `force_delay` and `force_randelay` env variables specify the delay;
+ # values are clamped to be less than 5 seconds. Information about the
+ # delay is added to the response headers for your enjoyment.
+ #
+ # @example simulate a highly variable (0.5-1.5 sec) response time (see examples/test_rig.rb):
+ # curl -v 'http://127.0.0.1:9000/?_force_delay=0.5&_force_randelay=1.0'
+ # => Headers: X-Resp-Delay: 0.5 / X-Resp-Randelay: 1.0 / X-Resp-Actual: 0.90205979347229
+ #
+ class ForceDelay
+ include Goliath::Rack::AsyncMiddleware
+
+ def post_process(env, status, headers, body)
+ delay = env[:force_delay].to_f
+ randelay = env[:force_randelay].to_f
+ #
+ if (delay > 0) || (randelay > 0)
+ be_sleepy(delay, randelay)
+ actual = (Time.now.to_f - env[:start_time])
+ headers.merge!( 'X-Resp-Delay' => delay.to_s, 'X-Resp-Randelay' => randelay.to_s, 'X-Resp-Actual' => actual.to_s )
+ end
+ [status, headers, body]
+ end
+
+ # sleep time limited to 5 seconds
+ def be_sleepy(delay, randelay)
+ total = delay + (randelay * rand)
+ total = [0, [total, 5].min].max # clamp
+ #
+ EM::Synchrony.sleep(total)
+ end
+ end
+ end
+ end
+end
View
42 lib/goliath/contrib/rack/force_drop.rb
@@ -0,0 +1,42 @@
+module Goliath
+ module Contrib
+ module Rack
+
+ # Middleware to simulate dropping a connection.
@igrigorik Owner

Is this purely for testing? What's the use case? Seems of limited application. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ #
+ # * if the force_drop env var is given, close the connection as soon as possible
+ # * if the force_drop_after env var is given, close the connection late (after all following middlewares have happened)
+ #
+ # @example drop the connection immediately (see examples/test_rig.rb):
+ # time curl 'http://localhost:9000/?_force_drop=true'
+ # => curl: (52) Empty reply from server
+ # real 0m0.027s user 0m0.008s sys 0m0.005s
+ #
+ # @example drop the connection with no response after waiting one second; the delay is provided by the `ForceDelay` middleware in `_drop_after` mode:
+ # time curl 'http://localhost:9000/?_drop_after=true&_delay=1'
+ # => curl: (52) Empty reply from server
+ # real 0m1.111s user 0m0.008s sys 0m0.005s
+ #
+ class ForceDrop
+ include Goliath::Rack::AsyncMiddleware
+
+ def call(env)
+ return super unless env[:force_drop].to_s == 'true'
+
+ env.logger.info "Forcing dropped connection"
+ env.stream_close
+ [0, {}, {}]
+ end
+
+ def post_process(env, status, headers, body)
+ return super unless env[:force_drop_after].to_s == 'true'
+
+ env.logger.info "Forcing dropped connection (after having run through other warez)"
+ env.stream_close
+ [0, {}, {}]
+ end
+
+ end
+ end
+ end
+end
View
37 lib/goliath/contrib/rack/force_fault.rb
@@ -0,0 +1,37 @@
+module Goliath
+
+ module Validation ; class InjectedError < Error ; end ; end
+
+ module Contrib
+ module Rack
+
+ # if either the 'force_fault' or 'force_fault_after' env attribute are
+ # given, raise an error. The attribute's value (as an integer) becomes the
+ # response code.
+ #
+ # @example simulate a 503 (see `examples/test_rig.rb`):
+ # curl -v 'http://127.0.0.1:9000/?_force_fault=503'
+ # => {"status":503,"error":"ServiceUnavailableError","message":"Injected middleware fault 503"}
+ #
+ class ForceFault
+ include Goliath::Rack::AsyncMiddleware
+
+ def call(env)
+ if fault_code = env[:force_fault]
+ raise Goliath::Validation::InjectedError.new(fault_code.to_i, "Injected middleware fault #{fault_code}")
+ end
+ super
+ end
+
+ def post_process(env, *)
+ if fault_code = env[:force_fault_after]
+ raise Goliath::Validation::InjectedError.new(fault_code.to_i, "Injected middleware fault #{fault_code} (after response was composed)")
+ end
+ super
+ end
+
+ end
+
+ end
+ end
+end
View
28 lib/goliath/contrib/rack/force_response.rb
@@ -0,0 +1,28 @@
+module Goliath
+ module Contrib
+ module Rack
+
+ # if force_status, force_headers or force_body env attributes are present,
+ # blindly substitute the attribute's value, clobbering whatever was there.
+ #
+ # @example setting headers with a JSON post body
+ # curl -v -H "Content-Type: application/json" --data-ascii '{"_force_headers":{"X-Question":"What is brown and sticky"},"_force_body":{"answer":"a stick"}}' 'http://127.0.0.1:9001/'
+ # => {"answer":"a stick"}
+ #
+ # @example force a boring response body so ab doesn't whine about a varying response body size:
+ # ab -n 10000 -c 100 'http://localhost:9000/?_force_body=OK'
+ #
+ class ForceResponse
+ include Goliath::Rack::AsyncMiddleware
+
+ def post_process(env, status, headers, body)
+ if (force_status = env[:force_status]) then status = force_status.to_i ; end
+ if (force_headers = env[:force_headers]) then headers = force_headers ; end
+ if (force_body = env[:force_body]) then body = force_body ; end
+ [status, headers, body]
+ end
+
+ end
+ end
+ end
+end
View
71 lib/goliath/contrib/rack/force_timeout.rb
@@ -0,0 +1,71 @@
+module Goliath
+ module Contrib
+ module Rack
+
+ #
+ # Force a timeout after given number of seconds
+ #
+ # ForceTimeout ensures your response takes *at most* N seconds. ForceDelay
+ # ensures your response takes *at least* N seconds. To have a response
+ # take *as-close-as-reasonable-to* N seconds, use an N-second ForceTimeout
+ # with an (N+1)-second ForceDelay.
+ #
+ #
+ # @example first call is 200 OK, second will error with 408 RequestTimeoutError (see examples/test_rig.rb):
+ # curl -v 'http://127.0.0.1:9000/?_force_timeout=1.0&_force_delay=0.5'
+ # => Headers: X-Resp-Delay: 0.5 / X-Resp-Randelay: 0.0 / X-Resp-Actual: 0.513401985168457 / X-Resp-Timeout: 1.0
+ # curl -v 'http://127.0.0.1:9000/?_force_timeout=1.0&_force_delay=2.0'
+ # => {"status":408,"error":"RequestTimeoutError","message":"Request exceeded 1.0 seconds"}
+ #
+ class ForceTimeout
+ include Goliath::Rack::Validator
+
+ # @param app [Proc] The application
+ # @return [Goliath::Rack::AsyncMiddleware]
+ def initialize(app)
+ @app = app
+ end
+
+ # @param env [Goliath::Env] The goliath environment
+ # @return [Array] The [status_code, headers, body] tuple
+ def call(env, *args)
+ timeout = [0.0, [env[:force_timeout].to_f, 10.0].min].max
+
+ if (timeout != 0.0)
+ async_cb = env['async.callback']
+ env[:force_timeout_complete] = false
+
+ # Normal callback, executed by downstream middleware
+ # If not handled elsewhere, mark as handled and pass along unchanged
+ env['async.callback'] = Proc.new do |status, headers, body|
+ unless env[:force_timeout_complete]
+ env[:force_timeout_complete] = true
+ headers.merge!('X-Resp-Timeout' => timeout.to_s)
+ async_cb.call([status, headers, body])
+ end
+ end
+
+ # timeout callback, executed by EM timer.
+ # This will always fire, we just don't do anything if already handled.
+ # If not handled elsewhere, mark as handled and raise an error
+ EM.add_timer(timeout) do
+ unless env[:force_timeout_complete]
+ env[:force_timeout_complete] = true
+ err = Goliath::Validation::RequestTimeoutError.new("Request exceeded #{timeout} seconds")
+ async_cb.call(error_response(err, 'X-Resp-Timeout' => timeout.to_s))
+ end
+ end
+ end
+
+ status, headers, body = @app.call(env)
+
+ if status == Goliath::Connection::AsyncResponse.first
+ env[:force_timeout_complete] = true
@igrigorik Owner

Instead of the checks above, should probably store the timer and simply cancel it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ end
+ [status, headers, body]
+ end
+
+ end
+ end
+ end
+end
View
63 lib/goliath/contrib/rack/handle_exceptions.rb
@@ -0,0 +1,63 @@
+module Goliath
+ module Contrib
+ module Rack
+
+ # Rescue validation errors in the app just as you do
+ # in middleware
@igrigorik Owner

We recently added some logic to behave differently based on which environment you're running in (dev, prod). This should probably follow same patter, and.. should this be in core instead? What's the difference between this and current behavior in master?

@mrflip Collaborator
mrflip added a note

This was written prior to the changes you mentioned, but compared to the old error code:

  • lets you preserve headers, and puts some details in header,
  • works with the exception object rather than shipping around all its pieces
  • emits a richer hash
  • handles errors in middleware same as errors in app (this has been partially addressed I think, but see below)

I'd prefer it in core, and will fix up a pull request for review.

You'll still want the HandleExceptions class, because you want "normal" errors (database errors, bad requests, 404, etc) handled right before the render part of the stack. HandleException and everything below it supply a hash as the body; the render layer and everything above it supply a string as the body. If an error occurs, this will emit it a hash and be serialized same as anything else: CORS headers, correct serialization format, reporting, etc still occur.

@igrigorik Owner

sgtm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ #
+ # Place this as early as possible in the request chain, but after the rendering.
+ #
+ # @example For JSON-encoded responses, good and bad:
+ # class AwesomeApp < Goliath::API
+ # use Goliath::Rack::DefaultMimeType # cleanup accepted media types
+ # use Goliath::Rack::Render, 'json' # auto-negotiate response format
+ # use Goliath::Contrib::Rack::HandleExceptions # turn raised errors into HTTP responses
+ # use Goliath::Rack::Params # parse & merge query and body parameters
+ # # ... awesomeness goes here ...
+ # end
+ #
+ class HandleExceptions
+ include Goliath::Rack::AsyncMiddleware
+ include Goliath::Rack::Validator
+
+ def call(env)
+ safely(env){ super }
+ end
+ end
+ end
+ end
+
+ module Rack
+ module Validator
+ module_function
+
+ # @param status_code [Integer] HTTP status code for this error.
+ # @param msg [String] message to inject into the response body.
+ # @param headers [Hash] Response headers to preserve in an error response;
+ # (the Content-Length header, if any, is removed)
+ def validation_error(status_code, msg, headers={})
+ err_class = Goliath::HTTP_ERRORS[status_code.to_i]
+ err = err_class ? err_class.new(msg) : Goliath::Validation::Error.new(status_code, msg)
+ error_response(err, headers)
+ end
+
+ # @param err [Goliath::Validation::Error] error to describe in response
+ # @param headers [Hash] Response headers to preserve in an error response;
+ # (the Content-Length header, if any, is removed)
+ def error_response(err, headers={})
+ headers.merge!({
+ 'X-Error-Message' => err.class.default_message,
+ 'X-Error-Detail' => err.message,
+ })
+ headers.delete('Content-Length')
+ body = {
+ status: err.status_code,
+ error: err.class.to_s.gsub(/.*::/,""),
+ message: err.message,
+ }
+ [err.status_code, headers, body]
+ end
+
+ end
+ end
+end
View
5 lib/goliath/contrib/version.rb
@@ -0,0 +1,5 @@
+module Goliath
+ module Contrib
+ VERSION = "1.0.0.beta1"
+ end
+end
View
6 spec/goliath/contrib_spec.rb
@@ -0,0 +1,6 @@
+
+describe Goliath::Contrib do
+ it 'has a version' do
+ Goliath::Contrib::VERSION.should be_a(String)
+ end
+end
View
14 spec/spec_helper.rb
@@ -0,0 +1,14 @@
+require 'bundler'
+
+Bundler.setup
+Bundler.require
+
+require 'goliath/test_helper'
+
+Goliath.env = :test
+
+RSpec.configure do |c|
+ c.include Goliath::TestHelper, :example_group => {
+ :file_path => /spec\/integration/
+ }
+end
Something went wrong with that request. Please try again.