middleware importance ? #88

enomado opened this Issue Apr 6, 2011 · 3 comments


None yet

2 participants


First of all im writing web crawler influenced by python's Scrapy. I just want to say that middleware in library level really dont needed because we simply can change headers like we want before request, and do anything we want with response. Although It could be per-request instead of per-connection.

I believe programs which using high-performance downloaders already must have scheduler level, they must hande opened connections and dispatch them on associated urls e.t.c. So this level is ideal for such things like middlewares...


Middleware is certainly an optional part of the stack - if you don't need it, don't pay attention to it! ;-) At the moment, you can register both, global and per request middleware.

Even though you can obviously achieve same or similar effect by fiddling with most of the variables prior to dispatching a request (or right after), the original use-case and idea for the middleware is/was to allow you to extract common patterns into reusable bits of code. For example, which very simple, automatic JSON parsing of the return body will save you from have to do an explicit decode every single time.

The OAuth middleware is a good example where having an abstracted layer to handle the necessary logic is very useful.

@igrigorik igrigorik closed this Apr 6, 2011

global and per request middleware.

Per connection you mean.

Currently JSON parsing middleware is only one string of code and we can do that by mixing in Response by

module JSONable
  def json
     @json || = parse(@body)

I just sad when i see lifting middleware logic on 3 levels: Global, Connection, Request(?) when we can do the same by directly inheritance Requests and Responses with the same code reuse...


It's a prototype feature, if you don't need it, we don't need to use it. :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment