Permalink
Browse files

async http stack! going meta, query yourself.. to hit async mysql!

  • Loading branch information...
1 parent 72ea384 commit 6307f3f416f21a40304d2f4a07509b923051744b @igrigorik committed Jun 19, 2010
Showing with 26 additions and 1 deletion.
  1. +4 −0 .gitignore
  2. +8 −1 Gemfile
  3. +12 −0 app/controllers/widgets_controller.rb
  4. +2 −0 app/models/widget.rb
View
@@ -0,0 +1,4 @@
+.bundle
+db/*.sqlite3
+log/*.log
+tmp/**/*
View
@@ -3,12 +3,19 @@ source 'http://rubygems.org'
gem 'rails', '3.0.0.beta4'
# async activerecord requires
-gem 'em-synchrony', :git => 'git://github.com/igrigorik/em-synchrony.git', :require => 'em-synchrony'
gem 'mysqlplus', :git => 'git://github.com/oldmoe/mysqlplus.git', :require => 'mysqlplus'
gem 'em-mysqlplus', :git => 'git://github.com/igrigorik/em-mysqlplus.git', :require => 'em-activerecord'
gem 'rack-fiber_pool', :require => 'rack/fiber_pool'
+# async http requires
+gem 'em-synchrony', :git => 'git://github.com/igrigorik/em-synchrony.git', :require => 'em-synchrony/em-http'
+gem 'em-http-request',:git => 'git://github.com/igrigorik/em-http-request.git', :require => 'em-http'
+gem 'addressable', :require => 'addressable/uri'
+
+
+# require 'em-synchrony/em-http'
+
# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'
@@ -0,0 +1,12 @@
+class WidgetsController < ApplicationController
+ def index
+ Widget.find_by_sql("select sleep(1)")
+ render :text => "Oh hai"
+ end
+
+ def http
+ # going meta, query yourself, on the same thin server!
+ http = EM::HttpRequest.new("http://localhost:3000/widgets").get
@jormon
jormon Apr 7, 2011

how would we catch an error here, if for instance we were trying to grab a url from an address/server that was not available? afaict, the only way would be something hacky like

failed = false
http = EM::HttpRequest.new(...).get
http.errback { failed = true }
if failed
    render : text => 'unreachable', :status => 500
else
    render :text => http.response
end
@igrigorik
igrigorik Apr 7, 2011 Owner

You don't need the errback. Check the status of the http object directly once its returned.

ex: http.response_header.status

@swistaczek
swistaczek Jun 9, 2012

Guys, is there any irc room that I can find you @igrigorik :)?

+ render :text => http.response
+ end
+end
@@ -0,0 +1,2 @@
+class Widget < ActiveRecord::Base
+end

11 comments on commit 6307f3f

@maxwell

hey Ilya,

This looks great, it is awesome see all of these async parts working together.

Perhaps a silly question: with em-synchrony, do you recommend querying your own app thru a recursive call as to make sure the DB calls are properly wrapped in fibers, or did you just set it up this way so you could just test your server using your server :)

Thanks so much for the clarification.

@igrigorik
Owner

The call to itself is more for the demo effect -- it shows that the same reactor can call on itself without causing a deadlock. In practice, calling on yourself would be kinda silly, since in theory, whatever action/controller you're calling.. you have access to in your Rails instance already. :-)

@shaokun
shaokun commented on 6307f3f Aug 5, 2010

http = EM::HttpRequest.new("http://localhost:3000/widgets").get
render :text => http.response

I don't quite get this. I thought you need to do render within a callback block from EM::HttpRequest.
EM::HttpRequest won't block the thread, right? So, how can you get the http object and do render right away?

@igrigorik
Owner

Ah, that's the real nice thing of this setup. Every request is wrapped into a fiber, which means we can drop the callbacks and emulate a completely blocking API without blocking the thread or the Ruby runtime. The request is fired in async fashion, but em-synchrony resumes the execution once the callback fires. Take a look at em-synchrony: http://github.com/igrigorik/em-synchrony and also, this might be helpful: http://www.slideshare.net/igrigorik/no-callbacks-no-threads-cooperative-web-servers-in-ruby-19

@rolfb
rolfb commented on 6307f3f Sep 27, 2010

What would the difference be with using open-uri instead of EM::HttpRequest in this example?

@igrigorik
Owner

Because you're running thin (app server), there is only one thread, so if open-uri blocks for 3 seconds while fetching data, then no other request will be served during that time.

@rolfb
rolfb commented on 6307f3f Sep 27, 2010

So, the entire request isn't async?

@igrigorik
Owner

Not sure what you mean. open-uri is not, it uses net-http under the hood. If you use em-http + em-synchrony then you're fine.

@rolfb
rolfb commented on 6307f3f Sep 27, 2010

My apologies.
By request I meant the one hitting the Rack app.

Scenario would be 5 people hitting the app at the same time and an open-uri call in the controller layer.
Would each hit be blocking since Thin is 1 thread and open-uri isn't using the Fiber pool?

I'm guessing it would since that mostly resembles what you are saying, but I'd like to make sure. Thanks for answering this far.

@igrigorik
Owner

Yep. Open-uri would block the main thread and that means the app-server would be unresponsive and could not process any other request. EM-http works around this by scheduling that fetch logic onto the Eventmachine reactor.

@rolfb
rolfb commented on 6307f3f Sep 27, 2010

Thanks for clearing that up. :)

Please sign in to comment.