Permalink
Browse files

async http stack! going meta, query yourself.. to hit async mysql!

  • Loading branch information...
igrigorik committed Jun 19, 2010
1 parent 72ea384 commit 6307f3f416f21a40304d2f4a07509b923051744b
Showing with 26 additions and 1 deletion.
  1. +4 −0 .gitignore
  2. +8 −1 Gemfile
  3. +12 −0 app/controllers/widgets_controller.rb
  4. +2 −0 app/models/widget.rb
View
@@ -0,0 +1,4 @@
+.bundle
+db/*.sqlite3
+log/*.log
+tmp/**/*
View
@@ -3,12 +3,19 @@ source 'http://rubygems.org'
gem 'rails', '3.0.0.beta4'
# async activerecord requires
-gem 'em-synchrony', :git => 'git://github.com/igrigorik/em-synchrony.git', :require => 'em-synchrony'
gem 'mysqlplus', :git => 'git://github.com/oldmoe/mysqlplus.git', :require => 'mysqlplus'
gem 'em-mysqlplus', :git => 'git://github.com/igrigorik/em-mysqlplus.git', :require => 'em-activerecord'
gem 'rack-fiber_pool', :require => 'rack/fiber_pool'
+# async http requires
+gem 'em-synchrony', :git => 'git://github.com/igrigorik/em-synchrony.git', :require => 'em-synchrony/em-http'
+gem 'em-http-request',:git => 'git://github.com/igrigorik/em-http-request.git', :require => 'em-http'
+gem 'addressable', :require => 'addressable/uri'
+
+
+# require 'em-synchrony/em-http'
+
# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'
@@ -0,0 +1,12 @@
+class WidgetsController < ApplicationController
+ def index
+ Widget.find_by_sql("select sleep(1)")
+ render :text => "Oh hai"
+ end
+
+ def http
+ # going meta, query yourself, on the same thin server!
+ http = EM::HttpRequest.new("http://localhost:3000/widgets").get

This comment has been minimized.

Show comment Hide comment
@jormon

jormon Apr 7, 2011

how would we catch an error here, if for instance we were trying to grab a url from an address/server that was not available? afaict, the only way would be something hacky like

failed = false
http = EM::HttpRequest.new(...).get
http.errback { failed = true }
if failed
    render : text => 'unreachable', :status => 500
else
    render :text => http.response
end
@jormon

jormon Apr 7, 2011

how would we catch an error here, if for instance we were trying to grab a url from an address/server that was not available? afaict, the only way would be something hacky like

failed = false
http = EM::HttpRequest.new(...).get
http.errback { failed = true }
if failed
    render : text => 'unreachable', :status => 500
else
    render :text => http.response
end

This comment has been minimized.

Show comment Hide comment
@igrigorik

igrigorik Apr 7, 2011

Owner

You don't need the errback. Check the status of the http object directly once its returned.

ex: http.response_header.status

@igrigorik

igrigorik Apr 7, 2011

Owner

You don't need the errback. Check the status of the http object directly once its returned.

ex: http.response_header.status

This comment has been minimized.

Show comment Hide comment
@swistaczek

swistaczek Jun 9, 2012

Guys, is there any irc room that I can find you @igrigorik :)?

@swistaczek

swistaczek Jun 9, 2012

Guys, is there any irc room that I can find you @igrigorik :)?

+ render :text => http.response
+ end
+end
View
@@ -0,0 +1,2 @@
+class Widget < ActiveRecord::Base
+end

11 comments on commit 6307f3f

@maxwell

This comment has been minimized.

Show comment Hide comment
@maxwell

maxwell Jun 20, 2010

hey Ilya,

This looks great, it is awesome see all of these async parts working together.

Perhaps a silly question: with em-synchrony, do you recommend querying your own app thru a recursive call as to make sure the DB calls are properly wrapped in fibers, or did you just set it up this way so you could just test your server using your server :)

Thanks so much for the clarification.

hey Ilya,

This looks great, it is awesome see all of these async parts working together.

Perhaps a silly question: with em-synchrony, do you recommend querying your own app thru a recursive call as to make sure the DB calls are properly wrapped in fibers, or did you just set it up this way so you could just test your server using your server :)

Thanks so much for the clarification.

@igrigorik

This comment has been minimized.

Show comment Hide comment
@igrigorik

igrigorik Jun 20, 2010

Owner

The call to itself is more for the demo effect -- it shows that the same reactor can call on itself without causing a deadlock. In practice, calling on yourself would be kinda silly, since in theory, whatever action/controller you're calling.. you have access to in your Rails instance already. :-)

Owner

igrigorik replied Jun 20, 2010

The call to itself is more for the demo effect -- it shows that the same reactor can call on itself without causing a deadlock. In practice, calling on yourself would be kinda silly, since in theory, whatever action/controller you're calling.. you have access to in your Rails instance already. :-)

@shaokun

This comment has been minimized.

Show comment Hide comment
@shaokun

shaokun Aug 5, 2010

http = EM::HttpRequest.new("http://localhost:3000/widgets").get
render :text => http.response

I don't quite get this. I thought you need to do render within a callback block from EM::HttpRequest.
EM::HttpRequest won't block the thread, right? So, how can you get the http object and do render right away?

http = EM::HttpRequest.new("http://localhost:3000/widgets").get
render :text => http.response

I don't quite get this. I thought you need to do render within a callback block from EM::HttpRequest.
EM::HttpRequest won't block the thread, right? So, how can you get the http object and do render right away?

@igrigorik

This comment has been minimized.

Show comment Hide comment
@igrigorik

igrigorik Aug 5, 2010

Owner

Ah, that's the real nice thing of this setup. Every request is wrapped into a fiber, which means we can drop the callbacks and emulate a completely blocking API without blocking the thread or the Ruby runtime. The request is fired in async fashion, but em-synchrony resumes the execution once the callback fires. Take a look at em-synchrony: http://github.com/igrigorik/em-synchrony and also, this might be helpful: http://www.slideshare.net/igrigorik/no-callbacks-no-threads-cooperative-web-servers-in-ruby-19

Owner

igrigorik replied Aug 5, 2010

Ah, that's the real nice thing of this setup. Every request is wrapped into a fiber, which means we can drop the callbacks and emulate a completely blocking API without blocking the thread or the Ruby runtime. The request is fired in async fashion, but em-synchrony resumes the execution once the callback fires. Take a look at em-synchrony: http://github.com/igrigorik/em-synchrony and also, this might be helpful: http://www.slideshare.net/igrigorik/no-callbacks-no-threads-cooperative-web-servers-in-ruby-19

@rolfb

This comment has been minimized.

Show comment Hide comment
@rolfb

rolfb Sep 27, 2010

What would the difference be with using open-uri instead of EM::HttpRequest in this example?

What would the difference be with using open-uri instead of EM::HttpRequest in this example?

@igrigorik

This comment has been minimized.

Show comment Hide comment
@igrigorik

igrigorik Sep 27, 2010

Owner

Because you're running thin (app server), there is only one thread, so if open-uri blocks for 3 seconds while fetching data, then no other request will be served during that time.

Owner

igrigorik replied Sep 27, 2010

Because you're running thin (app server), there is only one thread, so if open-uri blocks for 3 seconds while fetching data, then no other request will be served during that time.

@rolfb

This comment has been minimized.

Show comment Hide comment
@rolfb

rolfb Sep 27, 2010

So, the entire request isn't async?

So, the entire request isn't async?

@igrigorik

This comment has been minimized.

Show comment Hide comment
@igrigorik

igrigorik Sep 27, 2010

Owner

Not sure what you mean. open-uri is not, it uses net-http under the hood. If you use em-http + em-synchrony then you're fine.

Owner

igrigorik replied Sep 27, 2010

Not sure what you mean. open-uri is not, it uses net-http under the hood. If you use em-http + em-synchrony then you're fine.

@rolfb

This comment has been minimized.

Show comment Hide comment
@rolfb

rolfb Sep 27, 2010

My apologies.
By request I meant the one hitting the Rack app.

Scenario would be 5 people hitting the app at the same time and an open-uri call in the controller layer.
Would each hit be blocking since Thin is 1 thread and open-uri isn't using the Fiber pool?

I'm guessing it would since that mostly resembles what you are saying, but I'd like to make sure. Thanks for answering this far.

My apologies.
By request I meant the one hitting the Rack app.

Scenario would be 5 people hitting the app at the same time and an open-uri call in the controller layer.
Would each hit be blocking since Thin is 1 thread and open-uri isn't using the Fiber pool?

I'm guessing it would since that mostly resembles what you are saying, but I'd like to make sure. Thanks for answering this far.

@igrigorik

This comment has been minimized.

Show comment Hide comment
@igrigorik

igrigorik Sep 27, 2010

Owner

Yep. Open-uri would block the main thread and that means the app-server would be unresponsive and could not process any other request. EM-http works around this by scheduling that fetch logic onto the Eventmachine reactor.

Owner

igrigorik replied Sep 27, 2010

Yep. Open-uri would block the main thread and that means the app-server would be unresponsive and could not process any other request. EM-http works around this by scheduling that fetch logic onto the Eventmachine reactor.

@rolfb

This comment has been minimized.

Show comment Hide comment
@rolfb

rolfb Sep 27, 2010

Thanks for clearing that up. :)

Thanks for clearing that up. :)

Please sign in to comment.