Add EM-HttpRequest adapter #40
Conversation
|
This is useful to me. Thanks for writing it. Will it be merged in? |
|
I made use of this also -- please merge it! |
|
+1 |
|
|
||
| def _request(request) | ||
| options = client_options(request) | ||
| client = EventMachine::HttpRequest.new("#{request.url.scheme}://#{request.url.host}:#{request.url.port}#{request.url.path}") |
There was a problem hiding this comment.
I'm using this gem in ruby 1.9.3 and when this called, it was ignoring the options.
I added options:
client = EventMachine::HttpRequest.new("#{request.url.scheme}://#{request.url.host}:#{request.url.port}#{request.url.path}", options)
Has yield changed in 1.9.3 so the options are not passed? I only noticed because I was trying to set the timeouts, and they were using defaults regardless of what I set.
There was a problem hiding this comment.
We haven't upgraded to the latest EventMachine here yet, but from what I understand, a lot of options that used to be on the client are now on the request (or vice versa), so it's possible the gem needs to be updated to pass the correct options to the correct methods #new vs. #request.
There was a problem hiding this comment.
i unfortunately don't know about the current state of eventmachine, but if you guys can let me know whether this pull request is still up to date, i'll gladly merge it in and release a new version asap.
There was a problem hiding this comment.
It would also need to be upgraded to 0.9.5 which has some configuration changes.
|
Awesome! This is very useful to me. Will it be merged in? |
|
if anyone can update this to up-to-date versions of eventmachine and dependencies, i'll make sure to merge it. |
|
I'll see what I can do over the weekend. |
|
+1 A merge into the official gem would be fantastic. I'd like to use evented Savon as well. |
|
We are using this right now, with a recent version of eventmachine, and it works just fine. |
|
Hi All, Just wondering where this is at. I was wanting to use this in my gem github.com/lukebyrne/betfair. I want to be able to add a simple flag when the class is instantiated to switch to using em-http, in much the same way you can specify gzip => true Would someone please spell out to a simple man how I would get the forked version of httpi to override the bundled one in savon, and then to setup savon to use em-http? Cheers, LB |
|
We used a :git entry in our Gemfile, and then used bundler to install everything. Bundler won't install the vanilla httpi gem if you have already installed it from another source. |
|
i merged the changes, fixed a few things and pushed an em_http adapter branch which unfortunately fails the specs |
|
zond - and to need you need to specify that you want to explicitly use em-http when setting up a savon connection. I had a look and it seems that it will try it last or third. How do you change this order? |
|
lukebyrne: You only need to tell HTTPI to use em-http, Savon will use whatever HTTPI is configured to use iirc. We did: |
|
+1 |
|
@rubiii I pulled down the em_http adapter branch and ran the tests on 1.9.3 and they pass for me... Having problems with them running on JRuby 1.6.7 at the moment though (same em-synchrony problem that the travis log shows). I noticed, however, in the travis logs that bundler pulled down eventmachine 1.0.0.beta.4; 1.0.0.rc.4 is out now, so I wonder if the failure you encountered was related to that? I'll let you know if I can figure anything out with the jruby thing... |
|
@rubiii I got the JRuby tests to pass locally by making the Gemfile bundle for :java. The Changing this got them to pass for me on MRI 1.9 and JRuby: gem "curb", "~> 0.7.8", :require => false, :platforms => [:ruby, :jruby]
gem 'em-http-request', :require => false, :platforms => [:ruby, :jruby]
gem 'em-synchrony', :require => false, :platforms => [:ruby, :jruby] |
|
@turboladen thanks for following up on this. i applied your change and it solved the problem with jruby for me. for 1.9.x the integration specs fail with a runtime error. i can reproduce the problem locally, but would need to investigate. and jruby has problems with curb. you can ignore the rbx errors for now. they're related to a problem with gzip. |
|
travis doesn't support c extensions, so i excluded the specs for curb on jruby. |
|
Sorry I went dark on you... I plan on looking into these soon. If I get it working, I'll just submit a pull request. |
|
@zond, after updating the adapter to :em_http, how do you access the callback/errback for a request? |
|
@jonhyman, I don't, we only used this to enable EM connections in Savon so as not to block our Goliath server. |
|
Thanks. It seems like you can get the error by catching an exception. Here's a sample script I've been playing with in my console that illustrates it. I think it works this way because it's dependent on fibers (e.g., you can't run HTTPI methods inside an EM.run when using :em_http, it has to be EM.synchrony) # Note the invalid url for techcrunch...
urls = ["http://www.google.com", "http://www.yahoo.com", "http://www.techcrunch.c"]
results = []
concurrency = 3
HTTPI::Adapter.use = :em_http
EM.synchrony do
EM::Timer.new(2) {EM.stop}
EM::Synchrony::FiberIterator.new(urls, concurrency).each do |url|
r = HTTPI::Request.new
r.url = url
begin
resp = HTTPI.get(r)
results.push(resp)
rescue Exception => e
puts("Error for #{url}")
end
end
end |
|
@jonhyman, yeah that might be true. We only used it inside EM.synchrony. Anyway, it's a long time since I touched those things. Good luck :D |
|
@rubiii I finally got back to checking in to those failures and I can't get any unit tests to fail for me. (I also don't seem to find the failures on Travis anymore, due to your org change.) I wonder if those old failures might have something to do with a dependency that's now been updated? In any case, I forked your em_http branch and all unit tests were passing, but integration tests were not. I have the following rubies installed:
This resulted in all green before and after my changes:
This resulted in failures before my changes, but now passes:
https://github.com/turboladen/httpi/compare/savonrb:em_http...turboladen:em_http. Can you let me know if you're willing to take a pull request? |
|
sure. |
|
hey. i just merged support for em_http into master. please give it a try before it's going to be released. |
|
I'm not sure how I'd handle this, but perhaps you've got a stance on it already... When I use it from a project of mine, I get: My gem doesn't use em-synchrony (and imagine that a good portion of EventMachine users aren't), but the adapter does; this is resolved, of course, if I add em-synchrony to my project. Maybe this should be added to the docs somewhere? I can do that if you can tell me where... Secondly, my gem makes EM::HttpRequest calls (which is used by the adapter) using the standard EventMachine way--these calls are now failing with FiberErrors. This tells me that I need to make changes to my calls to do them em-synchrony style. Again, this still works for me (I think--I've never used em-syncrhony), but seems like it should be documented. FWIW, I made an attempt to remove the em-synchrony dep from the adapter a couple weeks ago, but couldn't figure out how to make it all work. At that point, I undid my changes, and decided I was happy to make my code em-sychnrony compatible, just so I can have an EventMachine compatible HTTPI. |
|
i'm not sure if we need em-synchrony. it sounds like a good idea to use fibers, but i'm not sure. |
|
Well, the adapter seems like it was designed with em-synchrony in mind; if it wasn't, I'm pretty sure there'd be a bunch of callbacks/errbacks defined and the flow would probably be a bit different. I've only been working with EM for a few months, so I'm no pro here though. It'd be great if em-sychrony wasn't a requirement, but maybe that's just something to change for the future? That would also open up compatibility to 1.8.x rubies... In any case, just documenting works great for me for now. Thanks for getting this in. |
|
hey guys. i added some integration spec (3904d3) to verify ssl authentication and it seems like the em_http adapter doesn't support this feature at all (see #65). so i'm wondering where the information about configuring ssl for the em_http adapter came from. it seems more appropriate to remove that method and raise an error when someone tries to use ssl authentication with em_http, because the ssl settings seem to be ignored?! /cc @RISCfuture |
|
I've never used it, but search for As far as I'm concerned, I'm fine without SSL support for this; your suggestion for raising if someone tries to use it seems great. |
|
i'm quite busy right now, so sorry for the delay. i just released 2.0.0.rc1 so you can test the new adapter before i'm releasing a stable version. thanks! |
|
released this feature with version 2.0.0. final just now. if you're using httpi with savon, thanks for your help! |
At my company (recurly.com) we use Active Merchant to communicate with a variety of different payment gateways. Normally Active Merchant just uses
Net::HTTP, but since we're using Thin, we wanted to take advantage of fibers and use EventMachine plus EM-HttpRequest.To do this, I added an em-http-request adapter to HTTPI, and created an
ActiveMerchant::Billing::Net::HTTPobject that responds to the same interface as::Net::HTTPexcept it just hands the request off to HTTPI (clever, eh?). That way, we can use the:em_httpadapter for most gateways, and if a particular gateway requires a feature not supported by EM-HttpRequest, we can fall back on the:net_httpadapter.In addition, we can use evented I/O over Savon to communicate with our vault.