Use Typhoeus instead plain Net::HTTP

[mikhailov]

Guys, would you consider to switch to advanced HTTP client like Typhoeus or similar? Advantages are simple: use keepalive, nice DSL to handle timeouts, retries, concurrent requests.

[carlosdp]

We're releasing a version of the library soon that will allow for pluggable HTTP clients.

[mikhailov]

👍

[viktornonov]

@mikhailov I'm using typhoeus through this monkey patch until the new version of the gem is released:

https://gist.github.com/viktornonov/487e0d56ea6ad8873f54

Hopefully it will help you.

[mikhailov]

@viktornonov it definitely helps! is that correct to say, your patch overwrites post method from

twilio-ruby/lib/twilio-ruby/rest/base_client.rb

Lines 44 to 56 in c4ffe31

	[:get, :put, :post, :delete].each do |method|
	method_class = Net::HTTP.const_get method.to_s.capitalize
	define_method method do |path, *args|
	params = twilify(args[0])
	params = {} if params.empty?
	# build the full path unless already given
	path = build_full_path(path, params, method) unless args[1]
	request = method_class.new(path, HTTP_HEADERS)
	request.basic_auth(@account_sid, @auth_token)
	request.form_data = params if [:post, :put].include?(method)
	connect_and_send(request)
	end
	end

? What exactly provides an improvements apart from Accept-Encoding (https://gist.github.com/viktornonov/487e0d56ea6ad8873f54#file-twilio-parallel-rb-L17)?

According to Ruby API docs:

Net::HTTP automatically adds Accept-Encoding for compression of response bodies and automatically decompresses gzip and deflate responses unless a Range header was sent.
Compression can be disabled through the Accept-Encoding: identity header.

I assume Typhoeus with its FFI C performance makes a significant differene, is there any chance to measure and provide results here please?

[viktornonov]

@mikhailov Yes, it overrides the Post method of the

twilio-ruby/lib/twilio-ruby/rest/base_client.rb

Lines 44 to 56 in c4ffe31

	[:get, :put, :post, :delete].each do |method|
	method_class = Net::HTTP.const_get method.to_s.capitalize
	define_method method do |path, *args|
	params = twilify(args[0])
	params = {} if params.empty?
	# build the full path unless already given
	path = build_full_path(path, params, method) unless args[1]
	request = method_class.new(path, HTTP_HEADERS)
	request.basic_auth(@account_sid, @auth_token)
	request.form_data = params if [:post, :put].include?(method)
	connect_and_send(request)
	end
	end

.

I had the problem with a segfault when I send multiple sms messages through a sidekiq worker and this override fixes it. The issue is related to Ruby's Net::HTTP module and openssl and to that Net::HTTP is not thread safe. You can see more details about the problem in #93.

It will be interesting to check the performance change, I can check this and post the results here. What do you think is the best thing that I can use to measure the performance - the Ruby's Benchmark module?

[mikhailov]

@viktornonov for performance testing is only rule - it shouldn't be Apache Bench, a good option is siege. Ruby stdlib benchmark can be good as well, but can you perform true parallel testing with Ruby benchmark class?

[carlosdp]

If you upgrade to the 5.0.0.rc2 release candidate, it ships with Faraday by default and a bunch of other features. You can use Typhoeus easily using:

@client.http_client.adapter = :typhoeus

See the upgrade wiki for more details!

[mikhailov]

@carlosdp 👍