New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load testing and got {:error, %HTTPoison.Error{id: nil, reason: :connect_timeout}} #291

Closed
dimun opened this Issue Sep 20, 2017 · 10 comments

Comments

Projects
None yet
5 participants
@dimun
Copy link

dimun commented Sep 20, 2017

I'm doing some load tests with locust, my software has a gateway for doing HTTP request in several servers, i'm using HTTPoison in the gateway, but when I got an error for connect_timeout (because of the amount of requests), each single request throw this error {:error, %HTTPoison.Error{id: nil, reason: :connect_timeout}}, even after stopping the locust test, the only way for recovering the gateway is by restarting the gateway server.

If anybody can help me, thank you in advance.

@dannote

This comment has been minimized.

Copy link

dannote commented Oct 22, 2017

benoitc/hackney#420 has been fixed recently. Bumping hackney to 1.10.1 should help.

@dannote

This comment has been minimized.

Copy link

dannote commented Oct 22, 2017

@edgurgel could you please?

@edgurgel

This comment has been minimized.

Copy link
Owner

edgurgel commented Oct 23, 2017

I think you can do mix deps.update hackney you should be able to download the newest hackney.

@dimun dimun closed this Oct 23, 2017

@edgurgel

This comment has been minimized.

Copy link
Owner

edgurgel commented Oct 23, 2017

@dimun, having the newest hackney solved the issues you were noticing?

@svileng

This comment has been minimized.

Copy link

svileng commented Oct 27, 2017

I updated Hackney to 1.10.1, after reading this thread, and sadly I'm still experiencing this issue. My case is not during load testing, but in a production environment. I send at least 300 HTTP requests every 2 minutes to various API servers and I often get %HTTPoison.Error{id: nil, reason: :connect_timeout}, even straight after a restart, and the error seems to persist until another restart, which is not great... Any thoughts? 😞

@dimun

This comment has been minimized.

Copy link
Author

dimun commented Oct 27, 2017

@edgurgel, the first thing that I've done is to change the phoenix environment by prod and that improved the amount of simultaneous users but the error still happened, so I had to change the library, but now that I saw this answer, I've mounted the previous version with httpoison and updated the hackney version and it worked for me. thank you

@svileng

This comment has been minimized.

Copy link

svileng commented Oct 30, 2017

FWIW: To quote the person from the Hackney issue thread (above):

For reference, I noticed the problems still occurs but the fix here means hackney now actually recovers.

Although the error happens for me, still, it does not persist, so as long as you retry your request, it's all good. So upgrading to 1.10.1 is a big improvement.

@dsignr

This comment has been minimized.

Copy link

dsignr commented Jan 9, 2018

This issue unfortunately still persists on production while using Httpoison. Upgrading to Hackney 1.10.1 did reduce the frequency of timeouts, nevertheless, it still happens. Retrying requests isn't really a valid solution, atleast not for me because we're using this for our orders on our store with PayPal API and we got a handful of users complaining that their payment failed. And their API legitimately times out sometimes and we want to be able to track that. With Httpoison/Hackney timing out randomly, it makes it harder to discern between the library and the real API timeouts. Having that said, Httpoison hit 1.0 only like 2 days ago, so I'm going to give this some more time and see where it goes as its unreasonable of me to expect the author to adhere to my personal urgency. Not switching libs as we've invested lots of code w/ Httpoison and also, I love this lib :)

@edgurgel

This comment has been minimized.

Copy link
Owner

edgurgel commented Jan 9, 2018

Have you checked hackney metrics? It may give us more info why this is happening. It tracks number of connections, requests, etc, etc. You may find the reason why you are seeing so many connection timeouts. Maybe the pool is queuing your requests or something like this and it may need some tweaking on the options. https://github.com/benoitc/hackney#metrics

@dsignr

This comment has been minimized.

Copy link

dsignr commented Jan 10, 2018

@edgurgel Wow, thank you very much, I hadn't looked into those options (or anything beyond Httpoison's own options). Thank you :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment