increase default pool size #2
Conversation
| {env, [{connection_timeout, 300000}, {pool_size, 50}]} | ||
| %% make default pool ~ all possible ports for a single IP interface. e.g. Usual 1025-65535 | ||
| %% define your pool or other size if you want to limit it or have more IP interfaces | ||
| {env, [{connection_timeout, 300000}, {pool_size, 64000}]} |
There was a problem hiding this comment.
I think 64000 is too aggressive.
what if any other HTTP client is used on the same node?
There was a problem hiding this comment.
if there are lhttpC, hakney, gun, shotgun running together, consumer application has a different problem ;)
plus it does not differ from having other non-erlang apps running and everybody should be ready to handle error of no more TCP sockets
There was a problem hiding this comment.
please share alternative proposal.
There was a problem hiding this comment.
i'd left it as is and let a consumer application to decide how much connections is needed.
There was a problem hiding this comment.
motivation for the change is that I already lost count of how many times people faced an issue of unexpected client failures due to extremely small pool where it always came by surprise that there is any limit at all.
One may say "read the source" and there will be no issues but i feel failing with error on tcp open if more friendly and clearly indicates the issue to the consumer.
There was a problem hiding this comment.
My point is this change may brake existing applications (which use default pool size) after the next release which includes this change. Let's do not do any harm.
There was a problem hiding this comment.
I agree with raising the default limit, I am not sure 3 orders of magnitude increase is needed to ameliorate the problem, and I wonder if it's not likely to result in other unexpected problems (like OOM due to thousands of in-flight requests). Pool size is a back pressure mechanism, and I suspect that there are clients that rely on this subconsciously for whom this could cause a significant unexpected problem. is there a problem with just raising to, say, 500 or 1000?
There was a problem hiding this comment.
let it be 1000 by default but it does not address the problem that some do not know about the existence of default.
|
i'll fix travis in #3 |
|
@waisbrot please add me to owners of lhttpc to HEX so i can publish it |
|
thanks! |
Incorporate mabrek/alertlogic changes
Do canonical headers attempt #2
Problem:
50 connections as a default max is extremely small and outdated.
Solution:
~~make default to almost not limited for a single IP interface. ~~
make it bigger.
If consumer want to limit it or have more they still can
@nalundgaard