Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It should be possible to dynamically decide if the Curl_multi should use socketpair #4829

Closed
Andersbakken opened this issue Jan 17, 2020 · 6 comments

Comments

@Andersbakken
Copy link
Contributor

Andersbakken commented Jan 17, 2020

I did this

I did not use curl_multi_wait in my app

I expected the following

For curl not to allocate a socketpair for each multi handle

curl/libcurl version

master

[curl -V output]

operating system

linux

Ideally Curl should allow one or more of these options:

  1. configure switch to decide if you want to enable socketpairs for waking up the multi
  2. make socket pairs on demand with multi_opt to toggle enabledness

regards

@bagder
Copy link
Member

bagder commented Jan 19, 2020

I presume you want this because the socketpair creates some kind of problem or nuisance for you. Can you elaborate on why you want/need this?

@Andersbakken
Copy link
Contributor Author

Andersbakken commented Jan 21, 2020

Some of our platforms are rather constrained on resources and since we're not calling curl_multi_wakeup the sockets will just sit there.

This wouldn't be so bad for if it weren't for the fact that we have a multi handle per websocket connection. We've implemented websockets on top of CURLOPT_CONNECT_ONLY and ran into some issues with socket pooling.

@jay jay self-assigned this Jan 21, 2020
jay added a commit to jay/curl that referenced this issue Jan 21, 2020
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 21, 2020
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 21, 2020
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 22, 2020
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 22, 2020
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 22, 2020
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 22, 2020
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 23, 2020
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
@rcombs
Copy link
Contributor

rcombs commented May 2, 2020

I'm running into this as well (with easy handles); the extra sockets just sit there and pile up until I run out of dynamic ports on loopback, even if I have connection reuse enabled for my main sockets.
Is there a reason this has to be a socket pair at all, rather than just an anonymous pipe? It's only ever used in one direction.

@jay
Copy link
Member

jay commented Jul 22, 2020

My PR to fix this did not gain enough support. I still think some remedy for this issue is a good idea. Someone will have to implement something more agreeable, or eventually I will close it in known issues (or TODO).

@rcombs
Copy link
Contributor

rcombs commented Jul 22, 2020

Is this a problem for you on non-Windows platforms? If so, what resource are you running out of exactly?

@jay
Copy link
Member

jay commented Aug 27, 2020

#5850 has added a build-time option to --disable-socketpair.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants