Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

It should be possible to dynamically decide if the Curl_multi should use socketpair #4829

Closed
Andersbakken opened this issue Jan 17, 2020 · 6 comments
Closed

It should be possible to dynamically decide if the Curl_multi should use socketpair #4829

Andersbakken opened this issue Jan 17, 2020 · 6 comments
Assignees
@jay
Labels
enhancement libcurl API

Comments

@Andersbakken
Copy link
Contributor

I did this

I did not use curl_multi_wait in my app

I expected the following

For curl not to allocate a socketpair for each multi handle

curl/libcurl version

master

[curl -V output]

operating system

linux

Ideally Curl should allow one or more of these options:

  1. configure switch to decide if you want to enable socketpairs for waking up the multi
  2. make socket pairs on demand with multi_opt to toggle enabledness

regards
@bagder
Copy link
Member

bagder commented Jan 19, 2020

I presume you want this because the socketpair creates some kind of problem or nuisance for you. Can you elaborate on why you want/need this?

@Andersbakken
Copy link
Contributor Author

Some of our platforms are rather constrained on resources and since we're not calling curl_multi_wakeup the sockets will just sit there.

This wouldn't be so bad for if it weren't for the fact that we have a multi handle per websocket connection. We've implemented websockets on top of CURLOPT_CONNECT_ONLY and ran into some issues with socket pooling.

@jay jay self-assigned this Jan 21, 2020
jay added a commit to jay/curl that referenced this issue Jan 21, 2020
@jay
multi: add curl_multi_init2 for advanced multi init
f150cb2 
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 21, 2020
@jay
multi: add curl_multi_init2 for advanced multi init
c991bb6 
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
@jay jay mentioned this issue Jan 21, 2020
Closed
jay added a commit to jay/curl that referenced this issue Jan 21, 2020
@jay
multi: add curl_multi_init2 for advanced multi init
80b0bd3 
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 22, 2020
@jay
multi: add curl_multi_init2 for advanced multi init
571068a 
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 22, 2020
@jay
multi: add curl_multi_init2 for advanced multi init
3e2d5b6 
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 22, 2020
@jay
multi: add curl_multi_init2 for advanced multi init
71d11e5 
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 22, 2020
@jay
multi: add curl_multi_init2 for advanced multi init
dfc6a8d 
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
jay added a commit to jay/curl that referenced this issue Jan 23, 2020
@jay
multi: add curl_multi_init2 for advanced multi init
7390c90 
Allow initializing a multi with a specific socket table hash size,
connection cache hash size, and flags. Flag
CURL_MULTI_DISABLE_POLL_WAKEUP can be used to disable the
curl_multi_poll wakeup socketpair. A user requested the option to
disable creating those sockets due to resource constraints.

Prior to this change it was not possible to disable the wakeup
socketpair (added in 7.68.0) used by the multi.

Reported-by: Anders Bakken

Fixes curl#4829
Closes #xxxx
@elsamuko elsamuko mentioned this issue Feb 27, 2020
Closed
@rcombs
Copy link
Contributor

rcombs commented May 2, 2020

I'm running into this as well (with easy handles); the extra sockets just sit there and pile up until I run out of dynamic ports on loopback, even if I have connection reuse enabled for my main sockets.
Is there a reason this has to be a socket pair at all, rather than just an anonymous pipe? It's only ever used in one direction.

@jay
Copy link
Member

jay commented Jul 22, 2020

My PR to fix this did not gain enough support. I still think some remedy for this issue is a good idea. Someone will have to implement something more agreeable, or eventually I will close it in known issues (or TODO).

@rcombs
Copy link
Contributor

rcombs commented Jul 22, 2020

Is this a problem for you on non-Windows platforms? If so, what resource are you running out of exactly?

@jay
Copy link
Member

jay commented Aug 27, 2020

#5850 has added a build-time option to --disable-socketpair.

@bagder bagder closed this as completed in af09599 Sep 16, 2020
@bharathkumarkm bharathkumarkm mentioned this issue Apr 14, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jay jay

Labels
enhancement libcurl API
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

multi: add curl_multi_init2 for advanced multi init jay/curl
4 participants
@Andersbakken @bagder @rcombs @jay