curl_multi_get_handles - get all easy handles in a multi handle

[bagder]

Food for discussion. See https://curl.se/mail/lib-2023-08/0034.html

[MonkeybreadSoftware]

Looks good to me.

[bagder]

@jeroen you think this kind of API works for a binding such as yours, or would it be better served by an iterator-using one?

[jeroen]

Yes this works for me, but my consideration is that this api is not extensible, if you would want to expose additional status information about each handle in the multi. For example, if the request is in queued, in progress, or done, and which connection it uses, incl http version, etc.

There is a possibility to make this a more generic API to list the status of all the requests in the multi, which is something that can be useful e.g. to debug multiplex issues, for example what we discussed here: https://curl.se/mail/lib-2023-02/0024.html

A fun example to toy with might be to improve the crawler to show detailed progress about the pending transfers, and be a bit smarter how to limit the number of requests based on what is already in the queue.

[bagder]

if you would want to expose additional status information about each handle in the multi. For example, if the request is in queued, in progress, or done, and which connection it uses, incl http version, etc.

If we want to provide that info - per easy handle - I think the *get_handles() API is not the right spot anyway. Then we should provide that via curl_easy_getinfo() so you'd iterate over the handles and query each handle for that info.

There is a possibility to make this a more generic API

You mean by instead creating a curl_multi_getinfo() function and making this all-easy-handles be a first "option" that this function can return?

[bagder]

• add a CURLHND parameter to curl_multi_get_handles so that the user can request only their (user) easy handles, or internal handles, or both, in the multi.

Can we think of a use case where users actually want the internally created ones?

If they do, don't they then also want to know which kind each of them is? I mean, it seems like we should rather provide either non-internal or only-internal ones so that the caller knows what kind of handles they are. update: which is what this new option allows.

[jay]

Can we think of a use case where users actually want the internally created ones?

I can't. Maybe I overdid it I'll take away the filter parameter.

[jay]

I should mention my hackish ListEasyHandles example from several years ago that I just updated :)

[rwmjones]

This would work for our use case.

[bagder]

Any further comments before we merge this first version?

Should we mark it experimental in a first round to make it possible to tweak the API ?

[rwmjones]

I read the updated patch series that you posted a few days ago and it still looks fine and would work for us. I have no view on whether it should be experimental or not.

[rwmjones]

I have no view on whether it should be experimental or not.

We would use it right away, but conditionalize the code behind a ./configure-time test, so for us it wouldn't matter.

[bagder]

I intend to merge this as-is and give everyone a chance to try it out for a few weeks. If we have any issues or problems showing up, we can mark it experimental before we do the next release. Otherwise it can ship as-is.

This function is pretty simple and straight-forward. I don't think it should cause many problems.

[rwmjones]

Thanks, here are the changes to nbdkit incorporating this new API (optionally):

https://gitlab.com/nbdkit/nbdkit/-/commit/025db8b7fea237fd64e5231a1ebd03b30ca2f49a
https://gitlab.com/nbdkit/nbdkit/-/commit/81340d68643b5c18861f95c6f113764305d10c53