Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow setting multi-request concurrency #266

Open
LeMoussel opened this issue Feb 20, 2017 · 2 comments
Open

Allow setting multi-request concurrency #266

LeMoussel opened this issue Feb 20, 2017 · 2 comments

Comments

@LeMoussel
Copy link

LeMoussel commented Feb 20, 2017

With request_multipleis it possible to send multiple requests simultaneously.
Is it possible to set the limit for how many requests will be execute simultaneously?

Because if it's too high, requests are likely to fail more frequently or automated software may perceive as a DOS attack and automatically block further requests.

For example I have an array with 1000 urls

$requests = array( 1000 * array( 'url' => 'http://urlofsitetotest.com/), );
// Setup a callback
function my_callback(&$request, $id) {
	var_dump($id, $request);
}
// Tell Requests to use the callback
$options = array('complete' => 'my_callback',)
// Set 5 requests will be execute simultaneously
// Requests::setSimultaneousLimit(5) => Does option exists ?
// Send the request!
$responses = Requests::request_multiple($requests, $options);
@rmccue
Copy link
Collaborator

rmccue commented Jun 9, 2017

This isn't currently supported, but would make sense to do so.

@rmccue rmccue changed the title How to to set the limit for how many requests will be execute simultaneously Allow setting multi-request concurrency Jun 9, 2017
@soulseekah
Copy link
Contributor

soulseekah commented Feb 17, 2018

For cURL we should be using curl_multi_select and adding in new handles as the others are freed. This should be fairly straightforward to implement, but hard to test. I'd like to give it a shot.

Any ideas, preferences on naming the option? concurrent, threads (not technically), limit?

For fsockopen stream_select would actually work, but we need to rewrite the request_multiple in a better way, split out the read and parse parts. Definitely possible but out of scope of this issue, since fsockopen request_multiple is just a series wrapper around the regular request. TBD in #317

Possibly related #53

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants