Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement wget's --wait, --random-wait #5406

Closed
jidanni opened this issue May 17, 2020 · 4 comments
Closed

Implement wget's --wait, --random-wait #5406

jidanni opened this issue May 17, 2020 · 4 comments

Comments

@jidanni
Copy link

@jidanni jidanni commented May 17, 2020

curl could/should implement wget's

   --wait=seconds
       Wait the specified number of seconds between the
       retrievals... (I'm not talking about retries.)

   --random-wait
       Some web sites may perform log analysis to identify retriev...

Sure, curl has --limit-rate , but that's not the same.

@bagder
Copy link
Member

@bagder bagder commented May 17, 2020

Thanks, but this description sounds as if you're asking for a new feature/change. We use this tracker for bugs and issues only, we put ideas to work on in the future in the TODO document. We basically drown in good ideas so they don't do much use in our tracker.

If you really want to see this happen, start working on an implementation and submit a PR for it or join the mailing list and talk up more interest for it and see what help from others you can get!

@bagder
Copy link
Member

@bagder bagder commented May 18, 2020

Can you elaborate on your use case for these options?

@jidanni
Copy link
Author

@jidanni jidanni commented May 18, 2020

OK. Well for --random-wait my reason is the same as wget's (see their man page.)

And for --wait: a curl command might send a million requests. I want a
few seconds/minutes between each request, while testing, for me to make sure
nothing went nuts, on my end, the network, or the remote server.

Sure, I could make a shell script

for u in URL1 URL2 URL3...
do
        curl $u
        sleep 45
done

(But that might not always work the same as what we are hoping the single
curl run would do.)

@bagder
Copy link
Member

@bagder bagder commented May 19, 2020

Maybe a more interesting feature would be to implement support for the proposed RateLimit HTTP headers and make sure curl can adhere to them.

@bagder bagder closed this as completed in 39177b8 May 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants