Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

optimized delay between downloads #65

Merged
merged 5 commits into from
Sep 23, 2021
Merged

Conversation

mrx23dot
Copy link
Contributor

Delay between 2 downloads still exists.
But only executes if needed.

eg.

  • download file1,
  • do something else for long time,
  • no delay for file2 since delay already elapsed.

Also no delay if only downloading 1file.

Very useful if downloading many fillings.

//find_entry_file might have slipped in, github ui is horrible, might need removing

@manusimidt
Copy link
Owner

@mrx23dot
Thank you for your contributions!
I am sorry that I am currently so slow with answering the issues and pull requests. Since I've been on vacation for two weeks and am currently moving to another city (and even country) for my Master's degree, I unfortunately have very little time at the moment. I will look at your pull requests within the next 14 days and implement/comment.

Greetings,
Manu

@manusimidt manusimidt merged commit 1e0f83a into manusimidt:main Sep 23, 2021
@manusimidt
Copy link
Owner

@mrx23dot The optimized delay works great! Will definetly save much time in comparison to the stupid delay that was implemented before.
Thanks! 😊

@mrx23dot
Copy link
Contributor Author

mrx23dot commented Sep 23, 2021

Good to hear, aws still gave me hard time with SEC, saying I'm exceeding rate, turns out it uses shared IP.
Also it helps a lot when parsing takes more time than the backoff time.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants