Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BEAM-6318] Increase the amount of pip download retries to 10 from a default 5 #7519

Merged
merged 1 commit into from Jan 16, 2019

Conversation

tvalentyn
Copy link
Contributor

In BEAM-6318 we suspect that Beam test suites sometimes become flaky during PyPi outages, and we are exploring adding retries as a mitigation. This PR increases the number of retries used by pip from 5(default) to 10. In pip, retries happen with a back-off. I verified on a VM without internet access that with 10 retries pip attempts to download a package throughout 4 minutes, while first 5 attempts happen within 5 seconds.

If pip retry mechanism proves not sufficient to mitigate BEAM-6318, we can issue multiple calls to pip, also with a back-off.


Post-Commit Tests Status (on master branch)

Lang SDK Apex Dataflow Flink Gearpump Samza Spark
Go Build Status --- --- --- --- --- ---
Java Build Status Build Status Build Status Build Status
Build Status
Build Status
Build Status Build Status Build Status
Python Build Status --- Build Status
Build Status
Build Status --- --- ---

@tvalentyn
Copy link
Contributor Author

R: @aaltay

@aaltay aaltay merged commit 1891410 into apache:master Jan 16, 2019
@tvalentyn tvalentyn deleted the pip_retries branch October 23, 2019 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants