Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

retry conda environment install on Travis #921

Merged
merged 6 commits into from Feb 14, 2019

Conversation

tomkinsc
Copy link
Member

@tomkinsc tomkinsc commented Feb 8, 2019

installation of conda packages on Travis often fails due to transient web request errors. This wraps the install command in a retry block to attempt the install up to 3 times.

tomkinsc and others added 3 commits February 8, 2019 15:10
installation of conda packages on Travis often fails due to transient web request errors. This wraps the install command in a retry block to attempt the install up to 3 times.
@@ -1,5 +1,5 @@
#!/bin/bash
set -e -o pipefail
#set -e -o pipefail
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe better to disable error-checking locally, for commands allowed to fail, rather than for the whole script

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like conda network commands are incredibly flaky lately, like 1/3 gateway errors?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, so @tomkinsc 's fix would be great. though ideally conda itself would have a 'retries' setting.

@notestaff
Copy link
Contributor

it seems conda has a built-in config setting for retries:

$ conda config --describe remote_max_retries

# remote_max_retries (int)

# The maximum number of retries each HTTP connection should attempt.

remote_max_retries: 3

conda config --set remote_max_retries 6
@tomkinsc
Copy link
Member Author

tomkinsc commented Feb 13, 2019

Thanks, @notestaff, nice find. It looks like conda does indeed retry 3 times by default, and we're still seeing network errors. I guess I'll try raising the retry count...
Looking at the rest of the config, we can also try turning the knobs remote_read_timeout_secs and remote_connect_timeout_secs if it seems like we're seeing timeout failures.

@tomkinsc
Copy link
Member Author

Or maybe (some of) the network-related error messages are actually SSL/cert -elated?

add -y to conda update so we always update ssl/certs without user interaction
@tomkinsc tomkinsc merged commit e922e7a into master Feb 14, 2019
@tomkinsc tomkinsc deleted the ct-travis-retry-conda-install branch February 14, 2019 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants