Lots of failures in setting up the environment due to connection issues with PyPI

[whimboo]

I see lots of jobs which are failing during the creation of the environments. This might happen because of connection failures, or server side problems for PyPI:

11:48:33 Download error on https://pypi.python.org/simple/mozinstall/: '' -- Some packages may not be found!
11:48:33 Couldn't find index page for 'mozinstall' (maybe misspelled?)
11:48:33 Scanning index of all packages (this may take a while)
11:48:33 Reading https://pypi.python.org/simple/
11:48:38 No local packages or download links found for mozinstall>=1.12
11:48:38 error: Could not find suitable distribution for Requirement.parse('mozinstall>=1.12')

I wonder if we should better change our strategy and also make use of the internal pypi mirror from releng. That way we should have a trustworthy connection, even we would have to request uploads for every kind of package version we need. At least stability is more important, so I feel that would be the right choice. I really don't want to make use of pre-configured environments again, because that adds extra pain to the maintainer of the CI system.

@armenzg, I assume that you would agree to that proposal. Would you mind telling me how to force installing the packages from the internal mirror? Are there environment variables we can set on the slave node? I cannot add those options to the create_env.py script, because it also has to work for every contributor.

[armenzg]

pip install --no-index --find-links http://pypi.pub.build.mozilla.org/pub

If you get rid of --no-index it will also check on pypi.python.org

[whimboo]

That won't work as I mentioned above. But I checked pip's website and found that the following should be fine:

export PIP_FIND_LINKS="http://pypi.pub.build.mozilla.org/pub"
export NO_INDEX=1

I will check that tomorrow, also without NO_INDEX.

[whimboo]

PR 587 has been merged to master. I tested locally and all works fine. Will push to staging soon. If all is fine it will land on production early tomorrow at latest.

[whimboo]

I pushed this to staging and production and both systems are working fine now!