Restricting hosts where Python PreCommit runs due to environmental issues.

[pabloem]

No description provided.

[pabloem]

Host choice is due to having seen passing tests earlier today:
beam1 - https://builds.apache.org/job/beam_PreCommit_Python_GradleBuild/496/consoleFull
beam2 - https://builds.apache.org/job/beam_PreCommit_Python_GradleBuild/518/consoleFull
beam6 - https://builds.apache.org/job/beam_PreCommit_Python_GradleBuild/487/consoleFull

[pabloem]

Other latest Python PreCommits are running in beam3 and show failures in setupVirtualenv, which seems to indicate environment setup issues.
@aaltay do you think adding this restriction is reasonable?
beam3 - https://builds.apache.org/job/beam_PreCommit_Python_GradleBuild/534/console

[pabloem]

I'm also adding a check for existence of a directory before removal in gen_protos.

[aaltay]

Approving this to enable tests. I do not think I understand why nodes become bad or recover later. Is it possible that our tests are leaving things in an invalid state?

[aaltay]

I thought we decided to use
shutil.rmtree(build_path, ignore_errors=True instead of a conditional check (https://docs.python.org/2/library/shutil.html#shutil.rmtree)

[pabloem]

Ah I'm sorry. Good memory. We had not merged that fix, but I'll make the change now.

[pabloem]

I've checked with Jason. I asked why Python tests pass/fail on some machines:

The reasons for that are as many as the stars in the sky
Different package versions, transient network issues, permanent network issues, Jenkins instability, etc.
For Python, historically there were some machines with up-to-date pip and virtualenv and some without, and we restricted execution of Python tests to the ones with

It doesn't seem that old Maven-based precommit had any sort of restrictions on hosts, so I'm not sure what's going on in this case: https://github.com/apache/beam/blob/a25157bc6a9203362f6dd23acc8b7e1f30daad8e/.test-infra/jenkins/job_beam_PreCommit_Python_MavenInstall.groovy

Somehow it must be that the Gradle build and the Maven build had different assumptions?

[pabloem]

Run Seed Job

[pabloem]

Retest this please

[robertwb]

The maven-based precommit didn't use virtualenv (it modified and used the users install of Python). This should be more hermetic, assuming a recent virtualenv and pip.