Set up pants.remote.ini for remoting Python unit tests #7990
Conversation
pants.ini
Outdated
| # TODO(#7735): We need to add this entry for remoting to be able to discover a valid | ||
| # interpreter, because <PATH> will refer to the host PATH and not the remote value. This value | ||
| # was found by inspecting the docker image for remoting. | ||
| "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/python3.6/bin:/usr/local/go/bin", |
There was a problem hiding this comment.
I did not comment this out because it should not cause failures, at least in most instances. If any of these entries do not exist on the machine, then Pex / Pants will just try the other entries in <PEXRC> and <PATH> (which is the host PATH).
By leaving it in, it's one less thing that users have to change to run remoting.
I see why we might want to comment it out by default, though, and am happy to do so if you think it's worth it.
There was a problem hiding this comment.
IMO, we should either resolve the platform issue, or (if this is really a good default) submit it as the default to pex.
There was a problem hiding this comment.
or (if this is really a good default) submit it as the default to pex.
What do you mean?
we should either resolve the platform issue
I'd prefer we don't let the platform issue block this PR, because we seem to still be at least a few days away from a good solution. Agreed that it will be great to remove this hack asap.
pants.ini
Outdated
| remote_execution_extra_platform_properties: [ | ||
| # This allows network requests, e.g. to resolve dependencies with Pex. | ||
| "dockerNetwork=standard", | ||
| "container-image=docker://marketplace.gcr.io/google/rbe-ubuntu16-04@sha256:da0f21c71abce3bbb92c3a0c44c3737f007a82b60f8bd2930abc55fe64fc2729", |
There was a problem hiding this comment.
Stu and I discussed instead using our custom Centos7 image. For now, we'll stick with this because it's Ubuntu 16 so closer to what we have in CI. It's possible that we will need to revisit this sometime soon.
There was a problem hiding this comment.
We should use whatever is closest to our travis environment probably... the Centos6/7 would primarily be relevant to wheel building, but our unit tests successfully run PEX resolves directly in the travis environment currently, and it would probably reduce complexity a bit to not change the platform we're running unit/integration tests on right now.
|
Blocked by #7991. |
|
Works for me, as long as @stuhood's concerns are addressed :) |
### Problem We want to allow users to permanently configure all remoting options in their `pants.ini`, without that config triggering Pants always using remoting. Tangibly, we are trying to land #7990 to setup our own usage and can't get green CI because Pants is trying to remote things it shouldn't. Instead, users should be able to turn on remoting at the per-invocation level, just like we have `--enable-pantsd`. ### Solution Introduce `--[no-]remote-execution`. For now, this defaults to `False` as this is an alpha feature and it does not work without other remoting config. ### Result Users can have all their remoting config setup and still be able to entirely run locally. Also now more explicit when you are and are not remoting.
Eric-Arellano
force-pushed
the
remoting-local-unit-tests
branch
from
38fac31
to
713f24c
Compare
|
This is now ready for review. |
Eric-Arellano
changed the title
Per #7649, we are working to remote unit tests in CI.
This implements step #2, which allows us to run unit tests locally (several, but not all tests). Here, we set up
pants.remote.iniso that users only must point to it and provide the authentication token.