New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix] Pass full path to isdir rather than just basename. #6453

Merged
merged 2 commits into from Sep 5, 2018

Conversation

Projects
None yet
4 participants
@OniOni
Copy link
Contributor

OniOni commented Sep 4, 2018

(Waiting for tests to pass)

Problem

While debugging an issue with a build, I stumbled on this (unrelated) bug within pants. When we look for cached interpreters, we iterate through the files in self._cached_dir and then check if these are directories. However, it would seem like we're just passing the basename to isdir rather than the full path. Thus we're failing to find any of the cached interpreters if _cache_dir is different from the current working directory.

Solution

Pass in joined path to isdir.

Result

This should not affect end user.

@stuhood

This comment has been minimized.

Copy link
Member

stuhood commented Sep 4, 2018

@OniOni

This comment has been minimized.

Copy link
Contributor

OniOni commented Sep 4, 2018

For sure, I'll get to that asap.

@mateor

This comment has been minimized.

Copy link
Member

mateor commented Sep 4, 2018

@OniOni is a new hire at Foursquare, btw, and has a couple Pants-adjacent tasks in his queue. He has a good idea for a regression test, FWICT, so I will hold off review until that gets updated

@OniOni

This comment has been minimized.

Copy link
Contributor

OniOni commented Sep 5, 2018

Added some tests for this fix. For info, this is what test output is for master:

                     ==================== FAILURES ====================
                     __ TestInterpreterCache.test_setup_cached_warm ___
                     
                     self = <pants_test.backend.python.test_interpreter_cache.TestInterpreterCache testMethod=test_setup_cached_warm>
                     
                         def test_setup_cached_warm(self):
                           with self._setup_test(mock_setup_cached=False) as (cache, path):
                             safe_mkdir(os.path.join(path, 'python'))
                             cache._interpreter_from_path = mock.Mock(return_value=self._interpreter)
                             interpreters = list(cache._setup_cached(filters=[]))
                         
                     >       assert len(interpreters) == 1
                     E       AssertionError: assert 0 == 1
                     E        +  where 0 = len([])
                     
                     .pants.d/pyprep/sources/d887004e6be47c97a3124cfafff47cb3ad59b1c2/pants_test/backend/python/test_interpreter_cache.py:197: AssertionError
                      generated xml file: /Users/msabourin/code/pants/.pants.d/test/pytest/tests.python.pants_test.backend.python.interpreter_cache/junitxml/TEST-tests.python.pants_test.backend.python.interpreter_cache.xml 

Pants is not picking up our cached interpreter.

@stuhood stuhood requested review from benjyw , jsirois and CMLivingston and removed request for benjyw Sep 5, 2018

@jsirois

jsirois approved these changes Sep 5, 2018

Copy link
Member

jsirois left a comment

Thanks! I'll merge when this goes green.

@OniOni

This comment has been minimized.

Copy link
Contributor

OniOni commented Sep 5, 2018

Awesome! Also thanks to whomever reran the rust/osx shard, very much appreciated.

@jsirois jsirois merged commit f0f4b93 into pantsbuild:master Sep 5, 2018

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

jsirois added a commit to jsirois/pants that referenced this pull request Sep 7, 2018

[fix] Pass full path to isdir rather than just basename. (pantsbuild#…
…6453)

* [fix] Pass full path to isdir rather than just basename.

* [test] Add tests for fix.

@jsirois jsirois referenced this pull request Sep 7, 2018

Merged

Cherry-pick setup_py fixes #6472

jsirois added a commit that referenced this pull request Sep 10, 2018

Cherry-pick setup_py fixes (#6472)
* [fix] Pass full path to isdir rather than just basename. (#6453)

* [fix] Pass full path to isdir rather than just basename.

* [test] Add tests for fix.

* Tighten travis matrix and python activation. (#6440)

A few improvements and fixes to the travis setup:
+ Upgrade to to python 2.7.14 via `python: "2.7"`.
+ Use `name` uniformly to name shards and kill the `SHARD` env var which
  was only used to achieve the same end.
+ Use anchors to DRY up python version and clippy shard failure
  ignoring.
+ Ensure travis `pyenv` pythons are on `PATH`. This works around
  travis-ci/travis-ci#8315 and nets us both
  python 2.7 and python 3.6 on all `language: python` shards.

This last is needed to support #6428 which fixes #6415.

* Remove broken pyenv shims from the PATH. (#6469)

It turns out these would still get found in various contexts since the
interpreter cache tries to setup everything it finds on the PATH. Add
improved context in the wrapper script to show exactly which pythons
Pants will have access to.

* Fixup tests involving pexrc. (#6446)

Previously these tests weren't reliably reading a pexrc and had fragile
workarounds and paper-overs as a result. pexrc_util now reliably injects
a pexrc with a temporary HOME.

Also cleanup the affected tests - use @skipIf, respect 100 cols, ensure
path comparisons with judicious application of `os.realpath`.

* Detect ns packages using correct interpreter. (#6428)

Previously `__init__.py`s were parsed in the context of the ambient
interpreter executing the `SetupPy` task which could lead to parse
failures for new or removed language features used in those
`__init__.py`s. We now construct a PEX using the appropriate interpreter
and use it to run ns package detection.

Fixes #6415

* Fix setup.py rendering. (#6439)

We now render setup.py from both python 2 & 3 such that the result
is ingestible by both python 2 & 3.

Fixes #6401
Fixes #6392
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment