-
Notifications
You must be signed in to change notification settings - Fork 405
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Running tests under nox is flaxy #348
Comments
See https://pipxproject.github.io/pipx/contributing/
|
Sorry for the wall of noise, but I did want to show the whole command output; in both cases, I installed cs01/nox@5ea70723e9e6. Ex: |
It looks strange that you are installing nox from within nox. You can see command in the travis file. Unless something broke upstream, it should work since the travis tests work. https://github.com/pipxproject/pipx/blob/master/.travis.yml#L28. |
Another thing: it looks like you are having trouble with We haven't had a problem with it in a while though, so I had assumed that that problem is fixed. It may or may not be related to your test issues. #266 |
I'm using
I've removed that from the OP to avoid that concern. |
Yes, that does seem to be the issue. Maybe But here's the thing - I can build/install regex from source in my environment:
Or without pep517 (supplying the implicit setuptools dependency):
So the environment is able to build regex. It seems there's something about how nox or pipx is preparing the environment that it's not able to build regex. Perhaps some subprocess invocation is removing environment variables that allow extension modules to be compiled. |
Reading up on #266, that issue seemed only to be concerned with running with gcc on Linux, but this issue seems to reveal that the failure is more widespread, and that issues exist when any package needs extensions compiled from source, and the issue only affects macOS users because currently regex doesn't supply wheels for that platform. |
It looks like the way nox works, it removes all environment variables from the session when invoking commands ( To test my hypothesis, I created this nox environment:
Then ran nox on it:
As you can see, it didn't fail. Aaah - because |
Since the issue doesn't appear to be with how nox configures its environment, I started looking into the test suite, and I found the pipx_temp_env fixture, which is explicitly monkey-patching the environment in which packages are installed. I guess it's no big surprise if the test suite is removing all binaries from PATH that a compiler will fail to execute. Indeed, I'm able to replicate the failure mode by simply spamming path before attempting to install regex:
My guess is there's something about the compiler that requires other tools on the path to be present for the include directories to be resolved correctly. Is it reasonable for the pipx tests to clobber the path? |
We used to add /usr/bin to the PATH in the test environment, but stopped doing it when test isolation suffered and caused other problems |
In 5fa0243, I found that by not clobbering the path, the tests stop failing... but some new tests start failing because I have black installed elsewhere, so triggers warnings (and assertions) about that:
And an error about certificate verification failing (probably another clobbered environment setting for those certificates):
|
Gah, so pipx tests will fail if |
In master, the issue with regex failing to compile has gone away. In this comment, I explain why I think the issue went away. I'm pretty sure by running The CERTIFICATE_VERIFY_FAILED error persists. |
I can more simply elicit the error message by applying this patch:
And then running |
Okay, I've traced the CERTIFICATE_VERIFY_FAILED to my development environment having lost its certificates file. After installing certifi and symlinking the certificates, that issue goes away. |
I'm trying to run the tests. I created a virtualenv for installing nox and installed the prescribed version, but the tests fail:
The text was updated successfully, but these errors were encountered: