0.11.* coverage plugin causes issues during build deleting sys modules #211
Comments
|
Thanks for the report. We're actually specifically attempting not to touch ANY modules that belong to Python by inspecting the installation. If you have a custom installation, we're probably introspecting it in a wrong way.
Generally, yes, and we don't, except for coverage, where not cleaning out non-essential modules produces wrong results. Could you please run |
|
This looks like the Another possibility (if you can spare the time) is to insert import pdb
pdb.set_trace()in the |
|
Thanks for the quick reply. Here is the log. |
|
I added traceback and apparently sys is also considered not essential. and I get |
|
Could you describe your custom installation? What are the customizations? |
|
@mriehl given that we're forking, maybe try to do something like this:
If there are other plugins that load test code into the |
|
I think the main difference is that our environment doesn't use any of the shared libraries under /usr/lib64/python2.7 but our lib-dynload is not under the same directory, instead it's at so every module in env/python/2.7.6/lib/python2.7/lib-dynload/ gets skipped as it's not located in env/lib/python2.7/ (the package_dir.startswith(std_lib) line) This is what is causing the deletion of the useful modules. |
|
@arcivanov I'm okay with that. Forking by default makes it harder to produce wrong coverage numbers and the pybuilder metrics would still be accurate. |
|
@MirkoRossini I wonder if I can get the information about your environment somehow. Python definitely knows to look for those directories so we should be able to look where Python does. @mriehl even if I get more info from Python I think we might need to follow the method proposed above. |
|
@arcivanov my previous comment contains the explanation on why the code doesn't consider the lib-dynload directory as being system. Do you need other details? Is there something I should attach? |
|
@MirkoRossini I understood your explanation, I was simply thinking outloud on whether there was some way for me to query Python for those directories that you use. Thanks for your help, no more data necessary. |
|
@MirkoRossini Actually I do have a favor to ask you. When I run the following in my environment here's what I get: What do you get in yours running the same? |
|
@arcivanov thanks for working on this: I made progress on my side though. I checked and lib-dynload is actually symlinked in the correct location. realpath removes the symlink, so the libraries don't appear to be in the correct location. sys.path correctly shows Is there a reason why we need to follow the symlinks and get the real path? Otherwise I can work on a pull request |
|
Ah, that's the thing, my |
|
Thanks a lot :) |
arcivanov
added a commit
to arcivanov/pybuilder
that referenced
this issue
Sep 25, 2015
|
@MirkoRossini could you test with the following?
|
|
Hi, I actually get this: |
|
@MirkoRossini sorry, that was me and my travis woes. Can you retry? I fixed the branch. |
|
Thanks, I can install now! Here is what I get: |
|
maybe an approach could be to create a set of symlinks and realpaths and consider all of them as valid paths? It would definitely fix my problem (sys path is an actual directory, sys-dynload is a symlink), I don't think it would affect other people. |
|
I just wanted to say, I have a similar problem, but it is isn't consistent. On both machines, I do the following: Machine 1 is using python 2.7.9 on Ubuntu 15.04 with pyb 0.11.1, and works fine not sure if this helps. If you need any other details from either machine let me know |
|
Ok, I'm sticking to the original plan - we can't handle custom environments 😭 |
arcivanov
added a commit
to arcivanov/pybuilder
that referenced
this issue
Sep 25, 2015
arcivanov
added a commit
to arcivanov/pybuilder
that referenced
this issue
Sep 25, 2015
arcivanov
added a commit
that referenced
this issue
Sep 25, 2015
coverage plugin causes issues during build deleting sys modules #211
|
No more resetting modules by default and never on Windows. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hello,
I think the new coverage plugin should be refactored, as it's a potential source of problems.
Looks like it's deleting a bunch of global modules, sparing only the ones that it consider "essential".
We have a custom python installation, the coverage plugin gets a bit confused and considers non essential things like datetime. We end up with errors like:
Traceback (most recent call last):
File "/tmp/test/env-infra/bin/pyb", line 25, in
sys.exit(pybuilder.cli.main(*sys.argv[1:]))
File "/tmp/test/env-infra/lib/python2.7/site-packages/pybuilder/cli.py", line 366, in main
end = datetime.datetime.now()
AttributeError: 'module' object has no attribute 'datetime'
Because of this issue we are unable to update to 0.11.1
I know there should be a way to fix on our side by reverse-engineering your code and put modules in strategic locations, but honestly nothing in pybuilder (nothing in anything IMHO) should play with sys.modules, as it will cause issues that are a nightmare to debug.
Downgrading to 0.10.63 fixes the issue obviously, but the install task would be nice to have for our purposes.
The text was updated successfully, but these errors were encountered: