Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Difference between coverage results with source specifies full dir instead of module name #426

Closed
nedbat opened this issue Oct 8, 2015 · 14 comments
Labels
bug Something isn't working

Comments

@nedbat
Copy link
Owner

nedbat commented Oct 8, 2015

Originally reported by Alexander Todorov (Bitbucket: atodorov, GitHub: atodorov)


Anaconda, the Fedora Linux installer supports coverage collection via the API interface. If you boot with inst.debug then the main file /usr/sbin/anaconda will initialize coverage as one of the first things it does.

    cov = coverage.coverage(data_file="/mnt/sysimage/root/anaconda.coverage",
                            branch=True,
                            source=["/usr/sbin/anaconda", "pyanaconda"]
                            )
    cov.start()

The results are 105 files measured, which is short of 19 files which are also present in the pyanaconda module. OTOH if the source path is specified as the full path (e.g. /usr/lib64/python3.4/site-packages/pyanaconda) then all files inside are included in the coverage data. See:
rhinstaller/anaconda#397

I've tried to reproduce locally with the same directory structure (but not the same code) and was not able to. I will keep trying so it's easier to reproduce and easier to debug/fix.


@nedbat
Copy link
Owner Author

nedbat commented Dec 18, 2016

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)


I tried to reproduce the problem with a foo module containing a single a.py file

#!python

def main():
    print('Hello')

With the following we have 50% coverage and that is to be expected:

#!bash

$ cat ~/tmp/b.py 
from foo import a
import coverage
cov = coverage.coverage(source=["foo"])
cov.start()
a.main()
cov.stop()
cov.report()
$ PYTHONPATH=/tmp python ~/tmp/b.py 
Hello
Coverage.py warning: Module foo was previously imported, but not measured.
Name            Stmts   Miss  Cover
-----------------------------------
/tmp/foo/a.py       2      1    50%

Changing the file to import the module after starting coverage there is 100% coverage:

#!bash

$ cat ~/tmp/b.py 
import coverage
cov = coverage.coverage(source=["foo"])
cov.start()
from foo import a
a.main()
cov.stop()
cov.report()
$ PYTHONPATH=/tmp python ~/tmp/b.py 
Hello
Name                   Stmts   Miss  Cover
------------------------------------------
/tmp/foo/__init__.py       0      0   100%
/tmp/foo/a.py              2      0   100%
------------------------------------------
TOTAL                      2      0   100%

@atodorov am I missing something ? Note that this is with coverage master (i.e. > 4.2) and it could be that things evolved for the best since 4.0 ;-)

@nedbat
Copy link
Owner Author

nedbat commented Dec 20, 2016

Original comment by Alexander Todorov (Bitbucket: atodorov, GitHub: atodorov)


@dachary I don't think your comment is related to the originally reported issue. If you take a look at the PR I've linked above you will see that I import all modules I want to cover after coverage has been enabled. My problem was the way in which the source path was specified. However I don't have the time to investigate and see how things are with 4.x

@nedbat
Copy link
Owner Author

nedbat commented Dec 20, 2016

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)


@atodorov I apologize for the confusion. Your fix look right: the source argument is a list of directories. If you specify pyanaconda instead of the full path to the directory it will only work as expected if pyanaconda is a subdirectory of your current working directory. Am I making sense this time ?

@nedbat
Copy link
Owner Author

nedbat commented Dec 20, 2016

Original comment by Alexander Todorov (Bitbucket: atodorov, GitHub: atodorov)


Thanks for the update, now it makes sense and indeed I had a "pyanaconda" directory locally. I guess this is something else you'd like to document with a PR, isn't it ?

@nedbat
Copy link
Owner Author

nedbat commented Dec 20, 2016

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)


I'm not sure how to update the documentation. It states that the --source is a list of directories. Maybe there is a way to convey that more strongly ?

@nedbat
Copy link
Owner Author

nedbat commented Dec 22, 2016

Original comment by Alexander Todorov (Bitbucket: atodorov, GitHub: atodorov)


I guess I haven't read that :). Maybe just add the note from your previous comment about full path vs. relative directories if the later exist in the project ?

@nedbat
Copy link
Owner Author

nedbat commented Dec 22, 2016

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)


:-D I'm not a native english speaker, do you have a wording in mind to update the documentation ? I'd be happy to create a pull request with it.

@nedbat
Copy link
Owner Author

nedbat commented Dec 22, 2016

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)


@nedbat re-reading the documentation I see it reads directories or package names but I'm not sure in which case using a packaging name makes a difference ?

@nedbat
Copy link
Owner Author

nedbat commented Dec 29, 2016

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)


The --source is split into source and source_pkgs variables depending. It is then used to set source_match and source_pkgs_match to figure out if a source should be taken into account during the run.

After the run and before saving the data to .coverage, the --source is used again to find which files were never executed. But the find_python_files does not know how to iterate over the files of a package. It only knows about listing the files in a directory.

This difference between directories and packages in the --source argument should be documented. Or maybe this should be considered a bug and there should be no difference between files and modules.

@nedbat
Copy link
Owner Author

nedbat commented Jan 6, 2017

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)


@atodorov Here is a tentative fix so there is no difference between directories and modules https://bitbucket.org/ned/coveragepy/pull-requests/122/make-source-module-do-the-same-as-source/diff

The alternative would be to document the difference.

What do you think ?

@nedbat
Copy link
Owner Author

nedbat commented Jan 10, 2017

Do I understand this correctly? This bug is about a difference between source=dir and source=pkg, and the difference is that with source=dir, coverage will find completely unexecuted files, but with source=pkg, it will not.

Right?

@nedbat
Copy link
Owner Author

nedbat commented Jan 10, 2017

Original comment by Loic Dachary (Bitbucket: dachary, GitHub: dachary)


Right.

@nedbat
Copy link
Owner Author

nedbat commented Jan 14, 2017

make --source module do the same as --source directory #426

The --source argument can either be a module or a directory. The user
expects that it behaves the same in both cases. Make sure the module
is recursively explored so that files that are not run show in the
coverage report.

close #426

→ <<cset ed784d8f334e (bb)>>

@nedbat
Copy link
Owner Author

nedbat commented Jan 17, 2017

This was shipped in 4.3.2.

@nedbat nedbat closed this as completed Jan 17, 2017
@nedbat nedbat added major bug Something isn't working labels Jun 23, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant