Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5.0: Coverage report generation failure #716

Closed
tyacbovi opened this issue Oct 7, 2018 · 18 comments
Closed

5.0: Coverage report generation failure #716

tyacbovi opened this issue Oct 7, 2018 · 18 comments

Comments

@tyacbovi
Copy link

@tyacbovi tyacbovi commented Oct 7, 2018

After the current release of 5.0a3, the coverage generation is failing.

I am using the following packages for running my tests:
Python 2.7.15, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
plugins: xdist-1.23.2, forked-0.2, cov-2.6.0

The failure stack trace:

.......................................................INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR> gen.send(outcome)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR> self.cov_controller.finish()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/engine.py", line 297, in finish
INTERNALERROR> self.cov.save()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 527, in save
INTERNALERROR> data = self.get_data()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR> if self._collector and self._collector.flush_data():
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR> self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 231, in add_lines
INTERNALERROR> self._choose_lines_or_arcs(lines=True)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 277, in _choose_lines_or_arcs
INTERNALERROR> con.execute("update meta set has_lines = ?, has_arcs = ?", (lines, arcs))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR> raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> CoverageException: Couldn't use data file '/home/circleci/project/.coverage.905ab52c4304.2293.575964': no such table: meta
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR> gen.send(outcome)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR> self.cov_controller.finish()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/engine.py", line 297, in finish
INTERNALERROR> self.cov.save()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 527, in save
INTERNALERROR> data = self.get_data()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR> if self._collector and self._collector.flush_data():
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR> self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 231, in add_lines
INTERNALERROR> self._choose_lines_or_arcs(lines=True)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 277, in _choose_lines_or_arcs
INTERNALERROR> con.execute("update meta set has_lines = ?, has_arcs = ?", (lines, arcs))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR> raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> CoverageException: Couldn't use data file '/home/circleci/project/.coverage.905ab52c4304.2291.551230': no such table: meta
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR> gen.send(outcome)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR> self.cov_controller.finish()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/engine.py", line 297, in finish
INTERNALERROR> self.cov.save()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 527, in save
INTERNALERROR> data = self.get_data()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR> if self._collector and self._collector.flush_data():
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR> self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 231, in add_lines
INTERNALERROR> self._choose_lines_or_arcs(lines=True)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 277, in _choose_lines_or_arcs
INTERNALERROR> con.execute("update meta set has_lines = ?, has_arcs = ?", (lines, arcs))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR> raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> CoverageException: Couldn't use data file '/home/circleci/project/.coverage.905ab52c4304.2288.788971': no such table: meta
... [100%]INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR> gen.send(outcome)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR> self.cov_controller.finish()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/engine.py", line 249, in finish
INTERNALERROR> self.cov.stop()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 527, in save
INTERNALERROR> data = self.get_data()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR> if self._collector and self._collector.flush_data():
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR> self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 231, in add_lines
INTERNALERROR> self._choose_lines_or_arcs(lines=True)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 277, in _choose_lines_or_arcs
INTERNALERROR> con.execute("update meta set has_lines = ?, has_arcs = ?", (lines, arcs))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR> raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> CoverageException: Couldn't use data file '/home/circleci/project/.coverage': no such table: meta

@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Oct 7, 2018

Can you give me instructions for reproducing this problem?

@tyacbovi

This comment has been minimized.

Copy link
Author

@tyacbovi tyacbovi commented Oct 7, 2018

The relevant code is in this repo,
and this is the tox.ini.
Also a link to the circleci build

@tyacbovi

This comment has been minimized.

Copy link
Author

@tyacbovi tyacbovi commented Oct 7, 2018

Did not mention, but the pytest with coverage runs successfully for the 5.0a2 version.

@nedbat nedbat changed the title Coverage report generation failure 5.0: Coverage report generation failure Oct 7, 2018
@krisrogers

This comment has been minimized.

Copy link

@krisrogers krisrogers commented Oct 8, 2018

We are observing this error in one repo, and a similar one (shown below) in another repo. Both were working with 5.02a.

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 532, in execute
INTERNALERROR> return self.con.execute(sql, parameters)
INTERNALERROR> sqlite3.IntegrityError: UNIQUE constraint failed: file.path
INTERNALERROR>
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR> gen.send(outcome)
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR> self.cov_controller.finish()
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pytest_cov/engine.py", line 167, in finish
INTERNALERROR> self.cov.stop()
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/control.py", line 527, in save
INTERNALERROR> data = self.get_data()
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR> if self._collector and self._collector.flush_data():
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR> self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 236, in add_lines
INTERNALERROR> file_id = self._file_id(filename, add=True)
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 189, in _file_id
INTERNALERROR> cur = con.execute("insert into file (path) values (?)", (filename,))
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR> raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> coverage.misc.CoverageException: Couldn't use data file '/home/kris/dev/kapiche/chrysalis/.coverage': UNIQUE constraint failed: file.path

@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Oct 8, 2018

@krisrogers if you could provide reproduction instructions, it would be very helpful, thanks :)

@minrk

This comment has been minimized.

Copy link

@minrk minrk commented Oct 8, 2018

I'm hitting this in my test suites, and I think I've narrowed it down to tests run with pytest-cov which launch subprocesses. Take the self-contained module mymodule.py:

# mymodule.py
from subprocess import Popen
import sys


def foo():
    pass


def bar():
    pass


def test_foo():
    foo()


def test_subprocess():
    """this test triggers coveragepy#716"""
    p = Popen([sys.executable, __file__])
    p.communicate()


if __name__ == "__main__":
    bar()

test_foo has no trouble, but running test_subprocess produces this error:

(test-coverage) benjaminrk[13:53]/tmp $ pytest --cov mymodule -vk subprocess mymodule.py
======================================================================================================================================================================= test session starts =======================================================================================================================================================================
platform darwin -- Python 3.6.5, pytest-3.8.2, py-1.6.0, pluggy-0.7.1 -- /tmp/test-coverage/bin/python3
cachedir: .pytest_cache
rootdir: /private/tmp, inifile:
plugins: cov-2.6.0
collected 2 items / 1 deselected

mymodule.py::test_subprocess PASSED                                                                                                                                                                                                                                                                                                                         [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/sqldata.py", line 532, in execute
INTERNALERROR>     return self.con.execute(sql, parameters)
INTERNALERROR> sqlite3.IntegrityError: UNIQUE constraint failed: file.path
INTERNALERROR>
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pluggy/hooks.py", line 258, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pluggy/manager.py", line 61, in <lambda>
INTERNALERROR>     firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pytest_cov/engine.py", line 167, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/control.py", line 527, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR>     self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/sqldata.py", line 236, in add_lines
INTERNALERROR>     file_id = self._file_id(filename, add=True)
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/sqldata.py", line 189, in _file_id
INTERNALERROR>     cur = con.execute("insert into file (path) values (?)", (filename,))
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR>     raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> coverage.misc.CoverageException: Couldn't use data file '/private/tmp/.coverage': UNIQUE constraint failed: file.path

============================================================================================================================================================= 1 passed, 1 deselected in 0.15 seconds ==============================================================================================================================================================

pip freeze:

atomicwrites==1.2.1
attrs==18.2.0
coverage==5.0a3
more-itertools==4.3.0
pluggy==0.7.1
py==1.6.0
pytest==3.8.2
pytest-cov==2.6.0
six==1.11.0

I don't know enough to say if this is an issue with coverage.py or something that should be updated in pytest-cov, but it seems like there's a race to create the lines corresponding to files touched in both processes. I don't know how touching the coverage database is coordinated between processes, but there is definitely a race where rows created by other processes between the initialization of _file_map() in _open_db and the addition of new lines in _file_id(), where _file_id assumes that _file_map is up-to-date.

@minrk

This comment has been minimized.

Copy link

@minrk minrk commented Oct 8, 2018

#718 fixes this issue for me by using insert or ignore to resolve the race. I don't know if there are other issues that may be indicated by a stale cache of data from the database.

@krisrogers

This comment has been minimized.

Copy link

@krisrogers krisrogers commented Oct 10, 2018

@nedbat Sorry, can't provide reproduction details; I'm observing this in a commercial code base; Here is the tox file though, if it helps

tox.ini.txt

@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Oct 12, 2018

For the "UNIQUE constraint failed: file.path" problem: you need to use [run] parallel=True in a .coveragerc file to make things behave properly. Once you have that setting, the results are correct.

@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Oct 13, 2018

@tyacbovi I can't reproduce the problem you show. Since reporting this problem, you've pinned coverage to 4.5.1. I tried using commit 7c5117b5 of your repo, but it also installs 4.5.1. I manually pinned coverage to 5.0a3, and still did not see the problem. Do you only see the issue on CircleCI? I'm running on a Mac.

Can you give me specific instructions to get the right version of the code, and how to run the tests? I used "tox -e py27_script_plugin" based on the output you showed.

@nedbat nedbat added the question label Oct 13, 2018
@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Oct 13, 2018

@tyacbovi Oops, never mind. I did it again and got the error you showed. Not sure what happened there.... I'm looking into it.

@nedbat nedbat removed the question label Oct 13, 2018
nedbat added a commit that referenced this issue Oct 15, 2018
The collector calls set_context() before any code is run.  If we touch
the database there, it will get created *very* early.  This causes
problems with pytest-cov, which will delete those early-created files
when erasing data.

By deferring the database access until add_lines is called, the data
file stays off the disk until the collection is done (or until the
context switches), which avoids the problem.
@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Oct 15, 2018

The "no such table: meta" problem is fixed in ae9f040.

@tyacbovi

This comment has been minimized.

Copy link
Author

@tyacbovi tyacbovi commented Nov 6, 2018

@nedbat I wanted to ask when are you planning to release a new version with this bug fix?

cygnusv added a commit to cygnusv/pyUmbral that referenced this issue Nov 15, 2018
@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Nov 25, 2018

This was released as part of 5.0a4, today.

@MartinThoma

This comment has been minimized.

Copy link

@MartinThoma MartinThoma commented Jan 13, 2020

edit: The original message was wrong. I fixed it to be correct.

With the following I get the error:

coverage==5.0.3

A quick-fix for me was to change the requirement to:

coverage<5.0.0
@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Jan 13, 2020

@MartinThoma Can you give us a way to reproduce the error you are seeing?

@MartinThoma

This comment has been minimized.

Copy link

@MartinThoma MartinThoma commented Jan 13, 2020

@nedbat Sorry, the original message I wrote was wrong. The message I got is here https://travis-ci.org/MartinThoma/clana/jobs/636595136 :

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/sqldata.py", line 1046, in execute
INTERNALERROR>     return self.con.execute(sql, parameters)
INTERNALERROR> sqlite3.IntegrityError: UNIQUE constraint failed: meta.key
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 196, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 246, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pytest_cov/plugin.py", line 254, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pytest_cov/engine.py", line 197, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/control.py", line 642, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/control.py", line 696, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR>     self.covdata.add_lines(self.mapped_file_dict(self.data))
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/sqldata.py", line 437, in add_lines
INTERNALERROR>     self._choose_lines_or_arcs(lines=True)
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/sqldata.py", line 497, in _choose_lines_or_arcs
INTERNALERROR>     ('has_arcs', str(int(arcs)))
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/sqldata.py", line 1063, in execute
INTERNALERROR>     raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, msg))
INTERNALERROR> coverage.misc.CoverageException: Couldn't use data file '/home/travis/build/MartinThoma/clana/.coverage': UNIQUE constraint failed: meta.key

Forcing coverage<5.0.0 resulted in coverage==4.5.4. This worked:

@MartinThoma

This comment has been minimized.

Copy link

@MartinThoma MartinThoma commented Jan 14, 2020

@nedbat I've added the details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.