5.0: Coverage report generation failure #716

tyacbovi · 2018-10-07T14:20:18Z

After the current release of 5.0a3, the coverage generation is failing.

I am using the following packages for running my tests:
Python 2.7.15, pytest-3.8.2, py-1.6.0, pluggy-0.7.1
plugins: xdist-1.23.2, forked-0.2, cov-2.6.0

The failure stack trace:

.......................................................INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR> gen.send(outcome)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR> self.cov_controller.finish()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/engine.py", line 297, in finish
INTERNALERROR> self.cov.save()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 527, in save
INTERNALERROR> data = self.get_data()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR> if self._collector and self._collector.flush_data():
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR> self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 231, in add_lines
INTERNALERROR> self._choose_lines_or_arcs(lines=True)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 277, in _choose_lines_or_arcs
INTERNALERROR> con.execute("update meta set has_lines = ?, has_arcs = ?", (lines, arcs))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR> raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> CoverageException: Couldn't use data file '/home/circleci/project/.coverage.905ab52c4304.2293.575964': no such table: meta
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR> gen.send(outcome)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR> self.cov_controller.finish()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/engine.py", line 297, in finish
INTERNALERROR> self.cov.save()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 527, in save
INTERNALERROR> data = self.get_data()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR> if self._collector and self._collector.flush_data():
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR> self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 231, in add_lines
INTERNALERROR> self._choose_lines_or_arcs(lines=True)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 277, in _choose_lines_or_arcs
INTERNALERROR> con.execute("update meta set has_lines = ?, has_arcs = ?", (lines, arcs))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR> raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> CoverageException: Couldn't use data file '/home/circleci/project/.coverage.905ab52c4304.2291.551230': no such table: meta
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR> gen.send(outcome)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR> self.cov_controller.finish()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/engine.py", line 297, in finish
INTERNALERROR> self.cov.save()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 527, in save
INTERNALERROR> data = self.get_data()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR> if self._collector and self._collector.flush_data():
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR> self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 231, in add_lines
INTERNALERROR> self._choose_lines_or_arcs(lines=True)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 277, in _choose_lines_or_arcs
INTERNALERROR> con.execute("update meta set has_lines = ?, has_arcs = ?", (lines, arcs))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR> raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> CoverageException: Couldn't use data file '/home/circleci/project/.coverage.905ab52c4304.2288.788971': no such table: meta
... [100%]INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR> gen.send(outcome)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR> self.cov_controller.finish()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/pytest_cov/engine.py", line 249, in finish
INTERNALERROR> self.cov.stop()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 527, in save
INTERNALERROR> data = self.get_data()
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR> if self._collector and self._collector.flush_data():
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR> self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 231, in add_lines
INTERNALERROR> self._choose_lines_or_arcs(lines=True)
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 277, in _choose_lines_or_arcs
INTERNALERROR> con.execute("update meta set has_lines = ?, has_arcs = ?", (lines, arcs))
INTERNALERROR> File "/home/circleci/project/.tox/py27_script_plugin/lib/python2.7/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR> raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> CoverageException: Couldn't use data file '/home/circleci/project/.coverage': no such table: meta

nedbat · 2018-10-07T14:23:38Z

Can you give me instructions for reproducing this problem?

tyacbovi · 2018-10-07T14:28:49Z

The relevant code is in this repo,
and this is the tox.ini.
Also a link to the circleci build

tyacbovi · 2018-10-07T14:41:36Z

Did not mention, but the pytest with coverage runs successfully for the 5.0a2 version.

krisrogers · 2018-10-08T01:26:49Z

We are observing this error in one repo, and a similar one (shown below) in another repo. Both were working with 5.02a.

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 532, in execute
INTERNALERROR> return self.con.execute(sql, parameters)
INTERNALERROR> sqlite3.IntegrityError: UNIQUE constraint failed: file.path
INTERNALERROR>
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR> session.exitstatus = doit(config, session) or 0
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR> config.hook.pytest_runtestloop(session=session)
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pluggy/hooks.py", line 258, in call
INTERNALERROR> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pluggy/manager.py", line 61, in
INTERNALERROR> firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR> gen.send(outcome)
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR> self.cov_controller.finish()
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/pytest_cov/engine.py", line 167, in finish
INTERNALERROR> self.cov.stop()
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/control.py", line 527, in save
INTERNALERROR> data = self.get_data()
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR> if self._collector and self._collector.flush_data():
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR> self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 236, in add_lines
INTERNALERROR> file_id = self._file_id(filename, add=True)
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 189, in _file_id
INTERNALERROR> cur = con.execute("insert into file (path) values (?)", (filename,))
INTERNALERROR> File "/home/kris/dev/kapiche/chrysalis/.tox/py35/lib/python3.5/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR> raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> coverage.misc.CoverageException: Couldn't use data file '/home/kris/dev/kapiche/chrysalis/.coverage': UNIQUE constraint failed: file.path

nedbat · 2018-10-08T02:13:30Z

@krisrogers if you could provide reproduction instructions, it would be very helpful, thanks :)

minrk · 2018-10-08T12:27:44Z

I'm hitting this in my test suites, and I think I've narrowed it down to tests run with pytest-cov which launch subprocesses. Take the self-contained module mymodule.py:

# mymodule.py
from subprocess import Popen
import sys


def foo():
    pass


def bar():
    pass


def test_foo():
    foo()


def test_subprocess():
    """this test triggers coveragepy#716"""
    p = Popen([sys.executable, __file__])
    p.communicate()


if __name__ == "__main__":
    bar()

test_foo has no trouble, but running test_subprocess produces this error:

(test-coverage) benjaminrk[13:53]/tmp $ pytest --cov mymodule -vk subprocess mymodule.py
======================================================================================================================================================================= test session starts =======================================================================================================================================================================
platform darwin -- Python 3.6.5, pytest-3.8.2, py-1.6.0, pluggy-0.7.1 -- /tmp/test-coverage/bin/python3
cachedir: .pytest_cache
rootdir: /private/tmp, inifile:
plugins: cov-2.6.0
collected 2 items / 1 deselected

mymodule.py::test_subprocess PASSED                                                                                                                                                                                                                                                                                                                         [100%]
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/sqldata.py", line 532, in execute
INTERNALERROR>     return self.con.execute(sql, parameters)
INTERNALERROR> sqlite3.IntegrityError: UNIQUE constraint failed: file.path
INTERNALERROR>
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR>
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/_pytest/main.py", line 178, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/_pytest/main.py", line 215, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pluggy/hooks.py", line 258, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pluggy/manager.py", line 67, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pluggy/manager.py", line 61, in <lambda>
INTERNALERROR>     firstresult=hook.spec_opts.get('firstresult'),
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pluggy/callers.py", line 196, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pytest_cov/plugin.py", line 228, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/pytest_cov/engine.py", line 167, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/control.py", line 527, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/control.py", line 581, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR>     self.covdata.add_lines(abs_file_dict(self.data))
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/sqldata.py", line 236, in add_lines
INTERNALERROR>     file_id = self._file_id(filename, add=True)
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/sqldata.py", line 189, in _file_id
INTERNALERROR>     cur = con.execute("insert into file (path) values (?)", (filename,))
INTERNALERROR>   File "/tmp/test-coverage/lib/python3.6/site-packages/coverage/sqldata.py", line 534, in execute
INTERNALERROR>     raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> coverage.misc.CoverageException: Couldn't use data file '/private/tmp/.coverage': UNIQUE constraint failed: file.path

============================================================================================================================================================= 1 passed, 1 deselected in 0.15 seconds ==============================================================================================================================================================

pip freeze:

atomicwrites==1.2.1
attrs==18.2.0
coverage==5.0a3
more-itertools==4.3.0
pluggy==0.7.1
py==1.6.0
pytest==3.8.2
pytest-cov==2.6.0
six==1.11.0

I don't know enough to say if this is an issue with coverage.py or something that should be updated in pytest-cov, but it seems like there's a race to create the lines corresponding to files touched in both processes. I don't know how touching the coverage database is coordinated between processes, but there is definitely a race where rows created by other processes between the initialization of _file_map() in _open_db and the addition of new lines in _file_id(), where _file_id assumes that _file_map is up-to-date.

minrk · 2018-10-08T12:42:48Z

#718 fixes this issue for me by using insert or ignore to resolve the race. I don't know if there are other issues that may be indicated by a stale cache of data from the database.

krisrogers · 2018-10-10T00:50:57Z

@nedbat Sorry, can't provide reproduction details; I'm observing this in a commercial code base; Here is the tox file though, if it helps

tox.ini.txt

nedbat · 2018-10-12T23:48:16Z

For the "UNIQUE constraint failed: file.path" problem: you need to use [run] parallel=True in a .coveragerc file to make things behave properly. Once you have that setting, the results are correct.

nedbat · 2018-10-13T00:09:05Z

@tyacbovi I can't reproduce the problem you show. Since reporting this problem, you've pinned coverage to 4.5.1. I tried using commit 7c5117b5 of your repo, but it also installs 4.5.1. I manually pinned coverage to 5.0a3, and still did not see the problem. Do you only see the issue on CircleCI? I'm running on a Mac.

Can you give me specific instructions to get the right version of the code, and how to run the tests? I used "tox -e py27_script_plugin" based on the output you showed.

nedbat · 2018-10-13T00:20:58Z

@tyacbovi Oops, never mind. I did it again and got the error you showed. Not sure what happened there.... I'm looking into it.

The collector calls set_context() before any code is run. If we touch the database there, it will get created *very* early. This causes problems with pytest-cov, which will delete those early-created files when erasing data. By deferring the database access until add_lines is called, the data file stays off the disk until the collection is done (or until the context switches), which avoids the problem.

nedbat · 2018-10-15T01:02:27Z

The "no such table: meta" problem is fixed in ae9f040.

nedbat/coveragepy#716

tyacbovi · 2018-11-06T08:53:26Z

@nedbat I wanted to ask when are you planning to release a new version with this bug fix?

See nedbat/coveragepy#716

nedbat · 2018-11-25T22:22:53Z

This was released as part of 5.0a4, today.

MartinThoma · 2020-01-13T09:53:07Z

edit: The original message was wrong. I fixed it to be correct.

With the following I get the error:

coverage==5.0.3

A quick-fix for me was to change the requirement to:

coverage<5.0.0

nedbat · 2020-01-13T12:53:19Z

@MartinThoma Can you give us a way to reproduce the error you are seeing?

MartinThoma · 2020-01-13T20:22:10Z

@nedbat Sorry, the original message I wrote was wrong. The message I got is here https://travis-ci.org/MartinThoma/clana/jobs/636595136 :

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/sqldata.py", line 1046, in execute
INTERNALERROR>     return self.con.execute(sql, parameters)
INTERNALERROR> sqlite3.IntegrityError: UNIQUE constraint failed: meta.key
INTERNALERROR> 
INTERNALERROR> During handling of the above exception, another exception occurred:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 196, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/_pytest/main.py", line 246, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pluggy/hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pluggy/manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pluggy/callers.py", line 203, in _multicall
INTERNALERROR>     gen.send(outcome)
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pytest_cov/plugin.py", line 254, in pytest_runtestloop
INTERNALERROR>     self.cov_controller.finish()
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/pytest_cov/engine.py", line 197, in finish
INTERNALERROR>     self.cov.stop()
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/control.py", line 642, in save
INTERNALERROR>     data = self.get_data()
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/control.py", line 696, in get_data
INTERNALERROR>     if self._collector and self._collector.flush_data():
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/collector.py", line 425, in flush_data
INTERNALERROR>     self.covdata.add_lines(self.mapped_file_dict(self.data))
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/sqldata.py", line 437, in add_lines
INTERNALERROR>     self._choose_lines_or_arcs(lines=True)
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/sqldata.py", line 497, in _choose_lines_or_arcs
INTERNALERROR>     ('has_arcs', str(int(arcs)))
INTERNALERROR>   File "/home/travis/build/MartinThoma/clana/.tox/py36/lib/python3.6/site-packages/coverage/sqldata.py", line 1063, in execute
INTERNALERROR>     raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, msg))
INTERNALERROR> coverage.misc.CoverageException: Couldn't use data file '/home/travis/build/MartinThoma/clana/.coverage': UNIQUE constraint failed: meta.key

Forcing coverage<5.0.0 resulted in coverage==4.5.4. This worked:

MartinThoma · 2020-01-14T22:38:36Z

@nedbat I've added the details.

nuwang · 2020-02-09T14:40:23Z

@nedbat This has come up again with the latest coverage (5.0.3):
https://travis-ci.org/CloudVE/cloudbridge/jobs/648014930

CoverageException: Couldn't use data file '/home/travis/build/CloudVE/cloudbridge/.coverage': UNIQUE constraint failed: meta.key

We are using nosetests in parallel mode with coverage.

The relevant setup.cfg:
https://github.com/CloudVE/cloudbridge/blob/master/setup.cfg

and tox.ini:
https://github.com/CloudVE/cloudbridge/blob/master/tox.ini

Downgrading to <5.0.0 seems to fix the problem.

MartinThoma · 2020-03-02T07:14:54Z

I missed it before, but @nedbat already mentioned a fix: Add the following lines to your setup.cfg:

[coverage:run]
parallel=true

No need to use an old coverage version

edit: I see UNIQUE constraint failed: meta.key again with coverage-5.1-cp38-cp38-manylinux1_x86_64.whl while having the parallel=true hack.

Again, I would recommend to set coverage<5.0.0

coverage-5.0 expects to set "parallel = True" on config file. refs: nedbat/coveragepy#716 (comment)

MartinThoma · 2020-05-18T07:08:33Z

I think this issue should be opened again.

nedbat changed the title ~~Coverage report generation failure~~ 5.0: Coverage report generation failure Oct 7, 2018

tyacbovi mentioned this issue Oct 8, 2018

coverage: setting to a specific version cloudify-cosmo/cloudify-common#105

Merged

minrk mentioned this issue Oct 8, 2018

pin coverage due to bug in coveragepy 5 prerelease jupyterhub/jupyterhub#2226

Merged

minrk mentioned this issue Oct 8, 2018

avoid creating duplicate file-id rows #718

Closed

nedbat added the 5.0-pre-release label Oct 9, 2018

effigies mentioned this issue Oct 10, 2018

Travis failure nipy/nipype#2727

Closed

nedbat added the question Further information is requested label Oct 13, 2018

nedbat removed the question Further information is requested label Oct 13, 2018

nedbat closed this as completed Oct 15, 2018

nedbat added a commit that referenced this issue Oct 17, 2018

A test for 'no such table: meta' as in #716.

745d8f2

cygnusv added a commit to cygnusv/pyUmbral that referenced this issue Oct 19, 2018

Use option parallel=True for coverage

32bd6ea

nedbat/coveragepy#716

cygnusv mentioned this issue Oct 19, 2018

Random oracles module and code reorganization nucypher/pyUmbral#219

Merged

This was referenced Oct 22, 2018

Coverage is not measured with Python 3.7.0 pytest-dev/pytest-cov#215

Closed

SQLite Crash on Coverage 5.0a2 inside Hypothesis #702

Closed

QuLogic mentioned this issue Oct 23, 2018

Use sudo:true for nightly builds. matplotlib/matplotlib#12596

Closed

cygnusv added a commit to cygnusv/pyUmbral that referenced this issue Nov 15, 2018

Use option parallel=True for coverage (Fixes nucypher#222)

f5bebc6

See nedbat/coveragepy#716

vorpal-buildbot mentioned this issue Dec 15, 2019

Update coverage to 5.0 PennyDreadfulMTG/Penny-Dreadful-Tools#6937

Merged

jbylund mentioned this issue Jan 22, 2020

Add whitelist for string.Formatter jendrikseipp/vulture#184

Merged

6 tasks

jendrikseipp added a commit to jendrikseipp/vulture that referenced this issue Jan 22, 2020

Use coverage < 5.0 until nedbat/coveragepy#716 is fixed.

805e82b

tk0miya mentioned this issue Jan 23, 2020

test: Unpin coverage sphinx-doc/sphinx#7050

Closed

miurahr mentioned this issue Feb 14, 2020

Replace concurrent with multiprocessing module miurahr/py7zr#71

Merged

tk0miya added a commit to sphinx-doc/sphinx that referenced this issue Mar 5, 2020

Unpin coverage package

3e7b18e

coverage-5.0 expects to set "parallel = True" on config file. refs: nedbat/coveragepy#716 (comment)

tk0miya mentioned this issue Mar 5, 2020

Unpin coverage package sphinx-doc/sphinx#7253

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

5.0: Coverage report generation failure #716

5.0: Coverage report generation failure #716

tyacbovi commented Oct 7, 2018

nedbat commented Oct 7, 2018

tyacbovi commented Oct 7, 2018

tyacbovi commented Oct 7, 2018 •

edited

Loading

krisrogers commented Oct 8, 2018 •

edited

Loading

nedbat commented Oct 8, 2018

minrk commented Oct 8, 2018

minrk commented Oct 8, 2018

krisrogers commented Oct 10, 2018

nedbat commented Oct 12, 2018

nedbat commented Oct 13, 2018

nedbat commented Oct 13, 2018

nedbat commented Oct 15, 2018

tyacbovi commented Nov 6, 2018

nedbat commented Nov 25, 2018

MartinThoma commented Jan 13, 2020 •

edited

Loading

nedbat commented Jan 13, 2020

MartinThoma commented Jan 13, 2020 •

edited

Loading

MartinThoma commented Jan 14, 2020

nuwang commented Feb 9, 2020

MartinThoma commented Mar 2, 2020 •

edited

Loading

MartinThoma commented May 18, 2020

5.0: Coverage report generation failure #716

5.0: Coverage report generation failure #716

Comments

tyacbovi commented Oct 7, 2018

nedbat commented Oct 7, 2018

tyacbovi commented Oct 7, 2018

tyacbovi commented Oct 7, 2018 • edited Loading

krisrogers commented Oct 8, 2018 • edited Loading

nedbat commented Oct 8, 2018

minrk commented Oct 8, 2018

minrk commented Oct 8, 2018

krisrogers commented Oct 10, 2018

nedbat commented Oct 12, 2018

nedbat commented Oct 13, 2018

nedbat commented Oct 13, 2018

nedbat commented Oct 15, 2018

tyacbovi commented Nov 6, 2018

nedbat commented Nov 25, 2018

MartinThoma commented Jan 13, 2020 • edited Loading

nedbat commented Jan 13, 2020

MartinThoma commented Jan 13, 2020 • edited Loading

MartinThoma commented Jan 14, 2020

nuwang commented Feb 9, 2020

MartinThoma commented Mar 2, 2020 • edited Loading

MartinThoma commented May 18, 2020

tyacbovi commented Oct 7, 2018 •

edited

Loading

krisrogers commented Oct 8, 2018 •

edited

Loading

MartinThoma commented Jan 13, 2020 •

edited

Loading

MartinThoma commented Jan 13, 2020 •

edited

Loading

MartinThoma commented Mar 2, 2020 •

edited

Loading