Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latest 5.0 version raises CoverageException: file is not a database #886

Closed
ellbosch opened this issue Dec 16, 2019 · 14 comments
Closed

Latest 5.0 version raises CoverageException: file is not a database #886

ellbosch opened this issue Dec 16, 2019 · 14 comments
Labels
bug fixed
Milestone

Comments

@ellbosch
Copy link

@ellbosch ellbosch commented Dec 16, 2019

Our coverage report is failing as of this weekend. It's failing on Python 2 and 3.

Stack trace below. Thanks for the help!

Traceback (most recent call last):
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 1025, in execute
    return self.con.execute(sql, parameters)
sqlite3.DatabaseError: file is not a database

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/bin/pytest", line 8, in <module>
    sys.exit(main())
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 72, in main
    config = _prepareconfig(args, plugins)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 223, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/helpconfig.py", line 89, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 793, in pytest_cmdline_parse
    self.parse(args)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 999, in parse
    self._preparse(args, addopts=addopts)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 957, in _preparse
    early_config=self, args=args, parser=self._parser
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pytest_cov/plugin.py", line 118, in pytest_load_initial_conftests
    plugin = CovPlugin(early_config.known_args_namespace, early_config.pluginmanager)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pytest_cov/plugin.py", line 164, in __init__
    self.start(engine.Central)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pytest_cov/plugin.py", line 186, in start
    self.cov_controller.start()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pytest_cov/engine.py", line 187, in start
    self.cov.load()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/control.py", line 387, in load
    self._data.read()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 741, in read
    with self._connect():       # TODO: doesn't look right
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 296, in _connect
    self._open_db()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 264, in _open_db
    self._read_db()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 268, in _read_db
    with self._dbs[get_thread_id()] as db:
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 1008, in __enter__
    self._connect()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 996, in _connect
    self.execute("pragma journal_mode=off").close()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 1027, in execute
    raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
coverage.misc.CoverageException: Couldn't use data file '/Users/ellbosch/Repos/mssql-cli/.coverage': file is not a database
Command '['pytest', '-l', '--cov', 'mssqlcli', '--doctest-modules', '--junitxml=junit/test-6d62ae30-2031-11ea-80ec-3af9d39e6caa-results.xml', '--cov-report=xml', '--cov-report=html', '--cov-append', '-o', 'junit_suite_name=pytest-3.7.4', '-s', '-m', 'not unstable', 'tests/jsonrpc']' returned non-zero exit status 1.
ERROR: InvocationError for command /Users/ellbosch/Repos/mssql-cli/.tox/py37/bin/python build.py unit_test (exited with code 1)
@ellbosch ellbosch added the bug label Dec 16, 2019
@ellbosch
Copy link
Author

@ellbosch ellbosch commented Dec 16, 2019

Following up--I've isolated this issue to the latest 5.0 release.

@ellbosch ellbosch changed the title Coverage report breaks with CoverageException: file is not a database Latest 5.0 version raises CoverageException: file is not a database Dec 16, 2019
@nedbat
Copy link
Owner

@nedbat nedbat commented Dec 16, 2019

Can you provide me with instructions to reproduce the issue?

@nedbat nedbat added this to the 5.0.1 milestone Dec 16, 2019
@ellbosch
Copy link
Author

@ellbosch ellbosch commented Dec 16, 2019

I only seem to run into this error when attempting to run tox with coverage reports. You should be able to repro this with our repo here. tox.ini shows what's run on tox. The reference to coverage is in the unit_test method inside build.py.

Interestingly, it seems to work fine when calling the unit_test method without tox.

@bhearsum
Copy link

@bhearsum bhearsum commented Dec 16, 2019

I can also reproduce in https://github.com/mozilla-releng/tooltool/ by running:

git clone https://github.com/mozilla-releng/tooltool/
cd tooltool/api
tox -e py37
coverage xml

bhearsum added a commit to bhearsum/tooltool that referenced this issue Dec 16, 2019
@willienillie
Copy link

@willienillie willienillie commented Dec 16, 2019

I saw a similar issue when running pytest-cov and coverage through tox. It was resolved by pinning coverage to back to 4.5.4.

raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> coverage.misc.CoverageException: Couldn't use data file '/my/repo/location/.coverage': Safety level may not be changed inside a transaction

@nedbat
Copy link
Owner

@nedbat nedbat commented Dec 17, 2019

@mozbhearsum Thanks for the steps. When I try them, it doesn't fail, I thought because you have already pinned coverage==4.5.4 on your master. But when I move back a few commits (to ac7a85d), requirements/test.txt still has coverage pinned. I tried hand-editing the file to use coverage==5.0, but it needs hashes so it wouldn't install.

Do you mind giving me a specific hash to check out from your repo that will show the problem?

@nedbat nedbat added the question label Dec 17, 2019
@bhearsum
Copy link

@bhearsum bhearsum commented Dec 17, 2019

@mozbhearsum Thanks for the steps. When I try them, it doesn't fail, I thought because you have already pinned coverage==4.5.4 on your master. But when I move back a few commits (to ac7a85d), requirements/test.txt still has coverage pinned. I tried hand-editing the file to use coverage==5.0, but it needs hashes so it wouldn't install.

Do you mind giving me a specific hash to check out from your repo that will show the problem?

Yeah, that's right. ac7a85d073fae891aa9166253069f46bb8b30c3e should reproduce.

@nedbat
Copy link
Owner

@nedbat nedbat commented Dec 17, 2019

@Urth
Copy link

@Urth Urth commented Dec 17, 2019

On our project this was caused by mixing versions of coverage.py
I think it would help users if coverage.py 5 would check if the .coverage file format is from version 4 and report it as a error.

$ head -c63 .coverage 
!coverage.py: This is a private format, don't read it directly!
$ coverage xml
Couldn't use data file '/home/harm/tooltool/api/.coverage': file is not a database
$ coverage --version
Coverage.py, version 5.0 with C extension
Full documentation is at https://coverage.readthedocs.io

@cjolowicz
Copy link

@cjolowicz cjolowicz commented Dec 19, 2019

On our project this was caused by mixing versions of coverage.py

Same here. The test suite used a pinned version (4.5.4) of coverage.py to generate coverage data. The session generating an XML report for upload to Codecov used the latest version (5.0), resulting in said error message.

Aligning the versions fixes the issue:

cjolowicz/hypermodern-python@32674c2

@nedbat
Copy link
Owner

@nedbat nedbat commented Dec 22, 2019

I've made the error message specific in 0d6a878. Now it will say "Couldn't use data file '.coverage': Looks like a coverage 4.x data file. Are you mixing versions of coverage?"

If you are encountering this error, I strongly encourage you not to simply pin coverage==4.5.4, but instead to figure out where the mixed versions are coming from, and upgrade everything to 5.x.

@nedbat
Copy link
Owner

@nedbat nedbat commented Dec 22, 2019

This is now available in coverage==5.0.1

Please upgrade. If the problem persists, let's debug why. :)

@lzkelley
Copy link

@lzkelley lzkelley commented Mar 25, 2020

I'm getting this new error message after upgrading coverage from 4.5.4 to 5.0.4, when running unit tests on a package that previously used coverage 4. Is there a procedure for moving from 4 --> 5? Is there a way to have it update the .coverage file? Thanks!

@nedbat
Copy link
Owner

@nedbat nedbat commented Mar 25, 2020

@lzkelley can you provide me with a way to reproduce your problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug fixed
Projects
None yet
Development

No branches or pull requests

7 participants