Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latest 5.0 version raises CoverageException: file is not a database #886

Closed
ellbosch opened this issue Dec 16, 2019 · 12 comments
Closed

Latest 5.0 version raises CoverageException: file is not a database #886

ellbosch opened this issue Dec 16, 2019 · 12 comments
Labels
Milestone

Comments

@ellbosch
Copy link

@ellbosch ellbosch commented Dec 16, 2019

Our coverage report is failing as of this weekend. It's failing on Python 2 and 3.

Stack trace below. Thanks for the help!

Traceback (most recent call last):
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 1025, in execute
    return self.con.execute(sql, parameters)
sqlite3.DatabaseError: file is not a database

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/bin/pytest", line 8, in <module>
    sys.exit(main())
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 72, in main
    config = _prepareconfig(args, plugins)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 223, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/helpconfig.py", line 89, in pytest_cmdline_parse
    config = outcome.get_result()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 793, in pytest_cmdline_parse
    self.parse(args)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 999, in parse
    self._preparse(args, addopts=addopts)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/_pytest/config/__init__.py", line 957, in _preparse
    early_config=self, args=args, parser=self._parser
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/hooks.py", line 286, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/manager.py", line 93, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/manager.py", line 87, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 80, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pytest_cov/plugin.py", line 118, in pytest_load_initial_conftests
    plugin = CovPlugin(early_config.known_args_namespace, early_config.pluginmanager)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pytest_cov/plugin.py", line 164, in __init__
    self.start(engine.Central)
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pytest_cov/plugin.py", line 186, in start
    self.cov_controller.start()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/pytest_cov/engine.py", line 187, in start
    self.cov.load()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/control.py", line 387, in load
    self._data.read()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 741, in read
    with self._connect():       # TODO: doesn't look right
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 296, in _connect
    self._open_db()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 264, in _open_db
    self._read_db()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 268, in _read_db
    with self._dbs[get_thread_id()] as db:
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 1008, in __enter__
    self._connect()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 996, in _connect
    self.execute("pragma journal_mode=off").close()
  File "/Users/ellbosch/Repos/mssql-cli/.tox/py37/lib/python3.7/site-packages/coverage/sqldata.py", line 1027, in execute
    raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
coverage.misc.CoverageException: Couldn't use data file '/Users/ellbosch/Repos/mssql-cli/.coverage': file is not a database
Command '['pytest', '-l', '--cov', 'mssqlcli', '--doctest-modules', '--junitxml=junit/test-6d62ae30-2031-11ea-80ec-3af9d39e6caa-results.xml', '--cov-report=xml', '--cov-report=html', '--cov-append', '-o', 'junit_suite_name=pytest-3.7.4', '-s', '-m', 'not unstable', 'tests/jsonrpc']' returned non-zero exit status 1.
ERROR: InvocationError for command /Users/ellbosch/Repos/mssql-cli/.tox/py37/bin/python build.py unit_test (exited with code 1)
@ellbosch ellbosch added the bug label Dec 16, 2019
@ellbosch

This comment has been minimized.

Copy link
Author

@ellbosch ellbosch commented Dec 16, 2019

Following up--I've isolated this issue to the latest 5.0 release.

@ellbosch ellbosch changed the title Coverage report breaks with CoverageException: file is not a database Latest 5.0 version raises CoverageException: file is not a database Dec 16, 2019
@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Dec 16, 2019

Can you provide me with instructions to reproduce the issue?

@nedbat nedbat added this to the 5.0.1 milestone Dec 16, 2019
@ellbosch

This comment has been minimized.

Copy link
Author

@ellbosch ellbosch commented Dec 16, 2019

I only seem to run into this error when attempting to run tox with coverage reports. You should be able to repro this with our repo here. tox.ini shows what's run on tox. The reference to coverage is in the unit_test method inside build.py.

Interestingly, it seems to work fine when calling the unit_test method without tox.

@mozbhearsum

This comment has been minimized.

Copy link

@mozbhearsum mozbhearsum commented Dec 16, 2019

I can also reproduce in https://github.com/mozilla-releng/tooltool/ by running:

git clone https://github.com/mozilla-releng/tooltool/
cd tooltool/api
tox -e py37
coverage xml
mozbhearsum added a commit to mozbhearsum/tooltool that referenced this issue Dec 16, 2019
@willienillie

This comment has been minimized.

Copy link

@willienillie willienillie commented Dec 16, 2019

I saw a similar issue when running pytest-cov and coverage through tox. It was resolved by pinning coverage to back to 4.5.4.

raise CoverageException("Couldn't use data file {!r}: {}".format(self.filename, exc))
INTERNALERROR> coverage.misc.CoverageException: Couldn't use data file '/my/repo/location/.coverage': Safety level may not be changed inside a transaction
@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Dec 17, 2019

@mozbhearsum Thanks for the steps. When I try them, it doesn't fail, I thought because you have already pinned coverage==4.5.4 on your master. But when I move back a few commits (to ac7a85d), requirements/test.txt still has coverage pinned. I tried hand-editing the file to use coverage==5.0, but it needs hashes so it wouldn't install.

Do you mind giving me a specific hash to check out from your repo that will show the problem?

@nedbat nedbat added the question label Dec 17, 2019
@mozbhearsum

This comment has been minimized.

Copy link

@mozbhearsum mozbhearsum commented Dec 17, 2019

@mozbhearsum Thanks for the steps. When I try them, it doesn't fail, I thought because you have already pinned coverage==4.5.4 on your master. But when I move back a few commits (to ac7a85d), requirements/test.txt still has coverage pinned. I tried hand-editing the file to use coverage==5.0, but it needs hashes so it wouldn't install.

Do you mind giving me a specific hash to check out from your repo that will show the problem?

Yeah, that's right. ac7a85d073fae891aa9166253069f46bb8b30c3e should reproduce.

@nedbat

This comment has been minimized.

@Urth

This comment has been minimized.

Copy link

@Urth Urth commented Dec 17, 2019

On our project this was caused by mixing versions of coverage.py
I think it would help users if coverage.py 5 would check if the .coverage file format is from version 4 and report it as a error.

$ head -c63 .coverage 
!coverage.py: This is a private format, don't read it directly!
$ coverage xml
Couldn't use data file '/home/harm/tooltool/api/.coverage': file is not a database
$ coverage --version
Coverage.py, version 5.0 with C extension
Full documentation is at https://coverage.readthedocs.io
@cjolowicz

This comment has been minimized.

Copy link

@cjolowicz cjolowicz commented Dec 19, 2019

On our project this was caused by mixing versions of coverage.py

Same here. The test suite used a pinned version (4.5.4) of coverage.py to generate coverage data. The session generating an XML report for upload to Codecov used the latest version (5.0), resulting in said error message.

Aligning the versions fixes the issue:

cjolowicz/hypermodern-python@32674c2

nedbat added a commit that referenced this issue Dec 22, 2019
nedbat added a commit that referenced this issue Dec 22, 2019
nedbat added a commit that referenced this issue Dec 22, 2019
nedbat added a commit that referenced this issue Dec 22, 2019
@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Dec 22, 2019

I've made the error message specific in 0d6a878. Now it will say "Couldn't use data file '.coverage': Looks like a coverage 4.x data file. Are you mixing versions of coverage?"

If you are encountering this error, I strongly encourage you not to simply pin coverage==4.5.4, but instead to figure out where the mixed versions are coming from, and upgrade everything to 5.x.

@nedbat

This comment has been minimized.

Copy link
Owner

@nedbat nedbat commented Dec 22, 2019

This is now available in coverage==5.0.1

Please upgrade. If the problem persists, let's debug why. :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.