Tests cannot be run from the source tree

[basak]

Thank you for pytest-trio! It looks like exactly what I'm looking for - in particular async fixtures. Unfortunately I haven't been able to get it to work at all in what I think should be the simplest and most obvious configuration.

On Debian or Ubuntu, I cannot find any configuration that allows the test to pass, following CHEATSHEET.rst as documented. My steps are as follows, having first installed only python3, virtualenv and git in a fresh container:

virtualenv -ppython3 venv
. venv/bin/activate
git clone git://github.com/python-trio/pytest-trio
cd pytest-trio
pip install -r test-requirements.txt
pytest pytest_trio

This works neither on Debian buster nor on Debian sid. I have also tried various other combinations of things, including Ubuntu, but cannot find any combination that works.

I found issue #66 so I tried using some previous versions of pytest, but nothing works. Although the error changes, I can't seem to find any version of pytest to use in which the test suite doesn't indicate that the pytest hook works at all.

The most helpful error message I can find is against pytest 4.4.2, which seems to be full of things like:

E       fixture 'autojump_clock' not found
>       available fixtures: LineMatcher, _config_for_test, _pytest, _sys_snapsho
t, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, cov, doctest_namespa
ce, linecomp, monkeypatch, no_cover, pytestconfig, record_property, record_xml_a
ttribute, recwarn, testdir, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
>       use 'pytest --fixtures [testpath]' for help on them.

Presumably this is something to do with pytest plugin integration?

The full test failure output against pytest 4.4.2 is attached.
test-failure.log

If you're doing something different that makes your tests pass, what is it please, and could you update CHEATSHEET.rst to match?

[njsmith]

Huh, that's weird. I get the same thing. I've definitely been able to run the tests locally in the past. It might have been something like PYTHONPATH=. pytest pytest_trio, but that doesn't work now either. I'm not sure what changed. Maybe something related to #76, or maybe new pytest changed something?

Our CI is still passing AFAIK. I'm pretty sure that what it's doing differently is actually installing pytest-trio before trying to test it. We do this to get a more realistic test (it makes sure that e.g. our setup.py isn't leaving out anything important), but it also means that we're generating pytest's plugin metadata. Having to install is annoying for local dev, so it would be nice to fix this, but I don't think this actually affects any users of pytest-trio. In particular, if you just want to use async fixtures in your own project, then I don't think you need to worry about this.

[njsmith]

Hmm, it looks like a workaround is to do:

virtualenv -ppython3 venv
. venv/bin/activate
git clone git://github.com/python-trio/pytest-trio
cd pytest-trio
pip install -r test-requirements.txt
pip install -e .
pytest --pyargs pytest_trio

I'm not a big fan of editable installs in general, because they involve creating static metadata that inevitably gets out of date with the actual source tree, but in this particular case I guess the risks aren't too bad and it does work.

[basak]

That works for me too. Thanks!

[Alessandro-Barbieri]

I would like to have this fixed

[njsmith]

@Alessandro-Barbieri What trouble is it causing for you? Are you trying to make changes to pytest-trio?

[Alessandro-Barbieri]

@njsmith I'm trying to package this for Gentoo. The pip workaround is unacceptable, I need another solution. The error I get is this https://gist.github.com/Alessandro-Barbieri/bb9b5fffa660637aac9c4e3a1fd3ff60

[basak]

FWIW, I packaged this for myself locally, and overrode the tests to not test at all to work around this for now.

[basak]

I packaged this for myself locally

(for Debian/Ubuntu, not Gentoo)

[njsmith]

@Alessandro-Barbieri If you know how to fix it then we would love to merge a patch, but I don't know how to fix it. Can you somehow install the package first before running the tests?

[Alessandro-Barbieri]

@njsmith yes, I already installed the package

[njsmith]

@Alessandro-Barbieri If you have a python environment with pytest-trio installed, then you should be able to run the pytest-trio tests using pytest --pyargs pytest_trio.

[Alessandro-Barbieri]

with pytest-5.4.2 I get

_______________________________________________________________________________________________________ ERROR collecting test session _______________________________________________________________________________________________________
Defining 'pytest_plugins' in a non-top-level conftest is no longer supported:
It affects the entire test suite instead of just below the conftest as expected.
  /var/tmp/portage/dev-python/pytest-trio-0.6.0/work/pytest-trio-0.6.0/pytest_trio/_tests/conftest.py
Please move it to a top level conftest file at the rootdir:
  None
For more information, visit:
  https://docs.pytest.org/en/latest/deprecations.html#pytest-plugins-in-non-top-level-conftest-files

[PureTryOut]

@Alessandro-Barbieri just a note, I got tests working with PYTHONPATH="." pytest --pyargs pytest_trio. This did not require me to install it first and works with Pytest 5.4.2.

[Alessandro-Barbieri]

I tried and now the tests run but with 33 failures, might open a new issue for that

[altendky]

Seems similar to #103 (comment). Maybe the same thing will help?

tl;dr: PYTHONPATH="." PYTEST_PLUGINS="pytest_trio.plugin" pytest --pyargs pytest_trio