Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ModuleNotFoundError when using the tests outside application code layout #2421

Closed
MFry opened this Issue May 20, 2017 · 17 comments

Comments

Projects
None yet
10 participants
@MFry
Copy link

MFry commented May 20, 2017

  • Include a detailed description of the bug or suggestion
    I was following the test layout in Good Integration Practices and I couldn't get my tests that were in a similar configuration to run or the simple example I created below.

  • pip list of the virtual environment you are using

  • pytest and operating system versions
    Ubuntu 16.04.2 LTS (GNU/Linux 4.4.0-77-generic x86_64)
    platform linux -- Python 3.6.1, pytest-3.0.7, py-1.4.33, pluggy-0.4.0

  • Minimal example if possible

\test_pytest
   mypkg/
       __init__.py
       app.py
   tests/
       test_app.py

and the files

# test_app.py
import app

# tried:
# import mypkg.app
# import ..app
# from mypkg import app
# app.py
print('It Works')
def test_pass():
    assert True == True

Running pytest from the test_pytest directory will result in an error similar to this one.

(py36env) ubuntu@ubuntu-xenial:/vagrant/test_pytest$ pytest

ImportError while importing test module '/vagrant/test_pytest/test/test_app.py'.
Hint: make sure your test modules/packages have valid Python names.
test/test_app.py:1: in <module>
    import app
E   ModuleNotFoundError: No module named 'app'

I can add to the sys path manually by adding a __init__.py to tests/

# test/__init__.py
import sys
sys.path.append('/vagrant/test_pytest/mypkg')

but that will complicate testing with Travis CI

@BeyondEvil

This comment has been minimized.

Copy link

BeyondEvil commented Jun 9, 2017

I have the same issue.

My project: https://github.com/Projectplace/pytest-setup

Running tests using tox and python 2.7 works.

Running tests using tox and python 3.6 fails. However, running the tests individually tox -- -k <test_name> passes.

The plugin works as expected when in use on both 2.7 and 3.6.

@yfpeng

This comment has been minimized.

Copy link

yfpeng commented Jul 28, 2017

from http://python-guide-pt-br.readthedocs.io/en/latest/writing/structure/

You can create a context.py file in tests:

from __future__ import absolute_import

import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

import mypkg

Then in your test file, add from .context import mypkg

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Jul 28, 2017

im closing this issue as it relates to a miss-understanding of the python import system, the errors are indeed intended by python itself

@florent-vial

This comment has been minimized.

Copy link

florent-vial commented Sep 26, 2017

@yfpeng @RonnyPfannschmidt i followed this exact same guide, but when i execute py.test-3 in test_pytest, i get following error:

Parent module '' not loaded, cannot perform relative import

I am reluctent to add an __init__.py in my tests directory since this is against what the documentation is offering.
Pytest documentation does not provide IMO enough information on how to execute tests properly in that case either.

Do you know how to execute tests properly in that case?

@obestwalter

This comment has been minimized.

Copy link
Member

obestwalter commented Sep 26, 2017

As a general recommendation: Always run tests against your installed package. If your package is not installable yet: make it installable. When you develop, install your package with pip install --editable <path/to/package>. When you need to release it somewhere: test the package with tox.

I am speaking from a place of major pain in the past, when I was not doing these things myself and when packaging was in a much worse state than it is today. That all went away now and is replaced by major packaging pains 😀

But joking aside: in most cases it is really much easier to create a simple setup.py to make your project installable and run tests against that, than going through all this hackery to make it work without.

@florent-vial

This comment has been minimized.

Copy link

florent-vial commented Sep 26, 2017

@obestwalter thanks for the hint. In my case, i very much more appreciate having a code base i can consistently and continuously test (without any additional packaging pain).
The scripts are made available through a pure git clone of a repo and IMO this use case should be supported "out of the box" for testing.

@obestwalter

This comment has been minimized.

Copy link
Member

obestwalter commented Sep 26, 2017

The scripts are made available through a pure git clone of a repo and IMO this use case should be supported "out of the box" for testing.

Yes, This is how I used to do it, because I thought it was simpler that way. Turns out it isn't in the long run. Learning about Python packaging is really worth the extra effort in my experience, especially because it works pretty darn good nowadays thanks to the folks at https://github.com/pypa.

@henryiii

This comment has been minimized.

Copy link

henryiii commented Jul 10, 2018

@obestwalter has the correct solution. However, if you need something temporary while you are developing a package, you can simple add the following file to your tests directory, and then pytest will run it before the tests (so no changes are needed per-file when you make a final package, you just delete this one file):

conftest.py

import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))

(This is one of the top google results for this error, so adding this even though the issue is closed)

PS: This is a hack, please use the editable install solution unless you have good reason to use this one. And do not forget to remove this before publishing to PyPI.

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Jul 10, 2018

@henryiii i strongly suggest to avoid such hacks and use editable installs instead

@henryiii

This comment has been minimized.

Copy link

henryiii commented Jul 10, 2018

@RonnyPfannschmidt I fully agree; this is just useful if you aren't ready to do an editable install or in special cases; and if so, this is a better solution than one non-editable example shown above because it converts much more easily to the correct solution.

In my case, I'm temporarily testing locally against CERN ROOT, and that has to be done against the system python on a Mac, and ROOT does not support pretty much anything nice in Python like virtual environments very well. This is a good way to temporarily test (once it's in Travis, I can test against system Python there). My packages (from cookiecutter) is already installable from moment 0, I just don't want to mess with my user directory by installing the system python one. So I'm sure there is at least one special case where this is useful. :)

@RonnyPfannschmidt

This comment has been minimized.

Copy link
Member

RonnyPfannschmidt commented Jul 10, 2018

@henryiii that one is a understandable exception, thanks for sharing the context

@GandalfSaxe

This comment has been minimized.

Copy link
Contributor

GandalfSaxe commented Aug 2, 2018

@RonnyPfannschmidt I think there is a documentation issue here. I also followed the guide Good Integration Practices, and I would've felt pretty lost if I hadn't stumbled upon this issue.

I think some sort of very short guide to how to use pip editable mode would be very welcome in said guide, if that is the recommended way to go no matter how you structure your tests, as long as it's the "Tests outside application code" type 🙂

@GandalfSaxe

This comment has been minimized.

Copy link
Contributor

GandalfSaxe commented Aug 2, 2018

Question 1:
Following the Tests outside application code model (which is mentioned as "often a good idea"), this only runs for me if I add the __init__.py, otherwise pytest complains that the module being tested does not exist. Why is the __init__.py file mentioned as optional and only strictly necessary when test files do not have unique names, when it seems to be always necessary?

Question 2:
What benefits does the pip install -e <path/to/package> method have over the the method from question 1?

@obestwalter

This comment has been minimized.

Copy link
Member

obestwalter commented Aug 19, 2018

I think some sort of very short guide to how to use pip editable mode would be very welcome in said guide, if that is the recommended way to go no matter how you structure your tests [...]

If you read a bit further down it is explained in the tox section: https://docs.pytest.org/en/latest/goodpractices.html#tox

@GandalfSaxe

This comment has been minimized.

Copy link
Contributor

GandalfSaxe commented Aug 20, 2018

Ah ok. It's good that it's there, but it should be front and center, I don't see it belonging in the tox section.

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Aug 22, 2018

@GandalfSaxe you make a good point. We would appreciate a quick PR to the docs if you got the time, or create a new issue so it doesn't get lost into this one. 👍

GandalfSaxe added a commit to GandalfSaxe/pytest that referenced this issue Aug 23, 2018

@YakDriver

This comment has been minimized.

Copy link

YakDriver commented Jan 18, 2019

I have my project setup the exact same way, had the same issue but fixed it. My problem was having pytest installed system wide (outside of my virtual environment) and a package installed only in my virtualenv. pytest was looking in system site-packages. By installing pytest in virtualenv it solved the issue. Adding this comment in case you end up here from Google (like me).

Longer explanation: https://medium.com/@dirk.avery/pytest-modulenotfounderror-no-module-named-requests-a770e6926ac5

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.