New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent ImportError #2027

Closed
hsienchiaolee opened this Issue Oct 25, 2016 · 7 comments

Comments

Projects
None yet
2 participants
@hsienchiaolee
Copy link

hsienchiaolee commented Oct 25, 2016

Hi,

I am seeing a really weird issue where if I run pytest inside my proj root directory, it gives me an import error; however, when I run with a specific test, pytest tests/repositories/test_repository.py, everything is fine. What am I doing wrong?

ImportError from pytest:

ImportError while importing test module '~/workspace/app/tests/repositories/test_repository.py'.
Original error message:
'cannot import name utils'
Make sure your test modules/packages have valid Python names.

Inside test_repository.py, I am importing with: from app import utils.

Project directory structure:

app
├── README.md
├── __init__.py
├── __init__.pyc
├── app
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── repositories
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── repository.py
│   │   └── repository.pyc
│   ├── utils.py
│   └── utils.pyc
├── app.egg-info
│   ├── PKG-INFO
│   ├── SOURCES.txt
│   ├── dependency_links.txt
│   ├── requires.txt
│   └── top_level.txt
├── setup.cfg
├── setup.py
└── tests
    ├── __init__.py
    ├── __init__.pyc
    ├── __pycache__
    │   └── test_utils.cpython-27-PYTEST.pyc
    ├── fakes
    │   ├── __init__.py
    │   ├── __init__.pyc
    │   ├── fake_service_resource.py
    │   └── fake_service_resource.pyc
    ├── repositories
    │   ├── __pycache__
    │   │   └── test_repository.cpython-27-PYTEST.pyc
    │   └── test_repository.py
    ├── test_utils.py
    ├── repositories
    │   ├── __pycache__
    │   │   └── test_dynamodb_repository.cpython-27-PYTEST.pyc
    │   └── test_dynamodb_repository.py

setup.py:

from setuptools import setup

setup(
    name='app',
    packages=[
      'app',
      'app.repositories',
      ],
    include_package_data=True,
    install_requires=[],
    setup_requires=[
        'pytest-runner',
    ],
    tests_require=[
        'pytest',
        'mock',
    ],
)
@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 26, 2016

Hey @hsienchiaolee,

Perhaps it is a problem on how you are setting up your application for development:

  1. Are you using a virtualenv and installing your application with pip install -e .?

  2. What happens when you run this from the root of the repository:

    $ python -c "import app;print(app.__file__)"
    
  3. From your description I assume you are running pytest with cwd pointing to ~/workspace, is that correct?

Is your repository public so we can take a look?

@hsienchiaolee

This comment has been minimized.

Copy link

hsienchiaolee commented Oct 26, 2016

Hi @nicoddemus ,

  1. Yes, I am using virtualenv and I did run pip install -e . before running pytest.
  2. It shows: app/__init__.pyc
  3. I am running pytest with cwd pointing ~/workspace/app

Unfortunately, the repository is private, and I am not at liberty to share. =[

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 26, 2016

I just noticed this:

app
├── README.md
├── __init__.py
├── __init__.pyc
├── app
│   ├── __init__.py
│   ├── __init__.pyc

It seems to me app/__init__.py should not be there because the actual package should be app/app/__init__.py.

When Python executes, it will add the path of the script as the first entry in sys.path, so it will pickup the package at ~/workspace/app instead of the correct package at ~/workspace/app/app.

This is not really a pytest issue, it is how the Python import mechanism works.

@hsienchiaolee

This comment has been minimized.

Copy link

hsienchiaolee commented Oct 26, 2016

One level higher tree looks like:

workspace
├── app
├──── app
│             ├── __init__.py
│             ├── __init__.pyc

So you are correct, the __init__.py is app/app/__.init__.py. By running pytest in ~/workspace/app, print(app.__file__) should show app/__init__.py

An additional information that could be helpful is that the tests were all working before until I moved app into another directory. e.g. ~/workspace/app/others/app/ -> ~/workspace/app/app/. I have ran pip install -e . after moving the directory, and this is when I started seeing the ImportErrors.

Also, before moving the directory, I ran sudo pip install -e . without virtualenv (I know...I didn't know what I was doing =P). Maybe there are some caches that I need to clear?

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 26, 2016

I think the formatting of your example is messed up?

So you are correct, the init.py is app/app/.init.py. By running pytest in ~/workspace/app, print(app.file) should show app/init.py

I'm not sure I follow, I see two __init__.py files in your original post:

  • ~/workspace/app/__init__.py
  • ~/workspace/app/app/__init__.py

If I'm getting the entire picture, the first one is the one causing problems.

I suggest using a different name for the root folder to avoid confusion (something like ~/workspace/app-root).

Also, before moving the directory, I ran sudo pip install -e . without virtualenv (I know...I didn't know what I was doing =P). Maybe there are some caches that I need to clear?

I would try pip uninstall app, but never tried it. If that doesn't work, you might want to remove the app.pth file from your site-packages directory which that is placed by pip install -e.

@hsienchiaolee

This comment has been minimized.

Copy link

hsienchiaolee commented Oct 26, 2016

Wow, you are right, there was an extra init.py in the app-root that you were talking about. I removed it and it works now. Thank you so much!

@nicoddemus

This comment has been minimized.

Copy link
Member

nicoddemus commented Oct 26, 2016

No worries, glad I could help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment