ModuleNotFoundError on local import

[abitrolly]

flit fails on import of local package that is present in subdir.

✗ pipenv run flit build
Traceback (most recent call last):
  File "/home/anatoli/.local/share/virtualenvs/ksykaitai-1rTdPa8n/bin/flit", line 8, in <module>
    sys.exit(main())
  File "/home/anatoli/.local/share/virtualenvs/ksykaitai-1rTdPa8n/lib/python3.9/site-packages/flit/__init__.py", line 154, in main
    main(args.ini_file, formats=set(args.format or []),
  File "/home/anatoli/.local/share/virtualenvs/ksykaitai-1rTdPa8n/lib/python3.9/site-packages/flit/build.py", line 45, in main
    sb = SdistBuilder.from_ini_path(ini_file)
  File "/home/anatoli/.local/share/virtualenvs/ksykaitai-1rTdPa8n/lib/python3.9/site-packages/flit_core/sdist.py", line 103, in from_ini_path
    metadata = common.make_metadata(module, ini_info)
  File "/home/anatoli/.local/share/virtualenvs/ksykaitai-1rTdPa8n/lib/python3.9/site-packages/flit_core/common.py", line 366, in make_metadata
    md_dict.update(get_info_from_module(module))
  File "/home/anatoli/.local/share/virtualenvs/ksykaitai-1rTdPa8n/lib/python3.9/site-packages/flit_core/common.py", line 173, in get_info_from_module
    docstring, version = get_docstring_and_version_via_import(target)
  File "/home/anatoli/.local/share/virtualenvs/ksykaitai-1rTdPa8n/lib/python3.9/site-packages/flit_core/common.py", line 156, in get_docstring_and_version_via_import
    m = sl.load_module()
  File "<frozen importlib._bootstrap_external>", line 469, in _check_name_wrapper
  File "<frozen importlib._bootstrap_external>", line 969, in load_module
  File "<frozen importlib._bootstrap_external>", line 794, in load_module
  File "<frozen importlib._bootstrap>", line 274, in _load_module_shim
  File "<frozen importlib._bootstrap>", line 711, in _load
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 790, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "ksykaitai.py", line 7, in <module>
    import kaitaiStructCompile
ModuleNotFoundError: No module named 'kaitaiStructCompile'
...

The package kaitaiStructCompile is perfectly visible and importable with plain Python 3.

Flit settings specifyksykaitai.py from root dir.

[tool.flit.metadata]
module = "ksykaitai"
author = "abitrolly"

The directory tree is below.

✗ tree              
.
├── Code_Of_Conduct.md
├── data
│   ├── squashfs_superblock_generated.py
│   ├── squashfs_superblock.ksy
│   └── yakshaveinc_eternal_amd64.snap
├── dist
├── Doxyfile
├── kaitaiStructCompile
│   ├── backend
│   │   └── cmdline.py
│   ├── backendSelector.py
│   ├── colors.py
│   ├── defaults.py
│   ├── ICompiler.py
│   ├── __init__.py
│   ├── KaitaiCompilerException.py
│   └── utils.py
├── kaitaistruct.py
├── ksykaitai.py
├── MANIFEST.in
├── Pipfile
├── Pipfile.lock
├── prepare.sh
├── pyproject.toml
├── README.md
├── release.sh
├── setup.cfg
├── tests
│   ├── ksys
│   └── test.py
└── UNLICENSE

To repeat.

git clone https://github.com/abitrolly/ksykaitai -b flit386
cd ksykaitai
pipenv install flit
pipenv run flit build

✗ pipenv run flit --version
Flit 3.0.0

[goodmami]

I have the same symptom but a different package structure. I have my version specified in a _meta.py file under the main package, and the top __init__.py imports the version:

$  python -m pip install .
[...]
      File "/tmp/pip-build-env-pwmxd890/overlay/lib/python3.8/site-packages/flit_core/common.py", line 165, in get_docstring_and_version_via_import
        m = sl.load_module()
      File "<frozen importlib._bootstrap_external>", line 462, in _check_name_wrapper
      File "<frozen importlib._bootstrap_external>", line 962, in load_module
      File "<frozen importlib._bootstrap_external>", line 787, in load_module
      File "<frozen importlib._bootstrap>", line 265, in _load_module_shim
      File "<frozen importlib._bootstrap>", line 702, in _load
      File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 783, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "/tmp/pip-req-build-id8hntg4/wn/__init__.py", line 37, in <module>
        from wn._meta import __version__
    ModuleNotFoundError: No module named 'wn'

This happens when I try to install via pip but not when I use flit install. Could this be a pip issue?

[takluyver]

In the latter case, does it work if you use a relative import (from ._meta import __version__) rather than an absolute one?

Background: Flit has two ways of getting the version number from your code. It prefers static analysis to find a __version__ = statement in __init__.py, and if that fails, it falls back to actually loading the module, which is necessarily more error prone. I'm guessing that what's happening in both cases is that it's importing the module without adding its parent directory to sys.path, so it doesn't know how to load other, associated modules.

One thing I've been thinking of is having some way to configure where it looks for the version, so you could use the simpler static analysis method even if __version__ was defined somewhere other than __init__.py. E.g. something like:

[tool.flit.version]
module = "wn._meta"  # Or a filename - not sure which is better yet
name = "__version__"  # This would be the default
method = "static"  # as opposed to "import"

[goodmami]

...does it work if you use a relative import (from ._meta import __version__) rather than an absolute one?

Yes, that does seem to help when installing with pip. I wonder if I had no trouble with flit install before because I did it from the project root, thereby putting the source directory on sys.path?

One thing I've been thinking of is having some way to configure where it looks for the version, so you could use the simpler static analysis method even if __version__ was defined somewhere other than __init__.py. E.g. something like:

[tool.flit.version]
module = "wn._meta"  # Or a filename - not sure which is better yet
name = "__version__"  # This would be the default
method = "static"  # as opposed to "import"

Glad to see you're thinking of this. I suggested something similar in #253 (comment), but without the option of customizing the variable name or retrieval method. E.g:

[tool.flit.metadata]
version-location = "wn._meta"

Here I assumed the module would be loaded completely independently of the main module (in which case maybe a filename is better), thus sidestepping the dependency issue with regular imports and also allowing people to use setuptools_scm or whatever they like in there. It now occurs to me that people may want to use such a module for other metadata as well (__author__, __author_email__, etc.), so maybe calling it version-location is too narrow.

[smidm]

The same problem as with __version__ is with __doc__. For me the flit build fails unless I add a comment (__doc__) and a __version__ in __init__.py. It gave me a bit of headache...

[goodmami]

@smidm good point. Being able to specify a location for all the metadata that is expected to be in the top __init__.py would also help with projects where the main package is a namespace package. That is, this is essentially the same solution as is proposed for #370. Two birds one stone?

[smidm]

From my usage so far a clear error message would suffice, e.g. "Can't find __doc__ and __version__, please specify in pyproject.toml or __init__.py". I just got a ModuleNotFound error and I searched for a problem in my package and in virtualenv until I fired pdb and checked the source.

…

On Fri, Apr 9, 2021 at 20:57, Michael Wayne Goodman ***@***.***> wrote: @smidm good point. Being able to specify a location for all the metadata that is expected to be in the top __init__.py would also help with projects where the main package is a namespace package. That is, this is essentially the same solution as is proposed for #370. Two birds one stone? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[mwtoews]

I've run into this issue too with flit. It is not an issue with setuptools' declarative configuration setup.cfg:

[metadata]
name = wn
version = attr: wn._meta.__version__

the proposed solutions above look like good candidates!