Broken editable install of the project itself? #649

pawamoy · 2021-09-11T13:07:20Z

I have searched the issue tracker and believe that this is not a duplicate.

There's something going on. Yesterday I was working on a project, and everything worked, and moments later, I couldn't run my CLI tool anymore. I'm investigating today again.

This used to work:

% python -m griffe
/usr/bin/python: No module named griffe.__main__; 'griffe' is a package and cannot be directly executed

So I'm like, OK, lets try with pdm:

% pdm run griffe
Traceback (most recent call last):
  File "/media/data/dev/griffe/__pypackages__/3.9/bin/griffe", line 5, in <module>
    from griffe.cli import main
ModuleNotFoundError: No module named 'griffe.cli'

What's happening? Lets list the contents of __pypackages__.

% ll __pypackages__/3.9/lib
total 716K
drwxr-xr-x 2 pawamoy users 4.0K Sep 11 14:49 griffe
drwxr-xr-x 2 pawamoy users 4.0K Sep 11 14:49 griffe-0.1.0+d20210911.dist-info
-rw-r--r-- 1 pawamoy users   14 Sep 11 14:49 griffe.pth
-rw-r--r-- 1 pawamoy users  139 Sep 11 14:49 _griffe.py

# relevant contents only

OK, what's in all this?

% ll __pypackages__/3.9/lib/griffe
total 0
-rw-r--r-- 1 pawamoy users 0 Sep 11 14:53 py.typed

That explains the ModuleNotFoundErrors. Nothing in this directory.

% cat __pypackages__/3.9/lib/griffe.pth
import _griffe

% cat __pypackages__/3.9/lib/_griffe.py
from editables.redirector import RedirectingFinder as F
F.install()
F.map_module('griffe', '/media/data/dev/griffe/src/griffe/__init__.py')

Does this come from PDM?

% cat __pypackages__/3.9/lib/griffe-0.1.0+d20210911.dist-info/RECORD 
../bin/griffe,sha256=c1929c0a3f0745abc2a583a6942b2de8516ba10510fc57666bca58b6f47e87aa,211
griffe/py.typed,sha256=e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855,0
griffe.pth,sha256=eb6dc9570253f46b83cbce7e9ed8be76563b9874dd3b14a20600854cd3f2c367,14
_griffe.py,sha256=e4d75e89e7f2c219d8b2628d65c665f3cb39b23d13e423e09d3a3ae6287a5e10,139
griffe-0.1.0+d20210911.dist-info/entry_points.txt,sha256=81028a1dd270a1f2cb9966cb7c5756b0d83ce176ac5a5106812012ea06c0fc45,42
griffe-0.1.0+d20210911.dist-info/WHEEL,sha256=a23c91d75e2282fd898cebe66304e98ede8608a4da60f79176b2881e5a853bad,87
griffe-0.1.0+d20210911.dist-info/METADATA,sha256=bb17b48ccc7093b255c0816ed563342c770d55e2c303c6fb224fb9888ef4992c,3043
griffe-0.1.0+d20210911.dist-info/LICENSE,sha256=2466f8a5d3c433c4d38e386bfae3423bba1792256bc06e4fcf425a99c8cd17fb,754
griffe-0.1.0+d20210911.dist-info/INSTALLER,sha256=2581b4c142e7c10ed0b1975de1e2b9f3db44b4be4964ee99881f6ca6b362c986,15
griffe-0.1.0+d20210911.dist-info/direct_url.json,sha256=56736bedb20e23873455a9325c8468360c726ba9938114630b561e9607206445,86
griffe-0.1.0+d20210911.dist-info/RECORD,,

At this point I tried everything: reinstalling PDM, switching the feature.install_cache on and off, deleting PDM's cache, reinstalling all dependencies, trying with PDM 1.8.2, trying with all Python versions (3.6, 3.7, 3.8, 3.9, 3.10), etc.
Nothing fixes it.

So I tried in another project. There I get this:

% python -m duty -h
Error processing line 1 of /media/data/dev/duty/__pypackages__/3.9/lib/duty.pth:

  Traceback (most recent call last):
    File "/usr/lib/python3.9/site.py", line 169, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "/media/data/dev/duty/__pypackages__/3.9/lib/_duty.py", line 1, in <module>
      from editables.redirector import RedirectingFinder as F
  ModuleNotFoundError: No module named 'editables'

Remainder of file ignored
/usr/bin/python: No module named duty.__main__; 'duty' is a package and cannot be directly executed

% pdm run duty -h
Error processing line 1 of /media/data/dev/duty/__pypackages__/3.9/lib/duty.pth:

  Traceback (most recent call last):
    File "/usr/lib/python3.9/site.py", line 169, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "/media/data/dev/duty/__pypackages__/3.9/lib/_duty.py", line 1, in <module>
      from editables.redirector import RedirectingFinder as F
  ModuleNotFoundError: No module named 'editables'

Remainder of file ignored
Error processing line 1 of /media/data/dev/duty/__pypackages__/3.9/lib/duty.pth:

  Traceback (most recent call last):
    File "/usr/lib/python3.9/site.py", line 169, in addpackage
      exec(line)
    File "<string>", line 1, in <module>
    File "/media/data/dev/duty/__pypackages__/3.9/lib/_duty.py", line 1, in <module>
      from editables.redirector import RedirectingFinder as F
  ModuleNotFoundError: No module named 'editables'

Remainder of file ignored
Traceback (most recent call last):
  File "/media/data/dev/duty/__pypackages__/3.9/bin/duty", line 5, in <module>
    from duty.cli import main
ModuleNotFoundError: No module named 'duty.cli'

OK so it seems to have something to do with the editables package?

Steps to reproduce

git clone https://github.com/pawamoy/griffe
cd griffe
pdm install
python -m griffe
pdm run griffe

Actual behavior

Problem with editable installation of the project.

Expected behavior

/

Environment Information

% pdm info && pdm info --env
PDM version:        1.8.3                                     
Python Interpreter: /usr/bin/python3.10 (3.10)                
Project Root:       /media/data/dev/griffe                    
Project Packages:   /media/data/dev/griffe/__pypackages__/3.10
{
  "implementation_name": "cpython",
  "implementation_version": "3.10.0c2",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.14.2-arch1-2",
  "platform_system": "Linux",
  "platform_version": "#1 SMP PREEMPT Thu, 09 Sep 2021 09:42:35 +0000",
  "python_full_version": "3.10.0rc2",
  "platform_python_implementation": "CPython",
  "python_version": "3.10",
  "sys_platform": "linux"
}

The text was updated successfully, but these errors were encountered:

pawamoy · 2021-09-11T13:17:40Z

Lead: the only dependency that has been updated recently is setuptools: version 58.0.4 released two days ago.
It is in the shared dependencies managed by pipx. That could explain how everything stopped working while I didn't do anything particular in projects themselves. pipx regularly updates shared dependencies when running some of its commands.

pawamoy · 2021-09-11T13:34:54Z

Nope, installed setuptools < 58 in both shared pipx env and as a dependency of my project, didn't solve it.

frostming · 2021-09-13T03:15:42Z

Is editables listed in __pypackages__/lib ? I can't reproduce with the steps since some local directories are missing: mkdocs and pytkdocs

frostming · 2021-09-13T03:18:38Z

Also please upgrade the version of pdm-pep517

laike9m · 2021-09-13T04:28:51Z

This happened to me as well

pawamoy · 2021-09-13T16:51:37Z

% ll __pypackages__/*/lib/editables*
-rw-r--r-- 1 pawamoy users   65 Sep 13 18:47 __pypackages__/3.10/lib/editables.pth
-rw-r--r-- 1 pawamoy users   65 Sep 13 18:48 __pypackages__/3.11/lib/editables.pth
-rw-r--r-- 1 pawamoy users   65 Sep 13 18:47 __pypackages__/3.8/lib/editables.pth
-rw-r--r-- 1 pawamoy users   65 Sep 13 18:47 __pypackages__/3.9/lib/editables.pth

__pypackages__/3.10/lib/editables-0.2.dist-info:
total 28K
-rw-r--r-- 1 pawamoy users   15 Sep 13 18:47 INSTALLER
-rw-r--r-- 1 pawamoy users 1.1K Sep 13 18:47 LICENSE.txt
-rw-r--r-- 1 pawamoy users 3.8K Sep 13 18:47 METADATA
-rw-r--r-- 1 pawamoy users  780 Sep 13 18:47 RECORD
-rw-r--r-- 1 pawamoy users   60 Sep 13 18:47 REFER_TO
-rw-r--r-- 1 pawamoy users   10 Sep 13 18:47 top_level.txt
-rw-r--r-- 1 pawamoy users   92 Sep 13 18:47 WHEEL

__pypackages__/3.11/lib/editables-0.2.dist-info:
total 28K
-rw-r--r-- 1 pawamoy users   15 Sep 13 18:48 INSTALLER
-rw-r--r-- 1 pawamoy users 1.1K Sep 13 18:48 LICENSE.txt
-rw-r--r-- 1 pawamoy users 3.8K Sep 13 18:48 METADATA
-rw-r--r-- 1 pawamoy users  780 Sep 13 18:48 RECORD
-rw-r--r-- 1 pawamoy users   60 Sep 13 18:48 REFER_TO
-rw-r--r-- 1 pawamoy users   10 Sep 13 18:48 top_level.txt
-rw-r--r-- 1 pawamoy users   92 Sep 13 18:48 WHEEL

__pypackages__/3.8/lib/editables-0.2.dist-info:
total 28K
-rw-r--r-- 1 pawamoy users   15 Sep 13 18:47 INSTALLER
-rw-r--r-- 1 pawamoy users 1.1K Sep 13 18:47 LICENSE.txt
-rw-r--r-- 1 pawamoy users 3.8K Sep 13 18:47 METADATA
-rw-r--r-- 1 pawamoy users  780 Sep 13 18:47 RECORD
-rw-r--r-- 1 pawamoy users   60 Sep 13 18:47 REFER_TO
-rw-r--r-- 1 pawamoy users   10 Sep 13 18:47 top_level.txt
-rw-r--r-- 1 pawamoy users   92 Sep 13 18:47 WHEEL

__pypackages__/3.9/lib/editables-0.2.dist-info:
total 28K
-rw-r--r-- 1 pawamoy users   15 Sep 13 18:47 INSTALLER
-rw-r--r-- 1 pawamoy users 1.1K Sep 13 18:47 LICENSE.txt
-rw-r--r-- 1 pawamoy users 3.8K Sep 13 18:47 METADATA
-rw-r--r-- 1 pawamoy users  780 Sep 13 18:47 RECORD
-rw-r--r-- 1 pawamoy users   60 Sep 13 18:47 REFER_TO
-rw-r--r-- 1 pawamoy users   10 Sep 13 18:47 top_level.txt
-rw-r--r-- 1 pawamoy users   92 Sep 13 18:47 WHEEL

% cat __pypackages__/*/lib/editables.pth
/home/pawamoy/.cache/pdm/packages/editables-0.2-py3-none-any/lib
/home/pawamoy/.cache/pdm/packages/editables-0.2-py3-none-any/lib
/home/pawamoy/.cache/pdm/packages/editables-0.2-py3-none-any/lib
/home/pawamoy/.cache/pdm/packages/editables-0.2-py3-none-any/lib

% ll /home/pawamoy/.cache/pdm/packages/editables-0.2-py3-none-any/lib/editables:
total 12K
-rw-r--r-- 1 pawamoy users 2.0K Sep 13 18:47 __init__.py
-rw-r--r-- 1 pawamoy users  761 Sep 13 18:47 redirector.py

% pdm show pdm-pep517 --version
0.8.3

pawamoy · 2021-09-13T16:54:20Z

I can't reproduce with the steps since some local directories are missing: mkdocs and pytkdocs

Try commenting these two, their latest releases are listed in pyproject.toml as well. You don't need my local versions to try python -m griffe 🙂

pawamoy · 2021-09-13T16:57:49Z

I'm using this simple Bash script for now to fix my installs:

#!/usr/bin/env bash
for package; do
  for py in 3.6 3.7 3.8 3.9 3.10 3.11; do
    dir=__pypackages__/$py
    if [ -d "${dir}" ]; then
      echo /media/data/dev/$package/src > $dir/lib/$package.pth
    fi
  done
done

% fix-pdm-editables griffe mkdocstrings pytkdocs
% python -m griffe
usage: griffe [-h] [-s SEARCH] PACKAGE [PACKAGE ...]
griffe: error: the following arguments are required: PACKAGE

(griffe modules can now be found)

frostming · 2021-09-14T02:48:36Z

@pawamoy I have updated pdm-pep517 backend, please run pdm install and see if the issue still exists. The workaround is not valid since it changes how editable installs work. You may need to delete those .pth files

pawamoy · 2021-09-14T16:36:22Z

That seems to work, thanks!

frostming · 2021-09-15T00:29:34Z

@laike9m If you believe your issue is the same, can you help verify it? Otherwise, I am going to close this.

laike9m · 2021-09-15T06:51:13Z

It's working fine for me.

q0w · 2021-09-20T13:16:13Z

I still have this bug, deleting .pth works only the first time, but installing project itself again raises the same error.
I'm using pdm-venv

pdm 1.8.5
pdm-pep517 0.8.4
editables 0.2

frostming · 2021-09-22T01:06:50Z

@q0w Can you provide more information on what package you installed?

q0w · 2021-09-24T10:13:22Z

@frostming Just initialized a new project with pdm-venv, add attrs and then removed it, got error:
ModuleNotFoundError: No module named '_project_name'

frostming · 2021-09-24T10:39:11Z

@q0w The information is not enough to figure out the problem.

q0w · 2021-09-24T11:50:21Z

@frostming so now i got ModuleNotFoundError: No module named 'editables' when i try pdm build. I deleted .pth files, then pdm install works fine, but pdm build failed

frostming · 2021-09-24T12:13:22Z

you should provide an example so that i can reproduce, otherwise i can't help

…

在 2021年9月24日，19:50，q0w ***@***.***> 写道： @frostming so now i got ModuleNotFoundError: No module named 'editables' when i try pdm build. I deleted .pth files, then pdm install works fine, but pdm build failed — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

pawamoy added the 🐛 bug Something isn't working label Sep 11, 2021

frostming closed this as completed Sep 15, 2021

CharString mentioned this issue Dec 8, 2021

No module named 'editables' pdm-project/pdm-venv#26

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Broken editable install of the project itself? #649

Broken editable install of the project itself? #649

pawamoy commented Sep 11, 2021 •

edited

pawamoy commented Sep 11, 2021 •

edited

pawamoy commented Sep 11, 2021

frostming commented Sep 13, 2021

frostming commented Sep 13, 2021

laike9m commented Sep 13, 2021

pawamoy commented Sep 13, 2021

pawamoy commented Sep 13, 2021

pawamoy commented Sep 13, 2021

frostming commented Sep 14, 2021 •

edited

pawamoy commented Sep 14, 2021

frostming commented Sep 15, 2021

laike9m commented Sep 15, 2021

q0w commented Sep 20, 2021 •

edited

frostming commented Sep 22, 2021

q0w commented Sep 24, 2021

frostming commented Sep 24, 2021

q0w commented Sep 24, 2021

frostming commented Sep 24, 2021 via email

Broken editable install of the project itself? #649

Broken editable install of the project itself? #649

Comments

pawamoy commented Sep 11, 2021 • edited

Steps to reproduce

Actual behavior

Expected behavior

Environment Information

pawamoy commented Sep 11, 2021 • edited

pawamoy commented Sep 11, 2021

frostming commented Sep 13, 2021

frostming commented Sep 13, 2021

laike9m commented Sep 13, 2021

pawamoy commented Sep 13, 2021

pawamoy commented Sep 13, 2021

pawamoy commented Sep 13, 2021

frostming commented Sep 14, 2021 • edited

pawamoy commented Sep 14, 2021

frostming commented Sep 15, 2021

laike9m commented Sep 15, 2021

q0w commented Sep 20, 2021 • edited

frostming commented Sep 22, 2021

q0w commented Sep 24, 2021

frostming commented Sep 24, 2021

q0w commented Sep 24, 2021

frostming commented Sep 24, 2021 via email

pawamoy commented Sep 11, 2021 •

edited

pawamoy commented Sep 11, 2021 •

edited

frostming commented Sep 14, 2021 •

edited

q0w commented Sep 20, 2021 •

edited