Upgrading from 23.0.1 to 23.1 causes ModuleNotFoundError

[wesk]

Description

Our GitHub Actions tests started failing two days ago when pip 23.1 was released, example test runs:

• https://github.com/awslabs/syne-tune/actions/runs/4716666731/jobs/8369933296?pr=624
• https://github.com/awslabs/syne-tune/actions/runs/4718647382?pr=625

Before 23.1, we were able to install our package locally using the following command: python -m pip install -e '.[extra]'

We were previously able to run scripts that had import statements like the following:

from examples.training_scripts.height_example.train_height import (
    METRIC_ATTR,
    METRIC_MODE,
    MAX_RESOURCE_ATTR,
)

Note that the examples module is a top-level folder in the GitHub repository. It is not under the syne_tune folder, since we don't want it to be included in the bundle that users typically pip install. If customers want to run our examples, we've been recommending that they check out the package locally and then run the examples.

However, with 23.1, now we get a ModuleNotFound:

Traceback (most recent call last):
  File "/Users/jkkndr/code/repositories/syne-tune/examples/launch_tensorboard_example.py", line 35, in <module>
    from examples.training_scripts.height_example.train_height import (
ModuleNotFoundError: No module named 'examples'

Is this behavior change intentional, perhaps from this removal from the changelog:

Remove support for the deprecated --install-options. ([#11358](https://github.com/pypa/pip/issues/11358))

Expected behavior

Expected to be able to run the following script without having a ModuleNotFound error thrown on line 35: https://github.com/awslabs/syne-tune/blob/main/examples/launch_tensorboard_example.py#L35

pip version

23.1

Python version

3.8

OS

both Ubuntu & Mac

How to Reproduce

Check out https://github.com/awslabs/syne-tune, then run:

python -m pip install --upgrade pip
python -m pip install -e '.[extra]'
python examples/launch_tensorboard_example.py 

If I just do pip install pip==23.0.1, (the old version), I no longer see the error

Output

(st_venv) jkkndr@88665a1f8b71 syne-tune % python examples/launch_tensorboard_example.py 
Traceback (most recent call last):
  File "/Users/jkkndr/code/repositories/syne-tune/examples/launch_tensorboard_example.py", line 35, in <module>
    from examples.training_scripts.height_example.train_height import (
ModuleNotFoundError: No module named 'examples'

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pfmoore]

This is almost certainly due to how setuptools implements editable installs.

In 23.1, we stopped invoking setup.py install automatically for non-pyproject packages, when the wheel package is not installed in the user's environment. This was the removal of previously-deprecated behaviour, but as a consequence we also switched the way that editable installs work when the wheel project isn't installed, from using the old setup.py develop approach to the newer PEP 660 mechanism.

With that background, the problem you are seeing is that setuptools uses a different method for editable installs under PEP 660, which has different semantics. Specifically, it is a lot more precise over what is exposed to the import mechanism. Your project depends on the examples directory, which is not part of your project as specified in setup.py being exposed to the import mechanism in an editable install. This is not correct behaviour and is in fact a bug in the legacy setuptools behaviour. What you are seeing is the result of pip/setuptools now using a code path that fixes that bug.

To get the old behaviour temporarily, you could get your users to install wheel before installing your project in editable mode. But that is only a short-term fix. In the longer term, you will need to implement a way for your users to get access to your examples directory that does not require relying on the setuptools editable install bug.

[wesk]

To get the old behaviour temporarily, you could get your users to install wheel before installing your project in editable mo0de.

So an example short-term fix would be adding the pip install wheel line like so:

python -m pip install wheel
python -m pip install --upgrade pip
python -m pip install -e '.[extra]'

In the longer term, you will need to implement a way for your users to get access to your examples directory that does not require relying on the setuptools editable install bug.

What would that longer-term fix look like -- is it possible to do with a setup.py or should we now cut over and upgrade to start using a pyproject.toml? The editable mode installation we're using of pip install -e . has the nice property of letting us develop and test additional code that we do not vend via the regular pip install syne-tune path, but is just available for customers who check out the repo directly. Are there any references/docs that you can point us (and future readers) to, for how to achieve this goal in the most forward-looking way?

[pfmoore]

Short-term, yes, I believe so. You should obviously check this.

Long term, I don't know. This is effectively a setuptools change in behaviour, so you would be best explaining your use case to them and asking for advice. I don't know if they provide any advice or documentation on how to transition from "traditional" editable installs to PEP 660-based ones. I imagine they would say that you should either include the examples in your package, or provide the examples as a separate project that the users can install normally. Or maybe just don't use editable installs and manually modify sys.path to put the examples on the import path.

But this isn't really a pip issue, so that's just my best guess.

[pfmoore]

Just for reference, the change in behaviour was caused by this change (from the changelog):

When the wheel package is not installed, pip now uses the default build backend instead of setup.py install for project without pyproject.toml. (#8559)

[luciansmith]

Just to note that the same thing happened to us: 23.1 came out, and now none of our github actions work. Reverting to 23.0.1 works for now.

[uranusjr]

Is there anything for pip to act on here? We are not going back to setup.py install, the error can be worked around by pip install wheel, and the underlying issue (PEP 517 backend not acting the same as setup.py install and setup.py bdist_wheel) needs to be reported to and worked on by setuptools maintainers instead.

[pfmoore]

No. The OP has raised a setuptools issue for this, which is where any further discussion can happen.