[BUG] setuptools auto discovery in 61.0 breaks py2app

[ronaldoussoren]

setuptools version

setuptools == 61.0

Python version

Python 3.10

OS

macOS

Additional environment information

This only happens when using py2app (https://pypi.org/project/py2app).

Description

py2app is a setuptools plugin command that basically uses distutils as a command runner with a similar interface as py2exe.

In general the setup() call for py2app does not contain arguments specifying the distribution name or modules/packages, the files acted upon are specified in py2app-specific arguments. The distribution name is optimally used as the name of the generated application, and by default calculates the name from the name of main script in the application.

With setuptools 61.0 release I see two common failures:

1. An application bundle is generated with the wrong name, due to the auto discovery feature calculating a name for the distribution, which overrides the default behaviour for py2app.
2. Users get an error when the directory containing setup.py contains multiple python files, due to the auto discovery assuming the directory contains a flat layout which shouldn't contains multiple toplevel names:

error: Multiple top-level modules discovered in a flat-layout: ['hello', 't'].

To avoid accidental inclusion of unwanted files or directories,
setuptools will not proceed with this build.

Expected behavior

Both behaviours are new and break py2app.

Please add an easy way to disable this feature, preferably a way in which a setuptools plugin command can disable auto discovery when it is used.

How to Reproduce

1. Be on a macOS system
2. Clone https://github.com/ronaldoussoren/py2app
3. "pip install -e py2app"
4. Go to "py2app/examples/Tkinter/hello_tk"
5. Add a file "world.py" (can be empty)
6. Run "python setup.py py2app"

This results in a working application bundle with setuptools 60.0, and breaks with the error below when using setuptools 61.0.

Output

error: Multiple top-level modules discovered in a flat-layout: ['hello', 'world'].

To avoid accidental inclusion of unwanted files or directories,
setuptools will not proceed with this build.

If you are trying to create a single distribution with multiple modules
on purpose, you should not rely on automatic discovery.
Instead, consider the following options:

1. set up custom discovery (`find` directive with `include` or `exclude`)
2. use a `src-layout`
3. explicitly set `py_modules` or `packages` with a list of names

To find more information, look for "package discovery" on setuptools docs.

[abravalheri]

Hi @ronaldoussoren, sorry to hear that you are facing problems with the auto-discovery.

Currently, you can disable auto-discovery by implementing a setuptools.finalize_distribution_options entry-point and then explicitly setting name + either packages or py_modules in the distribution object passed as argument to the hook.

If you already have distutils.setup_keywords in place, I suppose you can also do that in this function.

[abravalheri]

I think I can also add a environment variable to disable auto-discovery if that helps.

There are some aspects of the auto-discovery that are triggered before the command is called, so I don't think it is possible right now to have a per-command mechanism to disable it.

[ronaldoussoren]

finalize_distribution_options might work for me, but I'd have to be very careful because that entry-point will be called in every call to setuptools.setup() once py2app is installed and I'm not sure yet if I can cleanly detect if finalize_options is called for a setup() call that will later invoke the py2app command.

AFAIK distutils.setup_keywords won't work, because those extensions are only called if the named option is present in the call to setup().

[abravalheri]

@ronaldoussoren, if I understood the docs correctly, the setup(...) has to be called with the correct app= keyword parameter to be used by py2app, right?

If that is the case, finalize_distribution_options could check if app is set and then add default values for both name= and packages=/py_modules=.

Similarly, if you have a distutils.setup_keywords for app, you could use the same hook for the other options, couldn't you?

[ronaldoussoren]

Looking for the app and plugin attributes works for me.

I've added a setuptools.finalize_distribution_options entry point to py2app and that fixes the issue for me.

[abravalheri]

Thank you very much @ronaldoussoren .

[ronaldoussoren]

One problem with having a finalize_distribution_options entry point is that this makes it harder to build sdists and wheels, I had to install the install_requires of py2app manually before being able to do so. Adding setup_requires to the setup.py file did not help here.

Not a big problem, but worth noting.

Thanks for your help!

[abravalheri]

I don't now if I understand correctly, but for wheels and sdists specifically can you rely on pyproject.toml [build-system] requires (PEP 518)?

Or is it a bootstrapping issue? Using py2app to build itself?

[ronaldoussoren]

I don not yet use pyproject.toml in py2app, partially because the current release still minimally supports Python 2.7. That will change after the upcoming release.