[BUG] setuptools doesn't respect global distutils.cfg file for changing compiler

[eabase]

setuptools version

65.5.1

Python version

Python 3.10.0

OS

Win-10

Additional environment information

No response

Description

I have tried just about everything to try to make setuptools not use python setup.py ... to compile and install a C-built extension package.

As you have elegantly stated (now for several years) that one should not use python setup.py:

... running python setup.py directly as a script is considered deprecated. This also means that users should avoid running commands directly via python setup.py .

If you want to create sdist or wheel distributions the recommendation is to use the command line tool provided by build:

pip install build  # needs to be installed first
python -m build  # builds both sdist and wheel

However, because python -m build ignores the .../Lib/distutils/distutils.cfg file,
there is no known way to avoid using the good old:

python setup.py build --compiler=mingw32

Expected behavior

I expect to be able to build my package using any compiler of my choice, like MinGW, by specifying this in the python -m build command.

How to Reproduce

python -m build -n -w --config-setting="--compiler=mingw32"

Is ignored

Output

Always using MS Visual Studio VC....

[abravalheri]

Hi @eabase. At the moment, -C--build-option=... can only be used to pass arguments associated with the previous bdist_wheel command (not build).

If you wish to pass arguments associated with the previous build command that would be a feature request that needs to be discussed (and probably coordinated between the pypa/setuptools and pypa/wheel team).

Nevertheless, you mentioned in your title that setuptools doesn't respect global distutils.cfg file.

This is probably because python -m build will create a brand new virtual environment with a brand new version of setuptools/distutils installed in it.

Have you tried using %USERPROFILE%/pydistutils.cfg?
You can also try setting the environment variable DIST_EXTRA_CONFIG to a custom path.

[eabase]

Hi @abravalheri

This is probably because python -m build will create a brand new virtual environment ...

No, that's not right because I already use the -n that disable a new venv creation.

Have you tried using %USERPROFILE%/pydistutils.cfg?

Nope, and I never will.
However, I moved my distutils.cfg from "%PYTHONROOT" into setup.cfg in my project folder, and now it works as it should.
(So much for migrating away from deprecated setup.py/cfg usage...)

You can also try setting the environment variable DIST_EXTRA_CONFIG to a custom path.

Can you provide some examples for doing this?
(This should really be printed clearly somewhere in the help part of the tools!)

Conclusion

I have no idea why the python build tools just doesn't include a --compiler=whatever flag.
It would help a lot of people from a lot of places....

[abravalheri]

No, that's not right because I already use the -n that disable a new venv creation.

Sorry, I missed that -n... In this case you can try modifying site-packages/_distutils/distutils.cfg (setuptools has moved away from using stdlib's distutils.cfg) or you can also set SETUPTOOLS_USE_DISTUTILS=stdlib (env var) to force setuptools to use the version of distutils distributed with the standard library.

(The "own copy of distutils" behaviour is documented in https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html).

So much for migrating away from deprecated setup.py/cfg usage...

Please note that having setup.py and setup.cfg as configuration files is not deprecated. What is considered deprecated is running setup.py directly as a CLI tool.

Can you provide some examples for doing this?

You can try $env:DIST_EXTRA_CONFIG = "C:\path\to\a\distutils.cfg" on a powershell session just before running your build. This is part of the latest version of distutils, which is distributed as setuptools/_distutils (in the latest version of setuptools).

This should really be printed clearly somewhere in the help part of the tools!

This is part of distutils functionality and unfortunately after being kicked out of the Python core, the documentation for distutils does not have a clear home yet (if you are interested please feel free to engage in the documentation efforts in the community; setuptools has interest in covering some of it, so please feel free to submit a PR).

Meanwhile you can have a look on:

https://setuptools.pypa.io/en/latest/deprecated/distutils/configfile.html#writing-the-setup-configuration-file

I have no idea why the python build tools just doesn't include a --compiler=whatever flag.
It would help a lot of people from a lot of places....

Setuptools is part of a community effort, not a commercial product or service. If you or other members of the community have a strong interest about a specific feature that is not implemented yet, the best way of making that reality is proposing an implementation plan and (after the plan has been discussed and agreed) submit a PR (or a series of PRs) with the implementation.

[eabase]

Hello @abravalheri

Thank you so much for taking the time to explain this in more detail and clarifying what's actually deprecated. 💯

It also explains why moving my distutils.cfg to setup.cfg, worked so well. I also understand that keeping up the documentation is a huge effort, but also crucial to minimize the issues people face. In this era, python has now grown very mature and so maintaining the historical and enormous amount of documentation is hard. What is often missing are good practical (and complete examples) for may use cases. For example, and as in my case, compiling C-modules using your own compiler.

Please note that having setup.py and setup.cfg as configuration files is not deprecated. What is considered deprecated is running setup.py directly as a CLI tool.

This is in fact one of the confusion points. Stating on the one hand that the use of a file is deprecated, while it's presence is not... ❗

Thanks again. ✨

[abravalheri]

Thanks @eabase.

I added a few notes to clarify this behaviour in #3689.

This is in fact one of the confusion points. Stating on the one hand that the use of a file is deprecated, while it's presence is not...

At least this part we have covered in our docs 😄.
Please feel free to submit a PR to improve the docs, they are always very welcome!

[eabase]

@abravalheri
I think it should be made more clear. Even looking in the files changed in the PR (above), I do not see any reference to selecting a compiler. Perhaps add something in the build rst file, like:

---

4 ways to use a different Compiler (on Windows)

You want to specify the compiler in a config file. For example, to use the MinGW64 compiler, you need:

# setup.cfg
#----------------------------------------------
# This is a customized (global) distutils.cfg
# https://docs.python.org/3/install/index.html
#----------------------------------------------
# other locations (windows):
#   local       setup.cfg
#   system      prefix\Lib\distutils\distutils.cfg
#   personal    %HOME%\pydistutils.cfg
#----------------------------------------------

[global]
verbose=1

[build]
compiler=mingw32
force=1

[build_ext]
compiler=mingw32
force=1

1. Set the legacy distutils flag with: $env:SETUPTOOLS_USE_DISTUTILS=stdlib. This will use the disutils.cfg within the python disutils installation directory. (Usually in "%PYTHONROOT" under .../Lib/distutils/distutils.cfg.)

2. Use the (local) user specific configuration file: %USERPROFILE%/pydistutils.cfg

3. Either copy ../site-packages/_distutils/distutils.cfg to your project directory and rename it to setup.cfg,
   or create your own with the content shown above. ✅ The preferred method!

4. Use any *.cfg file (from anywhere?) with: $env:DIST_EXTRA_CONFIG = "C:\path\to\a\distutils.cfg".

---

Maybe even 5 with the following (not yet tested):

python -m build -C--global-option=--no-user-cfg -C--global-option=--verbose -C--build-option="build --compiler=mingw32 --force"