Deprecation of distutils - slated for Python 3.12

[Athanasius]

I've been noticing this... probably since I first started using Python 3.10:

0$ python setup.py py2exe
C:\Users\Athan\Documents\Devel\infi-systray_py2exe\setup.py:1: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
  from distutils.core import setup
running py2exe

Do you have any plans for migrating py2exe itself away from using distutils ?

I have a vague memory of you deprecating use of build_exe.exe, but even if that's correct then it's still installed along with py2exe and will output a setup.py that uses distutils:

Athan@emilia MINGW64 ~/Documents/Devel/infi-systray_py2exe (main)
0$ build_exe.exe -W setup-build_exe.py sample_systray.py
Created setup-build_exe.py.
Athan@emilia MINGW64 ~/Documents/Devel/infi-systray_py2exe (main)
0$ head setup-build_exe.py
#!/usr/bin/python3
# -*- coding: utf-8 -*-
# Created by: python.exe -m py2exe -W setup-build_exe.py sample_systray.py

from distutils.core import setup
...

Obivously there's currently plenty of time. Python 3.12 will likely be available around October 2023, so still 19 months to go yet. But it would still be better to address this sooner rather than later.

[albertosottile]

I am well aware of this issue and I have started working on this in the background. As you correctly mentioned, this is a deal breaker for py2exe, but we also have plenty of time to react.

[effigies]

Hi @albertosottile, just a note that Versioneer hooks into py2exe like so:

    if 'py2exe' in sys.modules:  # py2exe enabled?
        from py2exe.distutils_buildexe import py2exe as _py2exe

        class cmd_py2exe(_py2exe):
            def run(self):
                root = get_root()
                cfg = get_config_from_root(root)
                versions = get_versions()
                target_versionfile = cfg.versionfile_source
                print("UPDATING %s" % target_versionfile)
                write_to_version_file(target_versionfile, versions)

                _py2exe.run(self)
                os.unlink(target_versionfile)
                with open(cfg.versionfile_source, "w") as f:
                    LONG = LONG_VERSION_PY[cfg.VCS]
                    f.write(LONG %
                            {"DOLLAR": "$",
                             "STYLE": cfg.style,
                             "TAG_PREFIX": cfg.tag_prefix,
                             "PARENTDIR_PREFIX": cfg.parentdir_prefix,
                             "VERSIONFILE_SOURCE": cfg.versionfile_source,
                             })
        cmds["py2exe"] = cmd_py2exe

I'm following this thread, but please feel free to ping me if any changes related to moving away from distutils are going to affect us.

[albertosottile]

Interesting, I was not aware of this. It is too early for me to know exactly how this will change, but I could imagine that your interface will be affected. I will report here progresses on this topic, once I start working on it.

[albertosottile]

The setuptools branch contains code that does not explicitly depends on setuptools (following PEP 632) and still builds and passes all the CI tests.

I would say this serves as an initial implementation idea on how I plan to overcome the deprecation. Further things that I would like to do before actually merging this are:

• remove all the non-functional files inherited from the old repo
• remove the COM server/services code, targets, and interpreters for good
• explicitly remove support for zipfile=None

TIL that

Directly calling python setup.py ... is considered a deprecated practice. You should not add new commands to setuptools expecting them to be run via this interface.

(source: last line here)

This actually poses a serious threat to the current py2exe code base that I am currently unsure how to tackle.

@effigies If you inspect the branch, you might notice that the interface for your use case should be relatively unchanged, with the exception that now you have to import from py2exe.setuptools_buildexe.

[Athanasius]

The setuptools branch contains code that does not explicitly depends on setuptools (following PEP 632) and still builds and passes all the CI tests.

I finally thought to try building that setuptools branch and using the resultant py2exe .whl.

To be sure, was this the correct way to build?

1. git clone py2exe and checkout setuptools branch.
2. python setup.py bdist_wheel (thankfully I had Visual Studio 2019 of some flavour already installed, and it was found).
3. Install the resultant dist/py2exe-0.11.1.1-cp310-cp310-win32.whl (with --force, because I already have the release of that version installed).
4. Run my usual python setup.py py2exe.

Result was:

0$ python setup.py py2exe
C:\Users\Athan\Documents\Devel\EDMarketConnector\venv\lib\site-packages\setuptools\_distutils\dist.py:262: UserWarning: Unknown distribution option: 'windows'
  warnings.warn(msg)
C:\Users\Athan\Documents\Devel\EDMarketConnector\venv\lib\site-packages\setuptools\_distutils\dist.py:262: UserWarning: Unknown distribution option: 'console'
  warnings.warn(msg)
C:\Users\Athan\Documents\Devel\EDMarketConnector\venv\lib\site-packages\setuptools\dist.py:530: UserWarning: Normalizing '5.4.2-alpha0+b78a2742' to '5.4.2a0+b78a2742'
  warnings.warn(tmpl.format(**locals()))
Git short hash: b78a2742
running py2exe
Traceback (most recent call last):
  File "C:\Users\Athan\Documents\Devel\EDMarketConnector\setup.py", line 241, in <module>
    setup(
  File "C:\Users\Athan\Documents\Devel\EDMarketConnector\venv\lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
    return run_commands(dist)
  File "C:\Users\Athan\Documents\Devel\EDMarketConnector\venv\lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
    dist.run_commands()
  File "C:\Users\Athan\Documents\Devel\EDMarketConnector\venv\lib\site-packages\setuptools\_distutils\dist.py", line 973, in run_commands
    self.run_command(cmd)
  File "C:\Users\Athan\Documents\Devel\EDMarketConnector\venv\lib\site-packages\setuptools\dist.py", line 1217, in run_command
    super().run_command(command)
  File "C:\Users\Athan\Documents\Devel\EDMarketConnector\venv\lib\site-packages\setuptools\_distutils\dist.py", line 992, in run_command
    cmd_obj.run()
  File "C:\Users\Athan\Documents\Devel\EDMarketConnector\venv\lib\site-packages\py2exe\setuptools_buildexe.py", line 151, in run
    self._run()
  File "C:\Users\Athan\Documents\Devel\EDMarketConnector\venv\lib\site-packages\py2exe\setuptools_buildexe.py", line 169, in _run
    dist.console = runtime.fixup_targets(dist.console, "script")
AttributeError: 'Distribution' object has no attribute 'console'

So, presumably I need to adjust our setup.py now to fit the new paradigm. Any clues as to location of documentation for that appreciated.

[Athanasius]

Aha! I had a look through your tests, and found the simple change of:

from distutils.core import setup

to:

from py2exe import setup

That's now built EDMarketConnector properly, and the .exe appears to be working.

[albertosottile]

I finally thought to try building that setuptools branch and using the resultant py2exe .whl.

I apologize, I thought I had provided a link to the wheels, but it turns out I forgot to do that. Those were in the Actions artifacts, as usual, here: https://github.com/py2exe/py2exe/actions/runs/2700339964

Aha! I had a look through your tests, and found the simple change of:

I am glad you found it. Of course this interface change would have to be communicated clearly, most likely in the README. However, I am afraid it would not be the only one (see #127 (comment)).

That's now built EDMarketConnector properly, and the .exe appears to be working.

That's great news! Thank you for having tested this. I am taking some time to think about how to solve the "stop invoking setup.py" issue, before moving this forward. Up to now the likely alternatives are to:

• use still a Python file as recipe, with a CLI provided by py2exe, without directly calling the setuptools command
• use pyproject.toml to store the recipe, add a parser in py2exe

Those are both quite disruptive changes, but I am afraid I do not have any other choice. Would you have any preference?

[Athanasius]

Our setup.py does more than just invoke py2exe, but the rest of that could be moved to another script if needs be (it's an .msi build, including poking translations into that).

We have a pyroject.toml already, but currently just to set some options for tools, adding the py2exe bits into that so long as we can pass some customisation in is fine..

We currently pass in things like:

1. name of the 'script'
2. product_name
3. version (combination of our version string and git shorthash).
4. Our manifest for the .exe files into other_resources.

So, we'd at least need to be able to pass in some "dynamic" customisation to the invocation of py2exe.

Relevant parts of our setup.py -> https://github.com/EDCD/EDMarketConnector/blob/main/setup.py#L216-L271

It would be quite the retooling to have a script instead edit just the correct section of pyproject.toml to include the correct dynamic bits.

[albertosottile]

adding the py2exe bits into that so long as we can pass some customisation in is fine..

Of course, py2exe would have its own tool.py2exe table and subsequent sub-tables, if needed. Thus, no conflicts will occur with other tools that use the pyproject.toml file.

It would be quite the retooling to have a script instead edit just the correct section of pyproject.toml to include the correct dynamic bits.

So, do I interpret this right that you would prefer a solution based on the pyproject.toml file?

[Athanasius]

It would be quite the retooling to have a script instead edit just the correct section of pyproject.toml to include the correct dynamic bits.

So, do I interpret this right that you would prefer a solution based on the pyproject.toml file?

If it would make the dynamic parts of our py2exe setup more difficult, then no. The script option would make for less work converting that over.

[albertosottile]

Ok, but then I misunderstood your sentence entirely and so you would prefer a Python script to configure py2exe.

If it would make the dynamic parts of our py2exe setup more difficult, then no. The script option would make for less work converting that over

The thing is, I am not so sure that either of the approaches would be specifically better for you, as both of them will most likely require to decouple the py2exe-related part from your script and invoke a new command (e.g. py2exe recipe.py or py2exe build or py2exe.build(recipe='...'), irregardless of the solution chosen

Because of what setuptools is proposing, the single setup.py script would not be able to install your software, build wheels for your software, and build a distributable package with py2exe at the same time. Hence, I think the setup.py script will be used for stuff that setuptools intends to support, and a second, dedicated script would have to be done for py2exe. At that point, the customizations needed for creating the bundle can be integrated in this dedicated script. However, with this approach in mind, I honestly do not see a clear winner between "recipe in the script" vs "recipe in the pyproject file".

[Athanasius]

Except "recipe in the script" can easily be dynamic (e.g. importing current git HEAD short hash). Other than the script editing the pyproject.toml (possibly causing issues with preferred ordering of sections, and comments, depending on the tooling used to achieve this), I don't see how pyproject.toml would support this.

Or am I ignorant of a method for a .toml file to have dynamic "call out to another program" / "also import this other file" abilities here ?

[Athanasius]

Ah, I just found https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#dynamic-metadata ... I'll have a good look at that and see if I still forsee any problems with the pyproject.toml scenario.

[albertosottile]

Except "recipe in the script" can easily be dynamic

I see your problem now. Honestly, I have always perceived the py2exe recipe as something that is "static", as it should describe stuff like where the project files are, what are its dependencies, what are its data files, and so on. Up to now we had some dynamic parameters, like version, because of the interaction with distutils but, once that is removed, I do not think py2exe will still need that.

I see you found Dynamic Metadata, which could be a solution, but I think the best solution would be to avoid the need for py2exe to know any dynamic info on its own.

[Athanasius]

I somewhat prefer the dynamic way of things because then you only have to maintain the expression of that, rather than updating the same meta-data in many different places for a complete build.

It was only in the last year or so that I got around to making our WiX .wxs file be dynamically generated, in order to ensure it always picked up all of the necessary files, particularly when py2exe / a module's idea of what files were necessary, and where they were, changed.

To give another example, our 'appname' is both used in the py2exe build so that the value is in the resultant .exe's properties, but also within our code itself. The same goes for the version string. Currently we literally only define it in one place (config/__init__.py) and that's picked up by both the program and the build process, not only for inclusion in the .exe properties, but also to form part of the .msi file's name.

It is sounding like, whatever solution you decide on, we'll end up needing our wrapper script to create/change some file dynamically, whether that's a recipe.py or pyproject.toml. In that scenario I'd prefer the former, as then it's a file used only by py2exe, whereas pyproject.toml will always have non-py2exe/setuptools things in it that we want preserved both in terms of data and ordering/comments.

[effigies]

@effigies If you inspect the branch, you might notice that the interface for your use case should be relatively unchanged, with the exception that now you have to import from py2exe.setuptools_buildexe.

Thanks! I've opened a PR to smooth the transition at python-versioneer/python-versioneer#319.

I wonder if it makes sense to keep around a py2exe.distutils_buildexe that is just the following:

import warnings
warnings.warn("py2exe.distutils_buildexe is deprecated; please import from py2exe.setuptools_buildexe",
              DeprecationWarning)

from py2exe.setuptools_buildexe import *

It might help with people who don't upgrade everything in quite the right order.

[albertosottile]

@effigies I actually expect the vast majority of users will use the setup scripts and whatever interface py2exe will provide for those (for the moment, from py2exe import setup but that will have to change due to the deprecation of direct calling of setuptools commands).

[effigies]

@albertosottile My concern is for people with old versioneer. If they upgrade py2exe but not versioneer, then they'll get an ImportError when we call from py2exe.distutils_buildexe import py2exe as _py2exe.

[albertosottile]

@effigies I have been having second thoughts at all this, as it appears evident that setuptools wants to drop their setup.py CLI entirely (pypa/setuptools#3442). I see that versioneer actually just extends the setup.py py2exe command, which is going to disappear sooner rather than later. How do you intend to solve this in versioneer?

I am asking this because I am debating whether I should deprecate this interface now and drop it entirely in the next major version that removes distutils, instead of porting it to setuptools as the current branch is doing.

[effigies]

From the versioneer perspective, the commands sdist and build and build_ext will still exist and we will still need to patch them. So we'll still need to update the cmdclass argument of setup(), regardless of whether people call setup.py directly. As I have time, I'll look into whether we could provide a patched backend that wraps setuptools.build_meta to inject version information.

I don't really understand the path forward for py2exe. I can say that if your command calls out to sdist or build_py (respecting the cmdclass passed to setup()), then it should be the case that Versioneer will have already patched things for you, and we don't need to special-case py2exe anymore.

[albertosottile]

I don't really understand the path forward for py2exe.

Well, I am trying to draft one. Because we cannot rely on the setup.py CLI anymore, I honestly see no reason for continuing to extend a setuptools command. It complicates the code significantly and it does not provide any true advantage w.r.t. an independent API.

Therefore, I was now thinking of dropping the command extension entirely in favor of a new API that could look e.g. like this:

def freeze(console=[], windows=[], data_files=None, zipfile="library.zip", options={}): ...

and has to be invoked e.g. like this

from py2exe import freeze

freeze(console=[{ "script": "numpy_test.py"}],
      options={"py2exe": {
            "packages": ['numpy']}})

from any Python script.

Something like this would simplify both the codebase and the migration for users that currently rely on setup.py scripts. However, the deletion of distutils_buildexe will affect you and the other users that rely on further extending this interface. I believe adaptations on your side should still be feasible, but I would like your opinion before proceeding further.

[effigies]

I wonder if the cleanest thing would be for you to:

1. build sdist/wheel using whatever pep-517/518 backend the package specifies
2. build exe from this artifact

If that fits into your plans, then there's no real need for us to patch py2exe. People with a setuptools backend with versioneer hooks will give you an sdist/wheel with the dynamic bits already swapped out. And other build backends and version injection hacks would also be none of your problem.

So from a user perspective, I'd probably want a pyproject.toml table like:

[tool.py2exe]
console = [...]
...

And then the build just becomes:

python -m py2exe

[albertosottile]

This goes in the opposite direction of what @Athanasius suggested a few posts above, which is to not put freezing information in the pyproject.toml file but rather in a Python module, as it allows more dynamic control on such properties. However,

other build backends and version injection hacks would also be none of your problem.

I really like this and I have full intention of going towards this direction, as you may have noticed from the proposed freeze API and script example that I posted above.

Regarding this:

build exe from this artifact

I am afraid this would require a lot of effort and so, for the moment, I do not intend to do so nor to make py2exe a proper PEP 517 build backend. This might change later down the road, but I do not have the capacity for it right now and I am mainly focused on making py2exe survive all the distutils and setuptools upcoming changes.

I have prepared a branch with this new py2exe.freeze API concept, you may download wheels from here:
EDIT: updated link -> https://github.com/py2exe/py2exe/actions/runs/3002400251
if both of you could give it a try with your respective use cases, that would be great. Thank you again for your time.

@Athanasius
As a reminder, the setup scripts just need to be adapted as in

from py2exe import freeze

freeze(console=[...], options={})

and then called as a regular Python script, no py2exe command (python freeze.py). Existing setup.py scripts are still supported but deprecated with an explanatory warning message.

@effigies
Conversely, the usage of py2exe.distutils_buildexe should be dropped, as extending the setuptools CLI with a py2exe command. The build command (or other existing setuptools commands) can be extended with py2exe by making use of the API mentioned above.

The API implementation is defined in py2exe\__init__.py, in case you want to have more insight on what is actually called. Suggestions on how to extend/complement this API are more than welcome. Same as before, py2exe.distutils_buildexe will still be supported in the next major, but deprecated.

[Athanasius]

A basic test shows that freeze() branch as working. Note that what I've done is take a copy of our old setup.py and apply minimal edits to get it working with this py2exe branch:

1. Remove from distutils.core import setup
2. Add from py2exe import freeze
3. Change setup(...) to freeze(...)
4. Remove the following from this renamed freeze() call:
   1. name= - I'm not sure this really had a use
   2. version= - likewise
   3. py_modules = [] - We only added this as a part of compatibility with 'newer' setuptools - a55a92c0afba669b95a799b7730900778f684bf4

However...

1. It's not taking note of the options={ 'py2exe': { 'dist_dir': ... } setting that we have. It used plain dist, rather than the dist.win32 we specify in our old setup.py. TBH, it doesn't make much difference to us currently, as this is an artifact from when the original devleoper also built for macOS (using py2app), possibly from the same directory.
2. Comparing a setup.py/0.11.1.1 build to this freeze/0.12.0.0 build:

0$ diff -ur dist dist.win32
Binary files dist/EDMC.exe and dist.win32/EDMC.exe differ
Binary files dist/EDMarketConnector.exe and dist.win32/EDMarketConnector.exe differ
Only in dist.win32: _sqlite3.pyd
Binary files dist/library.zip and dist.win32/library.zip differ
Only in dist/plugins: __pycache__
Only in dist.win32: sqlite3.dll

So something has caused sqlite3 to not get included. That's in our options={'py2exe': { ... 'packages': ...} stanza. We do not have anything in our actual code that uses sqlite3, but we need it packaged up for third-party plugins to use, hence specifying it in packages. Now, I can see sqlite3 files in the dist/library.zip.

Then I remembered we have a plugintest plugin that attempts to make use of all the modules we want to ensure are there for third-parties, so tried that. This highlighted that we're also not getting our util/text.py packaged at all, despite util also being listed in packages. This is another piece of code that isn't in our core source, only being used by one of our own plugins (some of the core code is actually implemented as plugins if for no other reason than to ensure we keep the plugin API working), hence having to specify it in packages.

So, what should I be adding to the invocation in order to get more in the way of diagnostics as to why it chose to not include those packages ?

[albertosottile]

Thank you very much for having tested this.

It's not taking note of the options={ 'py2exe': { 'dist_dir': ... } setting that we have.

I realized this as well this morning and pushed a new build and edited the URL, but perhaps you still installed the old one. Could you try again with wheels from here: https://github.com/py2exe/py2exe/actions/runs/2998629292 ? Thanks

version= - likewise

This is actually the only "general" attribute from setup that was used by py2exe, somehow. I still have to decide how to include it (and the others) in the new API.

[Athanasius]

Thank you very much for having tested this.

It's not taking note of the options={ 'py2exe': { 'dist_dir': ... } setting that we have.

I realized this as well this morning and pushed a new build and edited the URL, but perhaps you still installed the old one. Could you try again with wheels from here: py2exe/py2exe/actions/runs/2998629292 ? Thanks

It showed it had been edited, and it shows the same URL up in that comment as this one, yet ... to be clear that URL includes a py2exe-0.12.0.0-cp310-cp310-win32.whl , which I downloaded using link there -> https://github.com/py2exe/py2exe/suites/8150435828/artifacts/353669909

This is still ignoring dist_dir.

[albertosottile]

Interesting... could you add 'verbose': True to your options['py2exe'] dictionary and paste here the build logs?

[Athanasius]

Interesting... could you add 'verbose': True to your options['py2exe'] dictionary and paste here the build logs?

That made no difference. Of course if somehow the rebuild didn't pick up that small patch to actually use the py2exe options then nothing in there will be used and that would explain both the issues I'm seeing and tthe lack of thise verbose flag working.

The py2exe-0.12.0.0-cp310-cp310-win32.whl file I downloaded has sha256sum c04fd599678377d54af67760f5cf291da62163d92f6305a4a170b4e9344d9d7d

Am I being blind and missing something ?

0$ pip install --force-reinstall ../../../Downloads/Devel/Python/py2exe/py2exe-0.12.0.0-cp310-cp310-win32.whl
Processing c:\users\athan\downloads\devel\python\py2exe\py2exe-0.12.0.0-cp310-cp310-win32.whl
Collecting cachetools
  Using cached cachetools-5.2.0-py3-none-any.whl (9.3 kB)
Collecting pefile
  Using cached pefile-2022.5.30-py3-none-any.whl
Collecting future
  Using cached future-0.18.2-py3-none-any.whl
Installing collected packages: future, cachetools, pefile, py2exe
  Attempting uninstall: future
    Found existing installation: future 0.18.2
    Uninstalling future-0.18.2:
      Successfully uninstalled future-0.18.2
  Attempting uninstall: cachetools
    Found existing installation: cachetools 5.2.0
    Uninstalling cachetools-5.2.0:
      Successfully uninstalled cachetools-5.2.0
  Attempting uninstall: pefile
    Found existing installation: pefile 2022.5.30
    Uninstalling pefile-2022.5.30:
      Successfully uninstalled pefile-2022.5.30
  Attempting uninstall: py2exe
    Found existing installation: py2exe 0.12.0.0
    Uninstalling py2exe-0.12.0.0:
      Successfully uninstalled py2exe-0.12.0.0
Successfully installed cachetools-5.2.0 future-0.18.2 pefile-2022.5.30 py2exe-0.12.0.0
0$ sha256sum ../../../Downloads/Devel/Python/py2exe/py2exe-0.12.0.0-cp310-cp310-win32.whl
c04fd599678377d54af67760f5cf291da62163d92f6305a4a170b4e9344d9d7d *../../../Downloads/Devel/Python/py2exe/py2exe-0.12.0.0-cp310-cp310-win32.whl

[Athanasius]

Wait, now I'm doubting that sys.platform is properly detecting I'm on win32. No, it is, just the order of output under 'git bash' is weird, so I didn't see the test print I put in.

[Athanasius]

OK, I'm not going mad, and I must have been using that build artifact.

I've gone and git pull'd your freeze_api branch, then edited py2exe/__init__.py so that:

 26     print('Checking for global py2exe options key...')
 27     if 'py2exe' in options:
 28         print('Setting options to found py2exe options')
 29         options = options['py2exe']
 30         print(f'Options are now:\n{options=}')

Using the resulting .whl from that I get:

Checking for global py2exe options key...
Setting options to found py2exe options
Options are now:
options={'verbose': True, 'dist_dir': 'dist.win32', 'optimize': 2, 'packages': ['asyncio', 'multiprocessing', 'sqlite3', 'util'], 'includes': ['dataclasses', 'shutil', 'timeout_session', 'zipfile'], 'excludes': ['distutils', '_markerlib', 'optparse', 'PIL', 'simplejson', 'unittest']}

And yet still:

Building 'dist\EDMC.exe'.
Building 'dist\EDMarketConnector.exe'.

[Athanasius]

Aha! This is because the code still has e.g. compress = getattr(options, "compressed", 0) when, due to options being a dictionary, not an object/class, it should be e.g. compress = options.get("compressed", 0). I did that change for dist_dir, and now that bit works.

[Athanasius]

And with that change for all those arguments my packages is now working as well. At least sqlite3.dll has made it into dist.win32.

[albertosottile]

Yeah I was just figuring that out, I used getattr instead of get, silly me... Will push a fix ASAP. Thanks for debugging that!

[albertosottile]

New wheels available here: https://github.com/py2exe/py2exe/actions/runs/3002400251 . Could you test the fix? Also, I would be glad if you could test the wheels against the former python setup.py py2exe interface, to confirm that still works for you and emits a DeprecationWarning. Thanks.

This endeavor showed me something interesting though: ALL the functional tests can work without any options dictionary which, considering that they involve complex packages such as numpy, matplotlib, pandas, ...,, tells me a lot about how good the new ModuleFinder is. But also, it tells me I should double check the tests that should fail without an explicit options dict (e.g. bundle_files).

[Athanasius]

Once I remembered the download is a .zip file that I need to extract the .whl from ....

1. It works with my new freeze-targeted script, the resultant .exe runs, and also works after I use the .msi the rest of our build process produces.
2. The old invocation method both spits out the deprecation warning and also still produces a working .exe.

[albertosottile]

Great news! I think this is the preferred solution to this issue for direct users of py2exe. Let's wait for @effigies ' feedback regarding "API users" (though, to be fair, py2exe never had a proper API...).

Once I remembered the download is a .zip file that I need to extract the .whl from ....

That constantly bothers me as well but... -> actions/upload-artifact#3 (comment)

[effigies]

Hi, sorry about the slowness. Should probably confess that I neither use Windows nor py2exe. I'm just trying not to break things for projects that do. Would it be possible to get a project set up with python setup.py py2exe and versioneer that can build on CI?

I've forked https://github.com/pypa/sampleproject to https://github.com/effigies/sample-py2exe-versioneer and will set up versioneer on it when I get a few minutes. Would either of you be able to guide me in setting up py2exe on a Windows CI runner?

[Athanasius]

I've forked pypa/sampleproject to effigies/sample-py2exe-versioneer and will set up versioneer on it when I get a few minutes. Would either of you be able to guide me in setting up py2exe on a Windows CI runner?

The workflow we use for EDMC is here: https://github.com/EDCD/EDMarketConnector/blob/main/.github/workflows/windows-build.yml - it also includes extra stuff, but has the necessary setup steps.

[albertosottile]

I see. You can give a look at what is done in the configuration of GitHub Actions of this repository:

py2exe/.github/workflows/CI.yml

Line 110 in 4301ec7

test:

.

I gave a look in how versioneer works, in general, and it seems to me the dropping of the setup.py CLI is also going to affect that project significantly...

[effigies]

Hi @albertosottile @Athanasius I'm really not figuring this out. I've tried to create a module that's versioned by versioneer and use py2exe to turn a script into an exe. If I include includes or packages in the py2exe options, it fails to build. If I don't, the script can't import the module.

https://github.com/effigies/sample-py2exe-versioneer/blob/main/setup.py

Hopefully I'm making an error that will be very obvious to you.

[albertosottile]

@effigies py2exe does not easily support the src/ and script/ separation out of the box, because it is not designed for that (as opposed to e.g. wheel). Incidentally, this is one of the reasons why piggy-backing setuptools without fully supporting is not a good idea...

In this case, I would suggest you to adapt your test project as it follows:

• in setup.py, adapt the includes option as in "includes": ["src.sample"]}
• adapt your src/sample/sample.py import statements as in:

try:
    import sample
except ImportError:
    from src import sample

I tried these changes and I can build the sample project, though when the frozen code runs I get this: Sample v0+unknown. Not sure if that is expected behavior or not.

Hope this helps, let me know if you have any further issues.

[effigies]

Thanks for the pointer! I think it's unlikely that people currently using versioneer+py2exe are doing it that way, if it's so painful, so I got rid of the src/ directory instead.

That worked for setup.py. And freeze.py builds but doesn't get version info.

On the other hand, if py2exe is looking in the Python path for modules to include, it seems like we should be able to piggy-back off of pip by pip installing the package into the current environment and calling freeze(). Now that doesn't work out-of-the-box because ./sample will take precedence over an installed sample module, which turns out to be the problem that src/ resolves. So if I restore src/, then I can do (effigies/sample-py2exe-versioneer#1):

$ pip install .
$ python build_exe.py

The two-step may not be entirely pleasing, but it does mean that py2exe does not have to do anything special for any build tools, as long as the packager can find some way to put their finalized module into their path. So I think we should probably just put a section in the versioneer docs, rather than hack in special code for py2exe.

[albertosottile]

I was not even able to get the version with setup.py, but I must have doing something wrong in my adaptations. Anyway, the whole point of this exercise was to prove that the freeze API is good enough for your needs, and it seems the answer is positive. Therefore, I think I will polish it a little more and release it as v0.12.0.0 in the upcoming days. As you know, the former setuptools command will be kept for this version, but marked as deprecated, giving you (and other developers) some time to adapt your code.

Thank you @effigies and @Athanasius for your effort!

[albertosottile]

Fixed in 0.12.0.0