[BUG] stderr duplication failed

[rayzchen]

setuptools version

setuptools==56.0.0

Python version

Python 3.9

OS

Windows

Additional environment information

No response

Description

I use gui_scripts with a blank function. If i use pip install -e . or py setup.py install nothing happens, as expected. If i do pip install . or use setup.py to make a binary dist then install it, i get stderr duplication failed in a prompt. This happens before any code gets run.

Expected behavior

Nothing

How to Reproduce

# setup.py
from setuptools import setup

setup(
    name="test",
    packages=["test"],
    entry_points={
        "gui_scripts": [
            "test-script=test:main"
        ]
    }
)

# test/__init__.py
def main():
    pass

Run above commands

Output

Code of Conduct

• I agree to follow the PSF Code of Conduct

[jaraco]

The reason you're seeing this error only in the selected environments (non-editable install) is because only editable installs use Setuptools' launcher scripts. The error you're getting is from the launcher scripts installed by pip and distlib. You'll want to report the issue to the distlib project and see that pip inherits the fix. For that reason, I'll move this issue to pip and I recommend you to file an issue with distlib.

[DiddiLeija]

@rayzchen Is this still happening? Could you provide some information of the pip version you are using?

[rayzchen]

It still happens with pip version 21.2.4

[uranusjr]

cc @vsajip since the launchers are from distlib.

[vsajip]

I can't reproduce this - it might be something specific in @rayzchen 's environment. I'm using Python 3.9.7 on Windows 10.

C:\Users\Vinay\Projects\scratch\distlib\pip_10444> type setup.py                                                                   
from setuptools import setup                                                                                                       
                                                                                                                                   
setup(                                                                                                                             
    name="test",                                                                                                                   
    packages=["test"],                                                                                                             
    entry_points={                                                                                                                 
        "gui_scripts": [                                                                                                           
            "test-script=test:main"                                                                                                
        ]                                                                                                                          
    }                                                                                                                              
)                                                                                                                                  
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> type test\__init__.py                                                           
def main():                                                                                                                        
    pass                                                                                                                           
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> python3 -m venv env                                                             
                                                                                                                                   
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> env\Scripts\python -m pip install -U pip wheel                                  
Requirement already satisfied: pip in c:\users\vinay\projects\scratch\distlib\pip_10444\env\lib\site-packages (21.2.3)             
Collecting pip                                                                                                                     
  Using cached pip-21.2.4-py3-none-any.whl (1.6 MB)                                                                                
Collecting wheel                                                                                                                   
  Using cached wheel-0.37.0-py2.py3-none-any.whl (35 kB)                                                                           
Installing collected packages: wheel, pip                                                                                          
  Attempting uninstall: pip                                                                                                        
    Found existing installation: pip 21.2.3                                                                                        
    Uninstalling pip-21.2.3:                                                                                                       
      Successfully uninstalled pip-21.2.3                                                                                          
Successfully installed pip-21.2.4 wheel-0.37.0                                                                                     
                                                                                                                                   
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> env\Scripts\pip install .                                                       
Processing c:\users\vinay\projects\scratch\distlib\pip_10444                                                                       
  DEPRECATION: A future pip version will change local packages to be built in-place without first copying to a temporary directory.
 We recommend you use --use-feature=in-tree-build to test your packages with this new behavior before it becomes the default.      
   pip 21.3 will remove support for this functionality. You can find discussion regarding this at https://github.com/pypa/pip/issue
s/7555.                                                                                                                            
Building wheels for collected packages: test                                                                                       
  Building wheel for test (setup.py) ... done                                                                                      
  Created wheel for test: filename=test-0.0.0-py3-none-any.whl size=1357 sha256=59ca865995d18af94cae5d4bf40bc7643ffa08ff8fdcd29e10b
9a8c940967239                                                                                                                      
  Stored in directory: C:\Users\Vinay\AppData\Local\Temp\pip-ephem-wheel-cache-fir7qy3v\wheels\6f\a7\45\f9fb5d415c64f56b92837eabf4b
edd182318b9abca73407cd5                                                                                                            
Successfully built test                                                                                                            
Installing collected packages: test                                                                                                
Successfully installed test-0.0.0                                                                                                  
                                                                                                                                   
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> dir env\Scripts\                                                                
 Volume in drive C has no label.                                                                                                   
 Volume Serial Number is 26B4-06F0                                                                                                 
                                                                                                                                   
 Directory of C:\Users\Vinay\Projects\scratch\distlib\pip_10444\env\Scripts                                                        
                                                                                                                                   
25/09/2021  06:49    <DIR>          .                                                                                              
25/09/2021  06:49    <DIR>          ..                                                                                             
25/09/2021  06:48             1,997 activate                                                                                       
25/09/2021  06:48               989 activate.bat                                                                                   
25/09/2021  06:48            19,408 Activate.ps1                                                                                   
25/09/2021  06:48               368 deactivate.bat                                                                                 
25/09/2021  06:49           106,383 pip.exe                                                                                        
25/09/2021  06:49           106,383 pip3.9.exe                                                                                     
25/09/2021  06:49           106,383 pip3.exe                                                                                       
25/09/2021  06:48           543,464 python.exe                                                                                     
25/09/2021  06:48           542,440 pythonw.exe                                                                                    
25/09/2021  06:49           100,222 test-script.exe                                                                                
25/09/2021  06:48           106,370 wheel.exe                                                                                      
              11 File(s)      1,634,407 bytes                                                                                      
               2 Dir(s)  11,182,489,600 bytes free                                                                                 
                                                                                                                                   
C:\Users\Vinay\Projects\scratch\distlib\pip_10444> env\Scripts\test-script.exe                                                     
                                                                                                                                   
C:\Users\Vinay\Projects\scratch\distlib\pip_10444>                                                                                 

And no error dialog box comes up. If I additionally install pywin32, change the test application to the following:

# setup.py
from setuptools import setup

setup(
    name="test",
    packages=["test"],
    entry_points={
        "gui_scripts": [
            "test-script=test:main",
            "hello=test:hello"
        ]
    }
)

# test\__init__.py
def main():
    pass
    
def hello():
    import win32api, win32con
    win32api.MessageBox (0, "Test application says 'Hello, world!'", 'Hello, world!', win32con.MB_OK)
    pass

and reinstall using env\Scripts\pip install . followed by running env\Scripts\hello, I get the expected message box:

[rayzchen]

I tried the exact same thing that you did and I still got the error. What does the error actually mean? I'm using python 3.9.4, and I upgraded setuptools to 57.0.0.

[rayzchen]

sys.version is 3.9.4 (tags/v3.9.4:1f2e308, Apr 6 2021, 13:22:44) [MSC v.1928 32 bit (Intel)]

[vsajip]

What does the error actually mean?

It means that the Windows DuplicateHandle API failed (returned an error) for the stderr process stream, and the launcher bails out because that happened. The launcher calls code to duplicate handles for stdin, stdout and stderr in that order; I don't know why the duplication of stderr is failing for you where stdin and stdout apparently succeed just before that. According to the linked Windows documentation,

The handle must have the PROCESS_DUP_HANDLE access right.

I have no way of telling if that's the case in your environment, of course. What Windows version is it where the failure occurs? That might be relevant.

[rayzchen]

if this helps

[vsajip]

Can you please confirm that it's running on a 64-bit processor? Can you please try with a 64-bit Python? Your version string seems to indicate a 32-bit Python.

[vsajip]

@rayzchen I used the script in this Gist to see if I could duplicate handles on my system using ctypes. When I ran it, I got

1
1
1

Indicating that stdin, stdout and stderr were successfully duplicated. You might want to try it, but make sure you're OK with what it does first.

[no-response]

This issue has been automatically closed because there has been no response to our request for more information from the original author. With only the information that is currently in the issue, we don't have enough information to take action. Please reach out if you have or find the answers we need so that we can investigate further.

[jeremyd2019]

This was reported at msys2/MSYS2-packages#2694. Your handles.py script returns 1 1 1 when run via python, but if I modify the script to use MessageBoxW instead of print, and run via pythonw instead, I got 0 0 0.

Outputting the handle values, input and output are 0, and only error is an actual handle.

def MessageBox(message):
    return windll.user32.MessageBoxW(0, str(message), "Your title", 1)

The last error from DuplicateHandle is 6 (ERROR_INVALID_HANDLE) for in and out, and 50 (ERROR_NOT_SUPPORTED) for error.

[jeremyd2019]

When run from cmd.exe, all 3 handles are 0, and all 3 DuplicateHandle calls error with ERROR_INVALID_HANDLE)

[vsajip]

Not sure exactly what you mean by "run from cmd.exe" - I originally ran the handles.py from a cmd.exe window to get the 1 1 1 result. It's possible that a windowed application like pythonw doesn't have these handles because it doesn't do terminal I/O. It's not clear why only stderr would be affected.

[jeremyd2019]

Sorry. The report to MSYS2 was that they got this error from a GUI launcher when launched from Git for Windows Bash but not when launched from cmd or powershell. I was testing using MSYS2 (Git for Windows uses a fork of MSYS2, and in turn MSYS2 is a fork of Cygwin).

[rayzchen]

So you're saying it only happens on MSYS and consequently on git bash?

[vsajip]

Any reason you can't download the built launchers in the distlib repo

https://github.com/pypa/distlib/raw/master/distlib/{t32,t64,w32,w64}.exe

and test with those? Just to ensure that what gets into the next distlib release is fit for purpose.

[carlkl]

I forgot that the binaries are contained in the repo - thanks.

[cbrnr]

I confirm that the mnelab launcher (a GUI app) works with the latest launchers.

[rayzchen]

If this helps, the error happens with pythonw when running any script. However a fresh install of Python 3.10.4 x64 seems to work fine. Is it because I'm using 32-bit 3.9? I'm not too keen on migrating everything from 3.9 to 3.10.

[vsajip]

@rayzchen Just to be clear - are you saying this error still occurs with the latest launchers patched into pip? Please list the exact steps you took to run the tests in the failure case, starting from the launchers in the latest distlib repo.

[rayzchen]

It's my current setup which shouldn't have been modified since I opened this issue, which I guess isn't much help anymore. How do I get the new launcher?

[vsajip]

You clone the distlib repository and look for the .exe files in the distlib directory under the project root.

[carlkl]

I tested the latest pip together with the updated binaries from the latest distlib. I can confirm this works with cli with both; msys2 as well as with the Windows cmd prompt.

[vsajip]

Thanks @cbrnr and @carlkl for redoing the tests. Looking promising 🤞

[uranusjr]

Not sure if we already upgraded distlib in 22.1, adding this to the milestone just to be sure.

[pradyunsg]

The distlib changes haven't been released. #11063 did not update to distlib 0.3.4 since that was the problematic version. Once there's a new distlib release containing the fixed launchers, we can update the vendored distlib. :)

[vsajip]

Just waiting for a response from @rayzchen (and anyone else) and then I should be able to cut a release soon after that.

[vsajip]

Didn't hear from @rayzchen, but distlib 0.3.5 has just been released on PyPI.

[pradyunsg]

@sbidoul FYI: when you do the vendoring update, this issue should be closed.

[rayzchen]

Sorry for the late reply, but which one do I download, w64.exe or t64.exe?

[rayzchen]

I tried both but I get Fatal error in launcher: Unable to find an appended archive. with both

[vsajip]

I tried both but I get Fatal error in launcher: Unable to find an appended archive. with both

You can't just run the launchers, if that's what you tried. What did you actually do with the launchers?

[rayzchen]

Double click it, I guess?

[sbidoul]

@sbidoul FYI: when you do the vendoring update, this issue should be closed.

Ok. I'll be glad if anyone is able to provide a changelog entry for this issue, though.

[vsajip]

Double click it, I guess?

No, that just runs the launcher by itself, which would fail in the way you saw. What needs to be done is for the executables from the latest distlib version to be copied over the vendored ones in pip, and then use that pip to install a package with scripts, and then run one of those scripts from wherever they were installed to ...

[rayzchen]

Ah, got it. The new executables work now, I don't get the error.

Just out of curiosity, how do I use the executables to create scripts? Is it something like running

copy w32.exe + some_file.pyz out.exe

like using 7z.sfx?

[vsajip]

Well, the easiest way is to use ScriptMaker from distlib. What you've said is more or less right, but you have to factor in the shebang, too. Otherwise the launcher can't tell which interpreter is to run the archive.