Strange side-effect on help()

[lschoe]

Hi, here's a thing I noticed a while ago when using the Python REPL in combination with winloop.

The problem is that certain calls to the built-in help() function don't work anymore, something to do with a redirection of stdout maybe.

import asyncio
from winloop import EventLoopPolicy
asyncio.set_event_loop_policy(EventLoopPolicy())  # 1st critical line for the problem to occur

print('1' * 80)    
help(pow)  # prints just fine
print('1' * 80)

asyncio.get_event_loop()  # 2nd critical line for the problem to occur

print('2' * 80)    
help(pow)  # only prints a blank line
print('2' * 80)

Running this code will produce:

11111111111111111111111111111111111111111111111111111111111111111111111111111111
Help on built-in function pow in module builtins:

pow(base, exp, mod=None)
    Equivalent to base**exp with 2 arguments or base**exp % mod with 3 arguments

    Some types, such as ints, are able to use a more efficient algorithm when
    invoked using the three argument form.

11111111111111111111111111111111111111111111111111111111111111111111111111111111
22222222222222222222222222222222222222222222222222222222222222222222222222222222

22222222222222222222222222222222222222222222222222222222222222222222222222222222

By omitting either (or both) critical lines, the problem disappears, and the help(pow) message will be printed in twice.

The actual use case for this problem is when I run the Python REPL using winloop (in combination with MPyC), which looks like this:

>python -m mpyc
asyncio REPL 3.11.8 (tags/v3.11.8:db85d51, Feb  6 2024, 22:03:32) [MSC v.1937 64 bit (AMD64)] on win32
Use "await" directly instead of "asyncio.run()".
Type "help", "copyright", "credits" or "license" for more information.
>>> import asyncio
>>> from mpyc.runtime import mpc
>>> secint = mpc.SecInt()
>>> secfxp = mpc.SecFxp()
>>> secflt = mpc.SecFlt()
>>> secfld127 = mpc.SecFld(127)
>>> secfld256 = mpc.SecFld(2**8)
>>> secsym = mpc.SecSymmetricGroup(5)
>>> secqr = mpc.SecQuadraticResidues(11)
>>> secsg = mpc.SecSchnorrGroup(l=1024)
>>> secec = mpc.SecEllipticCurve('Ed25519')
>>> seccl = mpc.SecClassGroup(-23)
>>> help(pow)

>>>

By disabling the use of winloop the problem disappears. Also, the problem does not occur on Linux using uvloop.

Do you have a clue about the cause? Something "bad" happens in the C(ython) code executed for the asyncio.get_event_loop() call.

[Vizonex]

@lschoe I have no idea on how this could happen. If I'm going to be absolutely honest with you. But this does seem to be very bad. A fix as well as debugging the problem would be a good idea but I do not know where we would start. There is a macro in the program you could try enabling that and compiling the project with debugging turned on and start tracking the bug.

[lschoe]

I've tracked the problem down to a call to os.system(). A call like help(pow) is handled by pydoc and on Windows ends up calling pydoc.tempfilepager(). At the end of that function the call os.system(cmd + ' "' + filename + '"') fails with exit code 1.

The same can be experienced in the examples above. For example, in the Python REPL instead of >>> help(pow) now do:

>>> import os; os.system('path')
1

Maybe this gives you a clue where to look?

[Vizonex]

On Wed, Apr 3, 2024 at 5:56 AM Berry Schoenmakers ***@***.***> wrote: I've tracked the problem down to a call to os.system(). A call like help(pow) is handled by pydoc and on Windows ends up calling pydoc.tempfilepager(). At the end of that function the call os.system(cmd + ' "' + filename + '"') fails with exit code 1. The same can be experienced in the examples above. For example, in the Python REPL instead of >>> help(pow) now do: >>> import os; os.system('path')1 Maybe this gives you a clue where to look? — Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3K7GGVNK4XWGED66KCBEWDY3PN6ZAVCNFSM6AAAAABFNFF5L2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZUGI2DONRVHE> . You are receiving this because you commented.Message ID: ***@***.***>

I’m a little busy at the moment but I will try to look at this when I have the time to.

[lschoe]

Hi @Vizonex, can you drop me an email at l.a.m.schoenmakers@tue.nl regarding the invitation?

[lschoe]

To investigate this issue, I've started to remove nonessential differences between (the current versions of) winloop and uvloop.

Removing these differences is also helpful to prepare for the ultimate thing: merging winloop into uvloop.

The latest commit b5946ea also suggests removing differences that are present for performance reasons. Leaving out "noexcept" seems to be no problem? Or is it essential to use "noexcept".

[Vizonex]

On Tue, May 14, 2024 at 10:43 AM Berry Schoenmakers < ***@***.***> wrote: To investigate this issue, I've started to remove nonessential differences between (the current versions of) winloop and uvloop. Removing these differences is also helpful to prepare for the ultimate thing: merging winloop into uvloop. The latest commit b5946ea <b5946ea> also suggests removing differences that are present for performance reasons. Leaving out "noexcept" seems to be no problem? Or is it essential to use "noexcept". — Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3K7GGR3ATXRBQDG2GVAMKDZCIWJVAVCNFSM6AAAAABFNFF5L2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMJQGU3DKMRSGU> . You are receiving this because you were mentioned.Message ID: ***@***.***>

I implemented noexcept for a performance enhancement since uvloop was doing it themselves.

[lschoe]

So, is it OK to remove these "noexcept"s throughout the code (but make a note of it) to unify the code with uvloop's code?

For example, removing "noexcept" from sslproto.pyx and sslproto.pxd makes these source files identical to their counterparts in uvloop.

[lschoe]

Hi @Vizonex , finally found the cause for this issue. I have made a PR, where the call to uv.uv_disable_stdio_inheritance() is commented out.

Can you reconsider the need for this call? Would be nice if it can be avoided.