macOS IDLE with Tk 8.6.10 in 3.9.1rc1 universal2 installer fails smoke test

[ned-deily]

BPO	42508
Nosy	@terryjreedy, @ronaldoussoren, @ned-deily, @serhiy-storchaka, @miss-islington
PRs	bpo-42508: Remove bogus idlelib.pyshell.ModifiedInterpreter attribute #23570 [3.9] bpo-42508: Remove bogus idlelib.pyshell.ModifiedInterpreter attribute (GH-23570) #23571 [3.8] bpo-42508: Remove bogus idlelib.pyshell.ModifiedInterpreter attribute (GH-23570) #23572 bpo-42508: Keep IDLE running on macOS #23577 [3.9] bpo-42508: Keep IDLE running on macOS (GH-23577) #23669 [3.8] bpo-42508: Keep IDLE running on macOS (GH-23577) #23670

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2021-01-05.08:10:47.837>
created_at = <Date 2020-11-30.03:17:54.166>
labels = ['OS-mac', 'expert-tkinter', '3.9', '3.10', '3.8', 'expert-IDLE']
title = 'macOS IDLE with Tk 8.6.10 in 3.9.1rc1 universal2 installer fails smoke test'
updated_at = <Date 2021-01-05.08:10:47.823>
user = 'https://github.com/ned-deily'

bugs.python.org fields:

activity = <Date 2021-01-05.08:10:47.823>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2021-01-05.08:10:47.837>
closer = 'terry.reedy'
components = ['IDLE', 'macOS', 'Tkinter']
creation = <Date 2020-11-30.03:17:54.166>
creator = 'ned.deily'
dependencies = []
files = []
hgrepos = []
issue_num = 42508
keywords = ['patch']
message_count = 13.0
messages = ['382110', '382165', '382166', '382167', '382201', '382203', '382210', '382327', '382435', '382619', '382620', '384374', '384375']
nosy_count = 6.0
nosy_names = ['terry.reedy', 'ronaldoussoren', 'wordtech', 'ned.deily', 'serhiy.storchaka', 'miss-islington']
pr_nums = ['23570', '23571', '23572', '23577', '23669', '23670']
priority = 'critical'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue42508'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

[ned-deily]

An IDLE installation manual smoke test gives odd behavior when run on macOS using the Tcl/Tk 8.6.10 introduced with the new macOS universal2 installer variant as of 3.9.1rc1. The test behaves as expected (without the duplicate and spurious error messages) when run using the traditional macOS 10.9 installer variant that includes Tcl/Tk 8.6.8 built on macOS 10.9.

The test consists of:

• launch the new IDLE by double-clicking on its icon in the applicable /Applications/Python 3.x folder
• press cmd-N to open a new edit window
• enter a three-line snippet with a syntax error, something like:
  x = 1
  y = $2
  print(x,y)
• press cmd-S to save the file entering any valid file name
• press F5 to run the snippet
• observe the popup "Invalid syntax" error window
• click OK to dismiss and observe previous "Invalid syntax" window replaced by an identical error window [ERROR]
• click OK to dismiss
• remove the syntax error and press cmd-S to resave
• press F5 to run again
• observe the output of the print function in the IDLE shell window
• observe popup error window over IDLE shell window with "The Python Shell window is already executing a command: please wait until it is finished." [ERROR]
• click OK to dismiss and observe previous "already executing" window replaced by an identical error window [ERROR]
• click OK to dismiss
• press cmd-Q to quit IDLE

As we are planning to introduce the universal2 installer variant on 3.8.x and 3.10 to natively support Apple Silicon Macs and macOS 11 Big Sur, a quick resolution of this issue is important. While automated tests in the test suite show no new failures, it would be good to manually exercise IDLE in the new installer configuration to look for other unusual behaviors.

https://www.python.org/downloads/release/python-391rc1/

[terryjreedy]

New changeset e41bfd1 by Terry Jan Reedy in branch 'master':
bpo-42508: Remove bogus idlelib.pyshell.ModifiedInterpreter attribute (GH-23570)
e41bfd1

[terryjreedy]

This issue exposed and is affected by an existing system and version independent bug. With that fixed as in PR 23570, running a file with "print('output')" with the new 3.9.1rc1 universal binary results in the following.

output
========= RESTART: /Users/.../tem1.py =========
<'... finished.' dialog box>
<click OK and box reappears>
<click again and>
========== RESTART: Shell =====================

>>
output
>>

The normal output is

========= RESTART: /Users/.../tem1.py =========
output

>>

I am investigating how 'RESTART' is being printed after 'output', even though the restart function is called before the compiled tem1 is sent to be executed.

[miss-islington]

New changeset f4389bf by Miss Islington (bot) in branch '3.8':
bpo-42508: Remove bogus idlelib.pyshell.ModifiedInterpreter attribute (GH-23570)
f4389bf

[terryjreedy]

After hours of failure to understand crazy 'impossible' behavior, I went back to 'timing issue' and this code.

    def run_module_event(self, event):
        if macosx.isCocoaTk():
            # Tk-Cocoa in MacOSX is broken until at least
            # Tk 8.5.9, and without this rather
            # crude workaround IDLE would hang when a user
            # tries to run a module using the keyboard shortcut
            # (the menu item works fine).
            self.editwin.text_frame.after(200,
                lambda: self.editwin.text_frame.event_generate(
                        '<<run-module-event-2>>'))
            return 'break'
        else:
            return self._run_module_event(event)

When I used the Run menu instead of F5, the problem went away. The problem also went away when I added 'and 0' to the condition above to disable the workaround. 3.8 also works fine with the workaround disabled. Ned, if you verify enough to agree, I will get rid of the wrapper and remove the leading '_' from the real handler.

[terryjreedy]

New changeset ff51e5e by Miss Islington (bot) in branch '3.9':
bpo-42508: Remove bogus idlelib.pyshell.ModifiedInterpreter attribute (GH-23570) (GH-23571)
ff51e5e

[terryjreedy]

PR 23577 makes fix suggested above for easier review.

[terryjreedy]

The new tk/tkinter bug in the macOS11 U2 build, other than those reported in bpo-42507 and bpo-42541, is that pressing F5 causes the corresponding pseudoevent handler, run_module_event, to be called twice. (Its code is above.) The interval between calls is about 1 millesecond -- .7 to 1.1 in 5 tries. See this by adding the following at the top of the function's and subtracting after running code with or without syntax error.
print('run', time.perf_counter())

Adding multiple prints in _run_module_event shows the following for code that compiles. With the 200 millesecond cocoa delay, the two calls run interleaved closely interleaved. I don't understand the details, but the result is the scrambled and incorrect output given above. With no delay as with PR 23577, the first call calls restart_subprocess in _run_module_event. Then call 2 gets to the top of this function, then call 1 appears to finish, and call 2 is somehow aborted.

With a syntax error, the 2nd call, redisplaying the error, is not aborted.

F5 resulting in two calls is not unique, even though I have not yet seen overt double call effects. In Shell, Control-F6 restarts. A print in its event handler shows that it is called twice even though there is only 1 restart.

A possible kludge workaround would be to call perf_counter when the class is created and for Cocoa, each run_module call. Return when new-old < 2milliseconds. But the double call should be fixed as it could show up elsewhere.

[terryjreedy]

The new commit to PR 23577 works around the double run_module_event call. Removing the delay changed timings, so I instead measured the time between closing the errorbox and the 2nd call. The allowed difference had to be increased to .015, so I selected .05 to be safer.

[ned-deily]

New changeset 57e5113 by Terry Jan Reedy in branch 'master':
bpo-42508: Keep IDLE running on macOS (GH-23577)
57e5113

[ned-deily]

New changeset 2a9a883 by Miss Islington (bot) in branch '3.9':
bpo-42508: Keep IDLE running on macOS (GH-23577) (GH-23669)
2a9a883

[terryjreedy]

New changeset d05b470 by Miss Islington (bot) in branch '3.8':
bpo-42508: Keep IDLE running on macOS (GH-23577) (bpo-23670)
d05b470

[terryjreedy]

I believe that this issue is done. Please open a new one if more is needed.