IDLE: Freeze when closing Settings (& About) dialog on MacOS

[vtudorache]

BPO	34120
Nosy	@rhettinger, @terryjreedy, @ronaldoussoren, @taleinat, @ned-deily, @serhiy-storchaka, @miss-islington
PRs	bpo-34120: fix IDLE freezing after closing dialogs #8603 [3.7] bpo-34120: fix IDLE freezing after closing dialogs (GH-8603) #8613 [2.7] bpo-34120: fix IDLE freezing after closing dialogs (GH-8603) #8614 [3.6] bpo-34120: fix IDLE freezing after closing dialogs (GH-8603) #8615 bpo-34120: fix text viewer to call grab_release() only when needed #8616 [3.7] bpo-34120: fix text viewer to call grab_release() only when needed (GH-8616) #8617 [3.6] bpo-34120: fix text viewer to call grab_release() only when needed (GH-8616) #8618

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2018-08-02.08:33:46.576>
created_at = <Date 2018-07-15.09:53:29.852>
labels = ['OS-mac', '3.8', 'expert-IDLE', 'type-bug', '3.7']
title = 'IDLE: Freeze when closing Settings (& About) dialog on MacOS'
updated_at = <Date 2018-08-02.08:33:46.574>
user = 'https://bugs.python.org/vtudorache'

bugs.python.org fields:

activity = <Date 2018-08-02.08:33:46.574>
actor = 'taleinat'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2018-08-02.08:33:46.576>
closer = 'taleinat'
components = ['IDLE', 'macOS']
creation = <Date 2018-07-15.09:53:29.852>
creator = 'vtudorache'
dependencies = []
files = []
hgrepos = []
issue_num = 34120
keywords = ['patch']
message_count = 27.0
messages = ['321686', '322646', '322654', '322698', '322717', '322718', '322764', '322825', '322837', '322845', '322850', '322851', '322874', '322875', '322880', '322884', '322889', '322891', '322893', '322917', '322919', '322922', '322923', '322924', '322926', '322928', '322929']
nosy_count = 11.0
nosy_names = ['rhettinger', 'terry.reedy', 'ronaldoussoren', 'taleinat', 'wordtech', 'ned.deily', 'serhiy.storchaka', 'miss-islington', 'walters', 'vtudorache', 'eamartin']
pr_nums = ['8603', '8613', '8614', '8615', '8616', '8617', '8618']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue34120'
versions = ['Python 2.7', 'Python 3.6', 'Python 3.7', 'Python 3.8']

[vtudorache]

When closing the IDLE's Preferences dialog the IDLE's Console window and/or Editor window (whichever was active before) doesn't show cursor/caret any more. One must click on Desktop or another window then click again in the IDLE's window in order to regain cursor/caret. This problem shows on macOS High Sierra, Python 3.6.N and 3.7.N (and even 3.5.N), with the provided installers (64 bit, 64/32 bit, Tk 8.6.8) and with self-compiled Python(s). After a number of openings IDLE will crash with an @autoreleasepool issue. I've compiled Tcl/Tk and Python with CC=clang CXX=clang++ MACOSX_DEPLOYMENT_TARGET=10.9 (10.10, 10.12 also), Tcl/Tk with and without --enable-threads, always with --enable-64bit --enable-framework and --enable-dtrace, clang 902.0.39.2 (Apple LLVM version 9.1.0), Python was compiled with and without --enable-dtrace and --enable-optimizations, with --enable-framework.

[terryjreedy]

On idledev thread "Mac IDLE 3.7.0 freezes when accessing Preferences", Walter Schnyder reported something similar. Combining two posts:

"using python.org Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24) [Clang 6.0 (clang-600.0.57)] on darwin [64-bit] and came across these [new] bugs on several Macs, all running OS X v 10.13.5.
[Has run IDLE before, back to 3.4.]

Bug A:

1. From the shell select Preferences in the IDLE menu.
2. Cancel the Preferences to get back to the Shell
3. The shell is frozen. It doesn’t even react to CONTROL-D.

The only way I found to resurrect it is to control-click the shell window and select “Go to file/line” causing an error message to appear. Click OK and the shell comes back to life.

Note: This doesn’t only happen when accessing preferences. It also happens when selecting “About IDLE” in the IDLE menu."

After starting with 'python3 -m idlelib'

"The same problems occur. There is no error in the terminal window.
But I learned something new: after running the steps for bug A, IDLE is frozen (no blinking cursor, not responding to Control+D. However, if I bring another application’s window to the front (by clicking on it) then click back on IDLE’s window, this resurrects IDLE, the cursor blinks and Control+D quits."

test_idle passes. When running 'python3 -m idlelib.configdialog' or 'idlelib.help_about', which run a human-viewed test, there is no problem when closing either dialog.

Vlad, if I understand your post, you *do* see the problem with 3.5? With the python.org installer and tk 8.5? As far as I know, no one has had a problem with this. Or only with a private compile with tk 8.6?

This is important because 3.5 IDLE has not been touched since about 18 months, and if there is only a problem when using tk 8.6, then I have to suspect that this and other new problems are result of tk 8.6 on Mac or tkinter needing a patch to support it.

[vtudorache]

I’m sorry for not being clear. This problem appears with Tk 8.6, even in
Python 3.5 (compiled by me).
I’ve succeeded to reproduce only once the @autorelease pool error (it
appeared after several open/close of the preferences or about window). Did
anyone else see it?

Le lun. 30 juil. 2018 à 02:51, Terry J. Reedy <report@bugs.python.org> a
écrit :

Terry J. Reedy <tjreedy@udel.edu> added the comment:

On idledev thread "Mac IDLE 3.7.0 freezes when accessing Preferences",
Walter Schnyder reported something similar. Combining two posts:

"using python.org Python 3.7.0 (v3.7.0:1bf9cc5093, Jun 26 2018, 23:26:24)
[Clang 6.0 (clang-600.0.57)] on darwin [64-bit] and came across these [new]
bugs on several Macs, all running OS X v 10.13.5.
[Has run IDLE before, back to 3.4.]

Bug A:

1. From the shell select Preferences in the IDLE menu.
2. Cancel the Preferences to get back to the Shell
3. The shell is frozen. It doesn’t even react to CONTROL-D.

The only way I found to resurrect it is to control-click the shell window
and select “Go to file/line” causing an error message to appear. Click OK
and the shell comes back to life.

Note: This doesn’t only happen when accessing preferences. It also happens
when selecting “About IDLE” in the IDLE menu."

After starting with 'python3 -m idlelib'

"The same problems occur. There is no error in the terminal window.
But I learned something new: after running the steps for bug A, IDLE is
frozen (no blinking cursor, not responding to Control+D. However, if I
bring another application’s window to the front (by clicking on it) then
click back on IDLE’s window, this resurrects IDLE, the cursor blinks and
Control+D quits."

test_idle passes. When running 'python3 -m idlelib.configdialog' or
'idlelib.help_about', which run a human-viewed test, there is no problem
when closing either dialog.

Vlad, if I understand your post, you *do* see the problem with 3.5? With
the python.org installer and tk 8.5? As far as I know, no one has had a
problem with this. Or only with a private compile with tk 8.6?

This is important because 3.5 IDLE has not been touched since about 18
months, and if there is only a problem when using tk 8.6, then I have to
suspect that this and other new problems are result of tk 8.6 on Mac or
tkinter needing a patch to support it.

----------
components: +macOS
nosy: +ned.deily, rhettinger, ronaldoussoren, taleinat, walters
stage: -> needs patch
title: IDLE Caret/Focus Lost -> IDLE: Freeze when closing Settings (&
About) dialog on MacOS
versions: +Python 3.8

---

Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue34120\>

---

[terryjreedy]

Thanks. This is evidence for my suspicion that this results from a problem with the new tkinter/tcl/tk8.6 delivered with 3.7. See bpo-34275.

PS: when replying by email, please delete the quote, which is redundant with the post on the web page.

[terryjreedy]

I closed bpo-34281 as a duplicate of this. Eric Martin said "After entering a line number, clicking OK brings to the foreground and makes active the first hidden window (in the simplest case where we have just one window for a .py file and the Shell window, the former being in the foreground, clicking OK brings the Shell window to the foreground)."

Normally, one window is active, as indicated by the traffic lights, and one widget has the focus, which for entry widgets is indicated by the cursor. I tested with 3.7.0 with MacOS 10.13.6 and a shell and editor window.

IDLE > About IDLE: closing either way turns lights on in original window but text does not get focus (no cursor). Neither TAB nor click return focus, but the window is not frozen either. Selection, deselection, and copy work, cut and paste do not.

As far as I know, on Mac, cut and paste as well as copy and selection are handled entirely by tk, and not IDLE. My interpretation is that the window is 'stuck' in an 'illegal' state, partly enabled, partly disabled. Clicking and activating the other window disables the stuck window, so that clicking on it enables it.

IDLE > Settings works the same. But...

Options > Configure IDLE works as Eric described above. Closing with OK or Cancel or Red light activates the *other* window, in a stuck state.
(If there is no other IDLE window, then something else must be clicked to unstick the originally active window.)

Edit > Go to line: If one cancels, behavior is normal. If one enters a number and clicks OK, the other window is activated (lights on) without a cursor while the cursor goes where requested in the original, apparently 'inactive' window. And keyboard input goes where the cursor is, in that unlighted window. AFAIK, this is 'impossible'.

[terryjreedy]

After more experiments on Windows, the 'stuck' window -- active, no cursor, only copy works -- is a read-only window. I looked at other windows.

Modal
Edit > Find (Command F)
Edit > Replace (Command R)
When closed, neither window is activated. This seems wrong (on Windows, the original is), but I don't know what normal is on Linux or Mac.

Options > Settings > Keys Tab > Get New Keys for Selection: When this is closed, the text window rather than the settings dialog is activated in read-only state. Must click on Settings dialog. New selections does appear in keys list.

Options > Settings > Highlights > Choose Color for: When closed, all blue highlights on tab disappear. Colorchooser is supplied by tk and might be from Apple.

File > Save (Copy) As: This OS supplied dialog is an oddball. Only the red light turns gray and cursor remains without blinking. The dialog has no lights and closed it turns the red back on and the cursor blinks.

Non-modal
Help > IDLE Help
Debug > Debugger
Debug > Stack Viewer
Help > Turtle Demo
When closed, last active other window is activated.

Conclusion: With the exception of Save and cancelled Goto, closing modal dialogs seems bugged in various ways.

[wordtech]

I've observed this behavior myself, and wonder if you are hitting some edge case in Tk-Mac event processing (there used to be a lot of issues with this and we thought we had addressed them). I don't want to code-dive into Python's implementation of these dialogs, so can you provide a simple script that demonstrates the issue and I'll take a closer look? Often bugs of this sort can be addressed at the script level with some tweaks.

[terryjreedy]

As with the tooltip modules (see bpo-34275, msg322824), the dialog modules all have run-when-main test functions. Unittests are also run, before the human-verified test.

python3 -m idlelib.searchbases fails in that the driver-box is not activated (lighted) when the searchbase box is closed.
Replace 'searchbase' with 'search', 'replace', or 'grep' which text subclasses of Searchbase, and an intermediate lighted window with a Text widget and 2nd button appear. (For the grep test, click the window and maybe type something to put focus on the text.) Click the button to get the dialog. Close it. The text window does not get lighted. It does on Windows. (There are other glitches on Windows that do not appear in IDLE itself).

Reproducing the 'wrong' text window being activated will require something new with two Toplevel text windows.

[wordtech]

Removing the call "self.grab_set" in configdialog.py (line 87 or so) and help_about.py (line 47 or so) appears to fix the problem with the main window freezing when the modal dialog is destroyed on macOS. "Grab" has never worked properly on Tk on the Mac, but it has additional problems in the Cocoa implementation of Tk; it causes all kinds of problems with the event loop and is best avoided altogether. If the call to grab is crucial on other platforms, it can be wrapped in a call to "tk windowingsystem ne aqua" to exclude the Mac. If other modal dialogs present similar behavior on the Mac, look for calls to grab and try omitting that call. I'll leave it to someone else to propose a thorough patch, but this should point you in the right direction.

[terryjreedy]

The tkinter doc could use a section on 'OS-tk peculiarities and interactions' ;-). Anyway, thanks again. Some of the 'Windows peculiarities' I noted above, where the test behaved better on Mac, may be due to the absence of calls that should be avoided on Mac. Perhaps I can factor out some of the dialog setup and shutdown code, get it right for each platform, and then use it for all or at least multiple dialogs.

[taleinat]

I can confirm that removing .grab_set() calls fixes this issue on macOS 10.13.5 with Python 3.7.0 from python.org.

Searching for those calls in all of IDLE's code lead me to discover additional similar issues, e.g. with the search dialog and the text viewer. Removing the .grab_set() and .grab_release() calls fixes those issues as well.

I tried IDLE with these changes on Windows 10 with python build from the latest master branch. Things appear to be working well without these calls.

Is it reasonable to simply remove the grab calls entirely? I've created a PR with such a change to facilitate evaluating this approach.

[taleinat]

Confirmed on macOS 10.13.5 with the new Python 2.7.15 64-bit from python.org which bundles Tcl/Tk 8.6.8.

The same fix works: Removing the grab_set() calls fixes the freezing after closing the about and config dialogs.

[vtudorache]

I can confirm that removing the grab_set() calls fixes the locking. But am I the only one to notice that the dialogs aren't modal any more?

[vtudorache]

Adding self.grab_release() in the ok() method for the dialogs BEFORE destroy() solved the problem for me, keeping the grab_set() methods in place (uncommented, actives).

[vtudorache]

I've checked again the source code in config_key.py, configdialog.py, help_about.py, query.py, searchbase.py, textview.py and added a self.grab_release() in functions like ok() and cancel() before the call to self.destroy() and I see no more lock/focus issues. Maybe the grab was kept in the non-existent (destroyed) dialog window, so the click events could not be transferred to the pyshell or editor, but, because of a local (not global) grab, clicking on another application's window, by losing focus and grab, completely freed the event-chain, then clicks in IDLE were correctly received.
Can you confirm if adding the grab_release() calls works for you too (do not remove the grab_set() calls!)?

[taleinat]

Adding grab_release() calls also resolves the issue. Tested on mac and win.

PR updated with this approach, IMO ready for review.

[terryjreedy]

I wrote the following justification for adding grab_release while Tal was revising his PR to make the change. I will look at the PR now and test on Windows.
---

Vlad, I was wondering about modality. Properly de-modalizing a dialog requires more than just removing grab_set. The parent of a modal dialog is a window with a menu, an instance of idlelib. If a dialog were not modal, its parent could be deleted, which would also delete the dialog and any un-saved and un-acted-upon data. This is true of Debugger, whose parent is Shell. Duplicate dialogs would also be possible, as is true of IDLE Help, whose parent is root, unless explicitly prevented, as is true of Debugger (Shell knows if there is a Debugger instance).

Removing modality has been discussed in various places, such as in bpo-24760, which focuses on Settings. If it were de-modalized, its parent should be root and only one copy allowed. Mark Roseman noted on the issue that "because modal doesn't fully work on the Mac, you can actually create multiple copies of the config dialog!".

So my preferred short-term fix, once confirmed, for 3.7.1, would be the addition of grab_release.

Merely removing grab would have to be Mac-specific, and I would expect new problems and complaints. Properly de-modalizing on all platforms, where sensible, is a longer-term project. Notepad++ on Windows, for instance, has a non-modal singleton Find dialog (with Replace and Find in Files on separate tabs), designed as such, that operates on the 'current' editor tab, unless one selects 'Find All in All Open Documents' to work on all of them.

PS: Tal Einat also get 'autorelease pool' errors when trying to run IDLE on locally built 3.7.0.

[taleinat]

Why remove the 2.7 version? This bug occurs with 2.7.15 on macOS.

[terryjreedy]

Removing 2.7 was unintended, from simultaneous editing. This is one of the exceptional cases where a backport is worth the risk (which seems here extremely low). I will remove it from the PR because auto backport will fail because of file renames, in addition to internal changes. Here are the new name to old name translations, where not a just a case change, needed to prepare a 2.7 patch.

config_key keybindingDialog
help_about aboutDialog
query . # New in 3.6. Grep 2.7 code for 'grab_set' to find any usages in deleted code that query replaced.

[taleinat]

New changeset 10ea940 by Tal Einat in branch 'master':
bpo-34120: fix IDLE freezing after closing dialogs (GH-8603)
10ea940

[miss-islington]

New changeset d9fc795 by Miss Islington (bot) in branch '3.7':
bpo-34120: fix IDLE freezing after closing dialogs (GH-8603)
d9fc795

[miss-islington]

New changeset 9fcfb7b by Miss Islington (bot) in branch '3.6':
bpo-34120: fix IDLE freezing after closing dialogs (GH-8603)
9fcfb7b

[taleinat]

New changeset 894940b by Tal Einat in branch '2.7':
[2.7] bpo-34120: fix IDLE freezing after closing dialogs (GH-8603)
894940b

[taleinat]

New changeset dd74369 by Tal Einat in branch 'master':
bpo-34120: fix text viewer to call grab_release() only when needed (GH-8616)
dd74369

[miss-islington]

New changeset 60586de by Miss Islington (bot) in branch '3.7':
bpo-34120: fix text viewer to call grab_release() only when needed (GH-8616)
60586de

[miss-islington]

New changeset 8c4a005 by Miss Islington (bot) in branch '3.6':
bpo-34120: fix text viewer to call grab_release() only when needed (GH-8616)
8c4a005

[taleinat]

Many thanks for the report and follow through, Vlad!

Thank you too, Kevin, for your help in resolving this.