Fix leaks in idlelib

[terryjreedy]

BPO	30642
Nosy	@terryjreedy, @mlouielu
PRs	bpo-30642: IDLE: Fix test_query refleak #2147 [3.6]bpo-30642: IDLE: Fix test_query refleak [GH-2147] #2161 bpo-30642: Fix ref leak in idle_test.test_macosx #2163 [3.6]bpo-30642: Fix ref leak in idle_test.test_macosx (#2163) #2165

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2017-06-16.05:12:51.062>
created_at = <Date 2017-06-13.05:17:26.397>
labels = ['expert-IDLE', 'type-bug', '3.7']
title = 'Fix leaks in idlelib'
updated_at = <Date 2017-08-07.18:07:39.082>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2017-08-07.18:07:39.082>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2017-06-16.05:12:51.062>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2017-06-13.05:17:26.397>
creator = 'terry.reedy'
dependencies = []
files = []
hgrepos = []
issue_num = 30642
keywords = []
message_count = 8.0
messages = ['295846', '295852', '295911', '295918', '295923', '295932', '295981', '299856']
nosy_count = 2.0
nosy_names = ['terry.reedy', 'louielu']
pr_nums = ['2147', '2161', '2163', '2165']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30642'
versions = ['Python 3.6', 'Python 3.7']

[terryjreedy]

When Louie Lu posted a link to
https://blog.louie.lu/2017/06/12/diagnosing-and-fixing-reference-leaks-in-cpython/
on core-mentorship list, I tested idlelib.
python -m test -ugui test_idle # SUCCESS, no extraneous output
python -m test -R: test_idle # SUCCESS, no extraneous output
python -m test -R: -ugui test_idle # error output, FAILURE
[So people who leaktest without a screen see nothing in idlelib.]

Error output is about 20 copies of the following:
can't invoke "event" command: application has been destroyed
while executing
"event generate $w <<ThemeChanged>>"
(procedure "ttk::ThemeChanged" line 6)
invoked from within
"ttk::ThemeChanged"

At the end:
test_idle leaked [471, 471, 471, 471] references, sum=1884
test_idle leaked [209, 211, 211, 211] memory blocks, sum=842
[similar for python 3.6]

In a response email, I noted that test_idle gathers tests from idlelib.idle_test.test_* and that something extra is needed to pin leaks to specific test modules.

I don't know whether the absence of 'invoke event' error messages when not running -R means that there are also no refleaks, or not.
---

import os
import subprocess

os.chdir('f:/dev/3x/Lib/idlelib/idle_test')
testfiles = [name for name in os.listdir() if name.startswith('test_')]
for name in testfiles:
    os.rename(name, 'x'+name)
for name in testfiles:
    os.rename('x'+name, name)
    try:
        res = subprocess.run(
            ['f:/dev/3x/python.bat', '-m', 'test', '-R:', '-ugui', 'test_idle'],
             stdout=subprocess.PIPE, stderr=subprocess.PIPE)
        if res.returncode:
            print(name)
            print(res.stderr)
    except Exception as err:
        print(name, err)
    os.rename(name, 'x'+name)
for name in testfiles:
    os.rename('x'+name, name)

---

reports
test_macosx.py
b'beginning 9 repetitions\r\n123456789\r\n\r\ntest_idle leaked [31, 31, 31, 31] references, sum=124\r\ntest_idle leaked [19, 21, 21, 21] memory blocks, sum=82\r\n'
test_query.py
b'beginning 9 repetitions\r\n123456789\r\n\r\ntest_idle leaked [429, 429, 429, 429] references, sum=1716\r\ntest_idle leaked [190, 192, 192, 192] memory blocks, sum=766\r\n'
There are no 'invoke event' messages.

For further testing within each file, by commenting out code, as suggested in the link above, I replaced 'testfiles' in the middle loop with ['testmacosx.py'] or ['test_query.py']. For test_macosx, the culprit is class SetupTest. For test_query, the culprit is class QueryGuiTest. Adding cls.root.update_idletasks did not solve the problem by itself (as it has in other cases). I plan to continue another time.

[mlouielu]

test_query were fixed in PR 2147, which is leak by not removing mock.Mock() in dialog.

[terryjreedy]

New changeset b070fd2 by terryjreedy (mlouielu) in branch 'master':
bpo-30642: IDLE: Fix test_query refleak (bpo-2147)
b070fd2

[terryjreedy]

New changeset 2bfb45d by terryjreedy in branch '3.6':
bpo-30642: IDLE: Fix test_query refleak (bpo-2147) (bpo-2161)
2bfb45d

[terryjreedy]

The macosx leak came from repeated calls to
root.createcommand("::tk::mac::OpenDocument", doOpenFile)
in macosx.addOpenEventSupport(root, flist), which is called from macosx.setupApp, which is called in test_macosx.SetupTest.test_setupapp.

Attached is an improved version of findleak.py, which made it easy to isolate one test file by renaming, repeatedly determine whether the leak remained after modifying either macosx.py or test_macosx.py, and revert all the other names when done.

[terryjreedy]

New changeset b0efd49 by terryjreedy in branch '3.6':
[3.6]bpo-30642: Fix ref leak in idle_test.test_macosx (bpo-2163) (bpo-2165)
b0efd49

[terryjreedy]

f:\dev\36>python -m test -R: -ugui test_idle
gives about 40 invoke event messages, but the test passes. So the messages and leaks are not connected.

[terryjreedy]

Unlinked old findleak.py after uploading much improved version to bpo-31130.