Make IDLE tests less flashy

[terryjreedy]

BPO	27922
Nosy	@terryjreedy, @zhangyangyu
PRs	bpo-27922: Stop gui flash when run IDLE's test_parenmatch #2171 [3.6]bpo-27922: Stop gui flash from idle_test.test_parenmatch (#2171) #2172
Files	flashy.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/terryjreedy'
closed_at = <Date 2020-06-06.22:13:54.047>
created_at = <Date 2016-09-01.00:33:36.247>
labels = ['expert-IDLE', 'type-feature', '3.7']
title = 'Make IDLE tests less flashy'
updated_at = <Date 2020-06-06.22:13:54.046>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2020-06-06.22:13:54.046>
actor = 'terry.reedy'
assignee = 'terry.reedy'
closed = True
closed_date = <Date 2020-06-06.22:13:54.047>
closer = 'terry.reedy'
components = ['IDLE']
creation = <Date 2016-09-01.00:33:36.247>
creator = 'terry.reedy'
dependencies = []
files = ['44310']
hgrepos = []
issue_num = 27922
keywords = ['patch']
message_count = 14.0
messages = ['274080', '274081', '274083', '274378', '274389', '274390', '274415', '274495', '276737', '295946', '295948', '295955', '295956', '370856']
nosy_count = 3.0
nosy_names = ['terry.reedy', 'python-dev', 'xiang.zhang']
pr_nums = ['2171', '2172']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue27922'
versions = ['Python 3.6', 'Python 3.7']

[terryjreedy]

This issue follows-up on bpo-27732, which suppressed beeps during IDLE tests, I want to also suppress the flashing of tk widget boxes, which has become visually obnoxious with the increasing number of tests.

Adding root.withdraw() after root = Tk() solves the problem for all current tests. For test_textview, this requires the fix of a bad cleanup call and addition of another. It appears that the textview toplevels must be destroyed before the cleanup call to root.update_idletasks() or else they are made visible. The idletasks call is needed to avoid '''can't invoke "event" command''' (even with all toplevels gone).

The backports will have fewer changes because there are fewer tests.

[terryjreedy]

This is also a follow-up to bpo-27918. Importing test.support causes a flash in the process of testing whether tkinter/tk work. So I was never able to get IDLE tests to completely stop flashing. Adding 'root.withdraw' to test.support.__init__ solves the import flash, and with that fix, individual IDLE tests and test_idle no longer flash.

[python-dev]

New changeset bf0cb86c6219 by Terry Jan Reedy in branch '2.7':
Issue bpo-27922: IDLE tests no longer flash tk widgets.
https://hg.python.org/cpython/rev/bf0cb86c6219

New changeset ff3a6303c5b1 by Terry Jan Reedy in branch '3.5':
Issue bpo-27922: IDLE tests no longer flash tk widgets.
https://hg.python.org/cpython/rev/ff3a6303c5b1

New changeset fc711879c64a by Terry Jan Reedy in branch 'default':
Issue bpo-27922: IDLE tests no longer flash tk widgets (Merge 3.5).
https://hg.python.org/cpython/rev/fc711879c64a

[python-dev]

New changeset 6e4475894a79 by Terry Jan Reedy in branch '2.7':
Issue bpo-27922: IDLE test_idlehistory no longer flash tk widgets.
https://hg.python.org/cpython/rev/6e4475894a79

[zhangyangyu]

Hi Terry, I think this issue may be not completed. While running idlelib test with -ugui, I can sometimes still get a window flash. I think it's due to test_searchbase. Running python -m idlelib.idle_test.test_searchbase can reproduce the flash and adding withdraw doesn't work for me.

[zhangyangyu]

After some tries, I think it seems to be caused by the second open in test_open_and_close in SearchDialogBaseTest. The second open calls self.top.deiconify.

[terryjreedy]

Thanks for the good detective work. The flash is always there, somewhere, though sometimes minimal. SearchBase.open begins with

        if not self.top:
            self.create_widgets()
        else:
            self.top.deiconify()
            self.top.tkraise()

The first and second open()s call the first and second branch. I am puzzled that the first does not flash, as top is created 'normal' rather than 'withdrawn'. It also turn out that commenting out either of the two lines stops the flash.

The flash could be prevented by changing 'else' to 'else if not _utest' and adding '_utest' as a parameter and passing True, as I have elsewhere. However, the purpose of the second open() is to take that branch for near 100% coverage, and I have previously only used _utest to avoid blocking .wait_window calls.

I think I am going to leave this alone for now, but please feel free to comment on other IDLE issues.

[zhangyangyu]

It's fine, knowing that the maintainer thinks this behaviour is okay. :)

[terryjreedy]

It is more a matter of tradeoffs rather than 'okay'.

There is no general pydev rule against gui flashing. The tk and ttk gui tests still do, but that is primarily Serhiy's concern.

The IDLE tests are at most perhaps a third complete, so they would have become collectively 3 or more times worse. I should run the IDLE tests at least once for each patch and usually do it multiple times. When I work on a module or its tests, I tend work incrementally and run the test for the module 10 or 20 times or more. So my first concern is my own eyes and mental state.

Second, I plan to ask others to run test_idle to make up for the lack of buildbot coverage, and I will feel better about doing so after the patches already made. If one or a few flashes are left, I and others are still much better off.

In this case, there are two obvious fixes, but I don't like either. Maybe I will come up with a third I like better, perhaps after some refactoring of the module. I would also like to better understand the detailed behavior. In the meanwhile, I reopened to add a comment to the test, so I don't lose track of the source of the remaining flash.

[terryjreedy]

Using findleak.py, attached to bpo-30642, modified to always print the filename, I discovered that test_parenmatch also flashes (along with test_searchbase). The leak test runs each test_xyz module 9 times, so is a good way to see flashing.

Adding 'root.withdraw' stops the single flash when running test_parenmatch once. However, it does not stop flashing when running 9 times with -R: refleak test. Flash continues even if class is reduced to

class ParenMatchTest(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        cls.root = Tk()
        cls.root.withdraw()

    @classmethod
    def tearDownClass(cls):
        cls.root.destroy()
        del cls.root

    def test_dummy(self): pass

This is a puzzle since the same skeleton is in several other files. In fact, at least one, test_autocomplete, does not even have root.withdraw. In any case, I will add it here.

[terryjreedy]

New changeset 049cf2b by terryjreedy in branch 'master':
bpo-27922: Stop gui flash from idle_test.test_parenmatch (bpo-2171)
049cf2b

[terryjreedy]

New changeset d92ee3e by terryjreedy in branch '3.6':
[3.6]bpo-27922: Stop gui flash from idle_test.test_parenmatch (bpo-2171) (bpo-2172)
d92ee3e

[terryjreedy]

Using findleak.py, attached to bpo-30642, modified to always print the filename, I discovered that test_parenmatch also flashes (along with test_searchbase). The leak test runs each test_xyz module 9 times, so is a good way to see flashing.

Adding 'root.withdraw' stops the single flash when running test_parenmatch once. However, it does not stop flashing when running 9 times with -R: refleak test. Flash continues even if class is reduced to

class ParenMatchTest(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        cls.root = Tk()
        cls.root.withdraw()

    @classmethod
    def tearDownClass(cls):
        cls.root.destroy()
        del cls.root

    def test_dummy(self): pass

This is a puzzle since the same skeleton is in several other files. In fact, at least one, test_autocomplete, does not even have root.withdraw. In any case, I will add it here.

[terryjreedy]

After doing away with most of the flashes, we started adding tests using key and mouse event_generate, which seemed to require a visible GUI. So the goal seems a lost cause. If I revisit, a new issue.