unittest.TestResult.addSubTest should be called immediately after subtest finishes

[sir-sigurd]

BPO	30856
Nosy	@rbtcollins, @ezio-melotti, @voidspace, @serhiy-storchaka, @sir-sigurd
PRs	bpo-30856: Update TestResult early, without buffering in _Outcome #28180
Files	test_subtest.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-09-19.12:25:04.750>
created_at = <Date 2017-07-05.13:56:35.991>
labels = ['type-bug', 'library', '3.11']
title = 'unittest.TestResult.addSubTest should be called immediately after subtest finishes'
updated_at = <Date 2021-09-19.12:25:04.750>
user = 'https://github.com/sir-sigurd'

bugs.python.org fields:

activity = <Date 2021-09-19.12:25:04.750>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-19.12:25:04.750>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2017-07-05.13:56:35.991>
creator = 'sir-sigurd'
dependencies = []
files = ['46990']
hgrepos = []
issue_num = 30856
keywords = ['patch']
message_count = 3.0
messages = ['297756', '401102', '402153']
nosy_count = 5.0
nosy_names = ['rbcollins', 'ezio.melotti', 'michael.foord', 'serhiy.storchaka', 'sir-sigurd']
pr_nums = ['28180']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue30856'
versions = ['Python 3.11']

[sir-sigurd]

Currently TestResult.addSubTest() is called just before TestResult.stopTest(), but docs says that addSubTest is "Called when a subtest finishes". IMO that means that it will be called immediately after subtest finishes, but not after indefinite time.

Test is attached.

[serhiy-storchaka]

Before b3468f7 the TestResult methods (addFailure, addError, addSkip, addExpectedFailure, addUnexpectedSuccess) were called immediately after running the test method. After that change all skips, failures and errors were accumulated separately and TestResult was updated only after the test cleanup. I do not know whether there were reasons for this or it is just an implementation artifact.

The proposed PR makes addFailure, addError, addSkip and addSubTest be called immediately after raising an exception or finishing a subtest. addExpectedFailure, addUnexpectedSuccess and addSuccess still need to wait the end of the test cleanup.

[serhiy-storchaka]

New changeset 664448d by Serhiy Storchaka in branch 'main':
bpo-30856: Update TestResult early, without buffering in _Outcome (GH-28180)
664448d