Unexpected success of test_get_type_hints_modules_forwardref in test_typing

[serhiy-storchaka]

BPO	33758
Nosy	@benjaminp, @ambv, @serhiy-storchaka, @ilevkivskyi, @miss-islington
PRs	closes bpo-33758: Skip test_get_type_hints_modules_forwardref. #13977 [3.8] closes bpo-33758: Skip test_get_type_hints_modules_forwardref. (GH-13977) #13978 [3.7] closes bpo-33758: Skip test_get_type_hints_modules_forwardref. (GH-13977) #13979

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-06-11.16:18:36.114>
created_at = <Date 2018-06-03.15:52:01.434>
labels = ['3.7', '3.8', 'type-bug', 'tests']
title = 'Unexpected success of test_get_type_hints_modules_forwardref  in test_typing'
updated_at = <Date 2019-06-11.16:52:41.897>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2019-06-11.16:52:41.897>
actor = 'miss-islington'
assignee = 'none'
closed = True
closed_date = <Date 2019-06-11.16:18:36.114>
closer = 'benjamin.peterson'
components = ['Tests']
creation = <Date 2018-06-03.15:52:01.434>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 33758
keywords = ['patch']
message_count = 7.0
messages = ['318563', '318643', '345171', '345222', '345250', '345252', '345253']
nosy_count = 5.0
nosy_names = ['benjamin.peterson', 'lukasz.langa', 'serhiy.storchaka', 'levkivskyi', 'miss-islington']
pr_nums = ['13977', '13978', '13979']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33758'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[serhiy-storchaka]

$ ./python -m test -v -m test_get_type_hints_modules_forwardref test_typing
...
test_get_type_hints_modules_forwardref (test.test_typing.GetTypeHintTests) ... unexpected success

---

Ran 1 test in 0.001s

FAILED (unexpected successes=1)

[ilevkivskyi]

Oh yes, this is a "stateful" bug. It will not appear if run in isolation. Btw, the underlying bug will be worse with from __future__ import annotations, so it would make sense to fix this sooner than later.

[benjaminp]

In particular, test_typing has an unexpected success if you run it after test___all__. It would be nice at least to change the expected fail to a skip, since it's so subtle.

[ilevkivskyi]

Likely, the unexpected success can be avoided with clearing generic caches in test___all__ (it just imports whatever it can find and likely also test/mod_generics_cache.py), but I am also fine with just skipping it for now. Will you have time to make a PR?

[benjaminp]

New changeset 910b3fc by Benjamin Peterson in branch 'master':
closes bpo-33758: Skip test_get_type_hints_modules_forwardref. (GH-13977)
910b3fc

[miss-islington]

New changeset 1c31d19 by Miss Islington (bot) in branch '3.8':
closes bpo-33758: Skip test_get_type_hints_modules_forwardref. (GH-13977)
1c31d19

[miss-islington]

New changeset fad8904 by Miss Islington (bot) in branch '3.7':
closes bpo-33758: Skip test_get_type_hints_modules_forwardref. (GH-13977)
fad8904