asyncio test failure on appveyor

[terryjreedy]

BPO	35620
Nosy	@terryjreedy, @asvetlov, @1st1, @pablogsal, @tirkarthi

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-09-12.14:21:56.565>
created_at = <Date 2018-12-30.19:47:17.722>
labels = ['3.8', 'type-bug', 'tests', 'expert-asyncio']
title = 'asyncio test failure on appveyor'
updated_at = <Date 2019-09-12.14:21:56.564>
user = 'https://github.com/terryjreedy'

bugs.python.org fields:

activity = <Date 2019-09-12.14:21:56.564>
actor = 'asvetlov'
assignee = 'none'
closed = True
closed_date = <Date 2019-09-12.14:21:56.565>
closer = 'asvetlov'
components = ['Tests', 'asyncio']
creation = <Date 2018-12-30.19:47:17.722>
creator = 'terry.reedy'
dependencies = []
files = []
hgrepos = []
issue_num = 35620
keywords = []
message_count = 4.0
messages = ['332757', '332765', '332773', '352152']
nosy_count = 5.0
nosy_names = ['terry.reedy', 'asvetlov', 'yselivanov', 'pablogsal', 'xtreak']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue35620'
versions = ['Python 3.8']

[terryjreedy]

https://ci.appveyor.com/project/python/cpython/builds/21296354?fullLog=true
Blocked merge. Passed on Azure Pipeline +- same time. Appveyor re-run passed. Please disable or weaken the false positive tests.

I will propose a different solution elsewhere, perhaps pydev.

[asvetlov]

Do you mean an environment modification?

1 test altered the execution environment:
test_asyncio

[terryjreedy]

My position is that tests that are known to occasionally fail independently of any changes in particular PRs should not be allowed to block merging. In the context of CI, the failure is a lie.

One solution, and the easiest, is to disable the test. That is all I could do here.

If the module/test author wants to keep running the test, another solution is to change the test so that bogus results are no longer seen as a failure by the test framework. How depends on the situation. Perhaps turn particular exceptions into a warning message?

In this case, there are multiple exceptions listed in the log:
OSError: [WinError 6] The handle is invalid x 2
OSError: [WinError 995] The I/O operation has been aborted because of either a thread exit or an application request, leading to
ConnectionResetError: [WinError 995] ...
OSError: [WinError 64] The specified network name is no longer available, leading to another ConnectionResetError.

I don't see am a little surprised at and don't understand the vague end failure message. So I cannot suggest anything more specific.

The third solution I am thinking of is somehow change the test framework so we can somehow mark 'heisentests' as such and not count know false positives as merge-blocking failures.

[tirkarthi]

asyncio tests are stable now and I couldn't find related reports. I guess it's fixed in one of the commits and also AppVeyor is not used anymore. I guess this can be closed as outdated.