Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

asyncio: bug in _ProactorBasePipeTransport._force_close() #79863

Closed
vstinner opened this issue Jan 8, 2019 · 3 comments
Closed

asyncio: bug in _ProactorBasePipeTransport._force_close() #79863

vstinner opened this issue Jan 8, 2019 · 3 comments
Labels
3.7 (EOL) end of life 3.8 only security fixes OS-windows topic-asyncio

Comments

@vstinner
Copy link
Member

vstinner commented Jan 8, 2019

BPO 35682
Nosy @pfmoore, @vstinner, @tjguk, @asvetlov, @zware, @1st1, @zooba, @miss-islington
PRs
  • bpo-35682: Fix _ProactorBasePipeTransport._force_close() #11462
  • bpo-35682: Fix _ProactorBasePipeTransport._force_close() #11462
  • bpo-35682: Fix _ProactorBasePipeTransport._force_close() #11462
  • bpo-35682: Fix _ProactorBasePipeTransport._force_close() #11462
  • [3.7] bpo-35682: Fix _ProactorBasePipeTransport._force_close() (GH-11462) #11463
  • [3.7] bpo-35682: Fix _ProactorBasePipeTransport._force_close() (GH-11462) #11463
  • [3.7] bpo-35682: Fix _ProactorBasePipeTransport._force_close() (GH-11462) #11463
  • [3.7] bpo-35682: Fix _ProactorBasePipeTransport._force_close() (GH-11462) #11463
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = <Date 2019-01-08.02:15:57.499>
    created_at = <Date 2019-01-08.00:50:48.656>
    labels = ['3.7', '3.8', 'OS-windows', 'expert-asyncio']
    title = 'asyncio: bug in _ProactorBasePipeTransport._force_close()'
    updated_at = <Date 2019-01-08.02:15:57.499>
    user = 'https://github.com/vstinner'

    bugs.python.org fields:

    activity = <Date 2019-01-08.02:15:57.499>
    actor = 'vstinner'
    assignee = 'none'
    closed = True
    closed_date = <Date 2019-01-08.02:15:57.499>
    closer = 'vstinner'
    components = ['Windows', 'asyncio']
    creation = <Date 2019-01-08.00:50:48.656>
    creator = 'vstinner'
    dependencies = []
    files = []
    hgrepos = []
    issue_num = 35682
    keywords = ['patch', 'patch', 'patch', 'patch']
    message_count = 3.0
    messages = ['333192', '333197', '333202']
    nosy_count = 8.0
    nosy_names = ['paul.moore', 'vstinner', 'tim.golden', 'asvetlov', 'zach.ware', 'yselivanov', 'steve.dower', 'miss-islington']
    pr_nums = ['11462', '11462', '11462', '11462', '11463', '11463', '11463', '11463']
    priority = 'normal'
    resolution = 'fixed'
    stage = 'resolved'
    status = 'closed'
    superseder = None
    type = None
    url = 'https://bugs.python.org/issue35682'
    versions = ['Python 3.7', 'Python 3.8']

    @vstinner
    Copy link
    Member Author

    vstinner commented Jan 8, 2019

    Running ProactorEventLoopTests.test_sendfile_close_peer_in_the_middle_of_receiving() logs a bug in _force_close(): see logs below. Extract of _force_close():

        def _force_close(self, exc):
            if self._empty_waiter is not None:
                if exc is None:
                    self._empty_waiter.set_result(None)
                else:
                    self._empty_waiter.set_exception(exc)
            ...

    Problem: _empty_waiter can be already done. For example, it can be created directly as done:

        def _make_empty_waiter(self):
            ...
            self._empty_waiter = self._loop.create_future()
            if self._write_fut is None:
                self._empty_waiter.set_result(None)
            return self._empty_waiter

    Attached PR fixes _force_close(): do nothing if _empty_waiter is already done.

    The regression comes from the following change:

    commit a19fb3c
    Author: Andrew Svetlov <andrew.svetlov@gmail.com>
    Date: Sun Feb 25 19:32:14 2018 +0300

    bpo-32622: Native sendfile on windows (bpo-5565)
    
    * Support sendfile on Windows Proactor event loop naively.
    

    Logs:

    vstinner@WIN C:\vstinner\python\master>python -X dev -m test test_asyncio -m test.test_asyncio.test_sendfile.ProactorEventLoopTests.test_sendfile_close_peer_in_the_middle_of_receiving
    Running Debug|x64 interpreter...
    Run tests sequentially 
    0:00:00 [1/1] test_asyncio 
    Exception in callback _ProactorReadPipeTransport._loop_reading(<_OverlappedF...events.py:452>) 
    handle: <Handle _ProactorReadPipeTransport._loop_reading(<_OverlappedF...events.py:452>) created at C:\vstinner\python\master\lib\asyncio\windows_events.py:82>
    source_traceback: Object created at (most recent call last):
      File "C:\vstinner\python\master\lib\test\test_asyncio\test_sendfile.py", line 125, in run_loop
        return self.loop.run_until_complete(coro)
      File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 576, in run_until_complete
        self.run_forever()
      File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 315, in run_forever
        super().run_forever()
      File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 544, in run_forever
        self._run_once()
      File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 1729, in _run_once
        event_list = self._selector.select(timeout)
      File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 421, in select
        self._poll(timeout)
      File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 750, in _poll
        f.set_exception(e)
      File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 82, in set_exception
        super().set_exception(exception)
    Traceback (most recent call last):
      File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 444, in finish_recv
        return ov.getresult()
    OSError: [WinError 64] The specified network name is no longer available
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 256, in _loop_reading
        data = fut.result()
      File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 748, in _poll
        value = callback(transferred, key, ov)
      File "C:\vstinner\python\master\lib\asyncio\windows_events.py", line 448, in finish_recv
        raise ConnectionResetError(*exc.args)
    ConnectionResetError: [WinError 64] The specified network name is no longer available
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "C:\vstinner\python\master\lib\asyncio\events.py", line 81, in _run
        self._context.run(self._callback, *self._args)
      File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 283, in _loop_reading
        self._force_close(exc)
      File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 118, in _force_close
        self._empty_waiter.set_exception(exc)
    asyncio.exceptions.InvalidStateError: invalid state

    == Tests result: SUCCESS ==

    1 test OK.

    Total duration: 531 ms
    Tests result: SUCCESS

    @vstinner vstinner added 3.7 (EOL) end of life 3.8 only security fixes OS-windows topic-asyncio labels Jan 8, 2019
    @vstinner
    Copy link
    Member Author

    vstinner commented Jan 8, 2019

    New changeset 80fda71 by Victor Stinner in branch 'master':
    bpo-35682: Fix _ProactorBasePipeTransport._force_close() (GH-11462)
    80fda71

    @miss-islington
    Copy link
    Contributor

    New changeset 88bd26a by Miss Islington (bot) in branch '3.7':
    bpo-35682: Fix _ProactorBasePipeTransport._force_close() (GH-11462)
    88bd26a

    @vstinner vstinner closed this as completed Jan 8, 2019
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 (EOL) end of life 3.8 only security fixes OS-windows topic-asyncio
    Projects
    None yet
    Development

    No branches or pull requests

    2 participants