Implement loop.sock_sendfile method

[asvetlov]

BPO	32410
Nosy	@giampaolo, @ned-deily, @asvetlov, @1st1
PRs	bpo-32410: Implement loop.sock_sendfile() #4976 bpo-32410: Make SendfileNotAvailableError public #5243 bpo-32410: Improve sock_sendfile tests #5294 bpo-32410: Avoid blocking on file IO in sendfile fallback code #7172 [3.7] bpo-32410: Avoid blocking on file IO in sendfile fallback code (GH-7172) #7182

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-01-24.19:42:42.123>
created_at = <Date 2017-12-22.13:07:13.056>
labels = ['3.7', 'library', 'expert-asyncio']
title = 'Implement loop.sock_sendfile method'
updated_at = <Date 2018-05-29.01:42:47.417>
user = 'https://github.com/asvetlov'

bugs.python.org fields:

activity = <Date 2018-05-29.01:42:47.417>
actor = 'ned.deily'
assignee = 'none'
closed = True
closed_date = <Date 2018-01-24.19:42:42.123>
closer = 'asvetlov'
components = ['Library (Lib)', 'asyncio']
creation = <Date 2017-12-22.13:07:13.056>
creator = 'asvetlov'
dependencies = []
files = []
hgrepos = []
issue_num = 32410
keywords = ['patch']
message_count = 7.0
messages = ['308932', '310098', '310279', '310521', '310624', '317936', '317973']
nosy_count = 4.0
nosy_names = ['giampaolo.rodola', 'ned.deily', 'asvetlov', 'yselivanov']
pr_nums = ['4976', '5243', '5294', '7172', '7182']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue32410'
versions = ['Python 3.7']

[asvetlov]

The method should be low-level coroutine, implemented on loops with native sendfile support only (UnixEventLoop).

The signature should be close to socket.sendfile() method

Next step is implementing loop.sendfile(transport, ...) with fallback to sending by chunks if sendfile is not supported.

[asvetlov]

New changeset 6b5a279 by Andrew Svetlov in branch 'master':
bpo-32410: Implement loop.sock_sendfile() (bpo-4976)
6b5a279

[asvetlov]

New changeset 7464e87 by Andrew Svetlov in branch 'master':
bpo-32410: Make SendfileNotAvailableError exception public (bpo-5243)
7464e87

[1st1]

Andrew, test_cancel2 test fails 1 out of 10 times on my machine (log below). Could you please take a look?

Also, can you rename sendfile tests to 'test_sock_sendfile_*'?

{pydev} ~/d/p/cpython (master %) » ./python.exe -m test test_asyncio -v -m test_cancel2
== CPython 3.7.0a4+ (heads/master:9d411c119f, Jan 23 2018, 15:12:10) [Clang 9.0.0 (clang-900.0.39.2)]
== Darwin-17.3.0-x86_64-i386-64bit little-endian
== cwd: /Users/yury/dev/pydev/cpython/build/test_python_65406
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
Run tests sequentially
0:00:00 load avg: 1.80 [1/1] test_asyncio
test_cancel2 (test.test_asyncio.test_unix_events.SelectorEventLoopUnixSockSendfileTests) ... ERROR

======================================================================
ERROR: test_cancel2 (test.test_asyncio.test_unix_events.SelectorEventLoopUnixSockSendfileTests)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Users/yury/dev/pydev/cpython/Lib/test/test_asyncio/test_unix_events.py", line 487, in cleanup
    proto.transport.close()
AttributeError: 'NoneType' object has no attribute 'close'

---

Ran 1 test in 0.010s

FAILED (errors=1)
Task was destroyed but it is pending!
task: <Task pending coro=<BaseSelectorEventLoop._accept_connection2() done, defined at /Users/yury/dev/pydev/cpython/Lib/asyncio/selector_events.py:200> wait_for=<Future pending cb=[<TaskWakeupMethWrapper object at 0x10731c2e8>()]>>
test test_asyncio failed
test_asyncio failed

[asvetlov]

New changeset 0a5e71b by Andrew Svetlov in branch 'master':
bpo-32410: Improve sock_sendfile tests (bpo-5294)
0a5e71b

[1st1]

New changeset 7165754 by Yury Selivanov in branch 'master':
bpo-32410: Avoid blocking on file IO in sendfile fallback code (GH-7172)
7165754

[ned-deily]

New changeset 420092e by Ned Deily (Miss Islington (bot)) in branch '3.7':
bpo-32410: Avoid blocking on file IO in sendfile fallback code (GH-7172) (bpo-7182)
420092e