Add `os.sendfile` and `os.splice` support

[laurivosandi]

Basically os.sendfile can be used to copy bytes from file descriptor to socket a'la to serve static files and os.splice can be used to copy bytes from pipe to socket. In both cases copying bytes to/from userspace Python process are bypassed.

This is just example snippet what I would like to do with Sanic:

from prometheus_client import Counter

counter_streamed_bytes = Counter(
    "backup_service_streamed_bytes",
    "Bytes transmitted")
counter_finished_streams = Counter(
    "backup_service_finished_streams",
    "Seemingly successfully finished streamings")
    ["exitcode"])

class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header('Content-type', 'application/gzip')
        self.end_headers()
        cmd = "dd", "if=/dev/urandom", "bs=1024", "count=1024"
        proc = Popen(cmd, stdout=PIPE, stdin=PIPE)
        while True:
            bytes_sent = os.splice(proc.stdout.fileno(), self.wfile.fileno(), 65536)
            if not bytes_sent:
                break
            counter_streamed_bytes.inc(bytes_sent)
        proc.stdin.close()
        while True:
            buf = proc.stdout.read()
            if not buf:
                break
        proc.stdout.close()
        proc.wait()
        counter_finished_streams.labels(proc.returncode).inc()

[ahopkins]

Let me make sure I understand you correctly...

        self.send_response(200)
        self.send_header('Content-type', 'application/gzip')
        self.end_headers()

In this snippet, I am assuming you want to first:

• Send the HTTP first line
• Send some headers
• Wait until this is done

Is that correct? Did you read about response streaming in the docs?

@app.route("/")
async def test(request):
    response = await request.respond(content_type="text/csv")
    await response.send("foo,")
    await response.send("bar")

    # Optionally, you can explicitly end the stream by calling:
    await response.eof()

Next, seems like you are trying to track the bytes that are sent. You can do that with signals.

from sanic.signals import Event

app = Sanic(__name__)


@app.signal(Event.HTTP_LIFECYCLE_SEND)
async def sent_data(data: bytes):
    print(data)

Am I correct in assuming that the ultimate goal is to report on the amount of data transmitted to prometheus?

[laurivosandi]

Am I correct in assuming that the ultimate goal is to report on the amount of data transmitted to prometheus?

What you pointed out is obvious and irrelevant to the actual problem statement of adding os.sendfile and os.splice support to Sanic framework. These are useful syscalls to bypass copying bytes to/from Python userspace making the overall application significantly more performant.

The code snippet was to illustrate what I did try and what worked without Sanic and what I would like to do with Sanic, but could not find easy way to do.

The os.sendfile seems to be present in asyncio module here

However it seems os.splice hasn't been wrapped yet, but some asyncio stuff has been done here

[laurivosandi]

In other words this can be replaced with sendfile by the event loop

For splice I don't have a quick fix, it's a bit more esoteric syscall anyway

[ahopkins]

Working on a POC of this to see how it would work. One of the things on my todo list has been a better method of serving files. It has always bothered me that there are two methods. This might be a good path towards dropping a dependency also.

[ahopkins]

NotImplementedError

Looks like uvloop does not support sendfile.

[Tronic]

I don't see an easy way to implement this. If the file was mmapped, perhaps one copy would be avoided (reading directly from kernel memory) but there still are likely some that cannot be (especially when TLS is being used). All file I/O needs to occur in a separate thread to allow other async tasks to run within the worker process, and the chunks being transferred need to be fairly large (upwards of 1 MB) such that memcpy becomes the main bottle neck instead of Python or threads. Using mmap has the disadvantage that the process dies badly (on a signal) if there is any disk or filesystem error.

[ahopkins]

I am going to close this as the lack of uvloop support is a complete non-starter. If you would like to get it running in a custom protocol for your use, that is 100% why the API allows it.

[ahopkins]

Been putting more thought into this. I have an idea that I'm going to try over the weekend. Will report back.