Re-evaluate "http.response.zerocopysend"

[Archmonger]

Right now, none of the existing ASGI webservers support zerocopysend. This seems to be due to the additional code complexity needed to support this extension.

This lack of support has prevented me from adding zerocopysend to whitenoise, which would have been a big win for the Python web development community.

I'm opening this issue as a place to discuss what, if anything, can be done to encourage adoption for zerocopysend.

Here's a few questions to get this discussion started:

1. Is a zerocopysend2 interface needed to encourage adoption?
2. Does asgiref need some helper utilities to simplify adoption of the current extension?
3. It seems that adding zerocopysend would require facing the buffer. Is it worthwhile (or feasible) to engineer a way of avoiding this?

[Archmonger]

cc: @pgjones @Kludex @tomchristie

[Kludex]

The ones who know more about it are @synodriver and @abersheeran.

[abersheeran]

The reason for facing buffers is because of the design of asyncio. It seems possible to solve this problem if we modify the zerocopysend standard so that it can only be sent at the end of the message.

[Archmonger]

@abersheeran

In terms of what this would like in practice, do you have a proposed interface?

Trying to work backwards and see how/if to push through this issue.

Lack of zero copy send is pretty painful for Whitenoise and ReactPy.

[synodriver]

I have a fork of hypercorn with this extension, but it will only work under http1.1, since h2 doesn't provide this feature. As for facing the buffer, I personally think there's no problem because mine has been running for a long time.

[Archmonger]

@synodriver

Would an interface change allow h2 support without waiting on library support?

[synodriver]

@synodriver

Would an interface change allow h2 support without waiting on library support?

I don't think it's possible. hypercorn relys on those sans-io projects to parse raw data. You can send data directly, but that will break h2 library's internal state machine.

[Archmonger]

cc: @synodriver @pgjones @gi0baro

As I have recently adopted the maintenance burden of WhiteNoise via my approved fork ServeStatic, file streaming via ASGI webservers is now something I need to focus on.

Before I start shooting PRs into the dark, I'd like the SW leads of the most popular webservers to help brainstorm on what the path forward is here.

After revisiting this issue, my first thought was to effectively clone the WSGI FileWrapper extension within Uvicorn/Hypercorn/Granian. For example:

async def application(scope, receive, send):
    FileWrapper = scope["extensions"]["file_wrapper"]
    
    # Returning a FileWrapper allows the webserver to handle the file with whatever it feels is the best solution.
    return FileWrapper(open("example.css", "rb"), BLOCK_SIZE)

1. Does this FileWrapper interface make integration of this file handling solution any easier?
   • I'm assuming the answer is yes, since it could allow detection of file transmissions further down the stack & HTTP pipeline.
   • Perhaps I'm missing some knowledge here on why the zerocopysend interface was selected over this. This legacy interface seems to also allow for graceful fallbacks to pure-python file handling.
2. Are there any other ideas on adding file handling to ASGI frameworks in a way that is both performant and isn't painful to maintain for webservers?
3. Is it fair to assume the zerocopysend extension is dead to Uvicorn/Hypercorn/Granian due to ASGI webserver maintenance burden?

[tomchristie]

Is it fair to assume the zerocopysend extension is dead to Uvicorn/Hypercorn/Granian due to ASGI webserver maintenance burden?

Not necessarily, the previous PR on uvicorn wasn't going to be acceptable due to the behaviour changes it introduced. Probably just needs someone to step up to it again.

[gi0baro]

3. Is it fair to assume the zerocopysend extension is dead to Uvicorn/Hypercorn/Granian due to ASGI webserver maintenance burden?

Not really, but I can only speak in regards of Granian here.

Also I'd keep Granian out of the topic here, as:

• due to the huge difference in architecture when compared to other servers, moving file descriptors between the app and the server would be quite hard, and it's unclear what's the performance gain (if any)
• it already supports pathsend (introduced also because the inability to support zerocopysend), which even if less powerful in terms of features, produces a nice gain in performance (which it seems to be what you're looking for) that is probably higher of what you could get with Python anyway (and os.sendfile is only available on Unix).

Given these points, and also the fact Granian is quite young compared to other servers, I don't think zerocopysend will be part of the development priorities in the near future (but I'm open to any PRs for this).

In terms of the extension itself: I don't have any particular objection in how it was designed; I'd probably would only change the fact zerocopysend and response.body messages can be mixed, as it is not super clear to me what's the intended use-case for that, and it will probably simplify the overall state flow in servers willing to support this.

[tomchristie]

I'd probably would only change the fact zerocopysend and response.body messages can be mixed, as it is not super clear to me what's the intended use-case for that, and it will probably simplify the overall state flow in servers willing to support this.

Second guessing, I'd assume that the intent there was for zero-copy-send to be usable with Range requests.

[gi0baro]

I'd probably would only change the fact zerocopysend and response.body messages can be mixed, as it is not super clear to me what's the intended use-case for that, and it will probably simplify the overall state flow in servers willing to support this.

Second guessing, I'd assume that the intent there was for zero-copy-send to be usable with Range requests.

Isn't that the purpose of offset and count items in http.response.zerocopysend message? Why would you need to mix bytes from FDs with anything else? 🤔

[Archmonger]

Yes the zerocopysend user API does work for range requests, but unfortunately that ends up being a moot point. The problem is that the spec for zerocopysend is too invasive to integrate with existing webservers, which has resulted in nearly all of them not supporting this extension.

Worst case, I'd be okay with each webserver creating webserver-specific extensions outside the ASGI spec while the community decides what the best path forward is.

[tomchristie]

The problem is that the spec for zerocopysend is too invasive to integrate with existing webservers, which has resulted in nearly all of them not supporting this extension.

Yes.

while the community decides what the best path forward is.

I can't speak for "here's how it should be supported", tho I would suggest dropping or otherwise sidelining the extension as currently designed.