Chunking UploadFile: read(byte_count) awaits until upload is fully complete

[dorinclisu]

First check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.
• After submitting this, I commit to one of:
  • Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
  • I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
  • Implement a Pull Request for a confirmed bug.

Example

Here's a self-contained, minimal, reproducible, example with my use case:

import logging

from fastapi import FastAPI
from fastapi import File, UploadFile


app = FastAPI()

@app.post('/video')
async def upload_video(
    video: UploadFile = File(...)
):
    chunk = await video.read(100)
    logging.warning('Read the first 100 bytes')

    remaining = await video.read()
    logging.warning('Read all the file')

Environment

• OS: Linux
• FastAPI Version: 0.63.0

Description

Just run the example and try uploading a large file (100MB or above) from Swagger UI.
The expected behavior is that the first bytes are read quickly, before the upload is complete.

Unfortunately, the behavior I see is that it takes a rather long time for the whole file to upload, and only then the first chunked read is available, immediately followed by the second read. This kind of defeats the whole purpose of chunked read.

Is it not possible to start reading the file before upload is complete? The reason I'm asking is that I'd like to start uploading to s3 in parallel, to shorten the response wait time.

[Dustyposa]

You can solve it in here

[dorinclisu]

@Dustyposa I am not sure what exactly you refer to. The linked issue mentions 3 aspects:

• UploadFile - this is what I am already doing and not working well
• StreamingResponse - out of scope
• Using Request directly - https://fastapi.tiangolo.com/advanced/using-request-directly/

Assuming you mean the 3rd, the example is:

async for chunk in request.stream():
    ...

Does this mean that I need to do the form decoding myself, figure out where the file data starts and then handle the rest?

[Dustyposa]

Yes, u should use stream for your demand.

[hall-b]

@dorinclisu did you manage to do the form decoding yourself? If yes, could you share your code snippet ?
I have the exact same issue..