FastAPI return empty string in time consuming task

[rxy1212]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

import os
import subprocess
from multiprocessing import Pool

import uvicorn
from fastapi import FastAPI
from pydantic import BaseModel


class Param(BaseModel):
    shell_path: str

# time consuming task ！！！
def run(param: Param):
    subprocess.run(f'sh {param.shell_path}', shell=True)
    return 0


app = FastAPI()
pool = Pool(8)


@app.post('/faiss/action')
def yjy(param: Param):
    if not os.path.exists(param.shell_path):
        return {'code': 1, 'message': f'{param.shell_path} file not found!'}
    try:
        res = pool.map(run, [param])
    except Exception as e:
        return {'code': 2, 'message': f'shell execute error! {e}'}
    code = res[0]
    return {'code': code, 'message': 'success.'}


if __name__ == '__main__':
    uvicorn.run(app, host="0.0.0.0", port=16636)

Description

I try to build a time consuming task server with fastapi, But I found a strange question.
When I request my api /faiss/action with requests, fastapi return a empty string to requests after about 5 to 10mins.
At the begenning, it works right, requests block my client code (that's what i want), but after 5 to 10mins, fastapi just return a empty string to my client. That leads to a early end of my client, at the same time my time consuming task still run in server.
It's there a timeout mechanism in fastapi or How can I resolve this question?

Operating System

Linux

Operating System Details

No response

FastAPI Version

0.68.1

Python Version

3.6.8

Additional Context

No response

[STeveShary]

@rxy1212 The issue is that you need to make your long running subprocess async. If you look at the example here: https://docs.python.org/3/library/asyncio-subprocess.html, you will see the await part. That will allow your other requests to be run in parallel.

[rxy1212]

@rxy1212 The issue is that you need to make your long running subprocess async. If you look at the example here: https://docs.python.org/3/library/asyncio-subprocess.html, you will see the await part. That will allow your other requests to be run in parallel.

Thanks for your reply. But I don't think "make long running subprocess async" can solve my problem. I write a sync edition, and the problem still exist. My sync code:

import os
import subprocess

import uvicorn
from fastapi import FastAPI
from pydantic import BaseModel


class Param(BaseModel):
    shell_path: str


app = FastAPI()


@app.post('/faiss/action')
def create_item(param: Param):
    if not os.path.exists(param.shell_path):
        return {'code': 1, 'message': f'{param.shell_path} file not found!'}
    try:
        # long running task
        subprocess.run(f'sh {param.shell_path}', shell=True)
    except Exception as e:
        return {'code': 2, 'message': f'{e}'}
    return {'code': 0, 'message': 'success.'}


if __name__ == '__main__':
    uvicorn.run('server_v4:app', host="0.0.0.0", port=16636)

Now, I guess this is a bug in fastapi. When endpoint process a long running task, it will return early. The task still running in server, but return a empty string to client. Is there any way to prevent this problem?

[rxy1212]

This is a bug in fastapi, I do the same thing in flask, it works well and no early return. How to report bug to fastapi?

[Kludex]

fastapi return a empty string to requests after about 5 to 10mins.

Can you do the request with a parameter that allows verbosity? If curl, just add a -v. Also, show us the script you're running, if you don't mind.

This is a bug in fastapi, I do the same thing in flask, it works well and no early return.

It's not because it works in flask that it's a bug on FastAPI.

How to report bug to fastapi?

You already did. Creating the issue is the first step.

[rxy1212]

@Kludex Thanks for comment. I'm pretty sure this is a bug in fastapi.
You can reproduce this bug easily use code blow:

import time
import datetime

import uvicorn
from fastapi import FastAPI


app = FastAPI()


@app.post('/test')
def test():
    now = datetime.datetime.now()
    print(f"{now.hour}:{now.minute}:{now.second}")
    time.sleep(1800)
    return {'code': 0, 'message': 'success.'}


if __name__ == '__main__':
    uvicorn.run('server_v6:app', host="0.0.0.0", port=16636)

This code just wait for 30 mins and return a dict. But in fastapi, the client will get a empty string after 10 mins(precisely, i checked) and disconnect, meanwhile the code in server still running until 30 mins (1800 s). This makes no sense and no doc mention this situation or how to prevent it.

[Kludex]

I'm pretty sure this is a bug in fastapi.

It's highly unlikely to be a bug on FastAPI.

Can you do the request with a parameter that allows verbosity? If curl, just add a -v.

It looks like the client is timing out, not the server.

[rxy1212]

@Kludex

Can you do the request with a parameter that allows verbosity? If curl, just add a -v.

Sorry, I don't know how to operate. Can you please do this with the example above ?

It looks like the client is timing out, not the server.

I request fastapi with requests.post and not set timeout param, thus, client will not timeout.
I do the same thing in flask, reqeuests.post don't timeout too.

[Kludex]

I've run your code, just replacing 1800 by 660 (as I didn't want to wait, and it didn't matter), and everything looks normal.

(base) ➜  Development time curl -v -X POST localhost:16636/test
*   Trying 127.0.0.1:16636...
* Connected to localhost (127.0.0.1) port 16636 (#0)
> POST /test HTTP/1.1
> Host: localhost:16636
> User-Agent: curl/7.71.1
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< date: Fri, 10 Sep 2021 10:41:40 GMT
< server: uvicorn
< content-length: 31
< content-type: application/json
< 
* Connection #0 to host localhost left intact
{"code":0,"message":"success."}
curl -v -X POST localhost:16636/test 0,02s user 0,02s system 0% cpu 11:00,12 total

On the server side:

(base) ➜  Development python test.py
INFO:     Will watch for changes in these directories: ['/home/marcelo/Development']
INFO:     Uvicorn running on http://0.0.0.0:16636 (Press CTRL+C to quit)
INFO:     Started reloader process [92115] using watchgod
INFO:     Started server process [92125]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
12:41:41
INFO:     127.0.0.1:45312 - "POST /test HTTP/1.1" 200 OK

I request fastapi with requests.post and not set timeout param, thus, client will not timeout.
I do the same thing in flask, reqeuests.post don't timeout too.

Can you paste it here?

What's your uvicorn version?

[STeveShary]

Thanks for your reply. But I don't think "make long running subprocess async" can solve my problem. I write a sync edition, and the problem still exist.

That actually is the exact problem. If we take your example problem and change it to async, it will completely go away:

import asyncio
import time
import datetime

import uvicorn
from fastapi import FastAPI


app = FastAPI()


@app.post('/test')
async def test():
    now = datetime.datetime.now()
    print(f"{now.hour}:{now.minute}:{now.second}")
    await asyncio.sleep(1800)
    return {'code': 0, 'message': 'success.'}


if __name__ == '__main__':
    uvicorn.run('server_v6:app', host="0.0.0.0", port=16636)

NOTE: @Kludex is also correct in looking at a client. Most HTTP clients will not wait a full 5 - 10 minutes for a response. Make sure to extend your client timeout beyond 5 - 10 minutes.

[Kludex]

Being async or not shouldn't have any relevance to the current issue.

[STeveShary]

Sorry about that. @Kludex it seems like you understand the issue more.

[rxy1212]

Can you paste it here?

import json
import requests


if __name__ == '__main__':
    host = 'http://ip:16636/test'
    res = requests.post(host).text
    print(res)

What's you uvicorn version?

0.14.0

[ghandic]

I agree with @Kludex since you're using the requests module see the docs link below.

https://docs.python-requests.org/en/master/user/quickstart/#timeouts

If that doesn't work I'll try to recreate