-
-
Notifications
You must be signed in to change notification settings - Fork 6.3k
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Persisting memory state between requests in FastAPI #5045
Comments
I don’t understand the issue, this is very much by design. If you don’t want to share state of a class, then don’t use static methods? This is what they are supposed to do, and is certainly not a security concern (unless you don’t know what you are doing, but then everything is a security concern). Maybe it helps if you explain what you are trying to achieve using a classmethod, and we can give you some pointers? edit: |
This is probably not the thing you want to do. Your example should work, but only with 1 worker, as soon as you get 2 workers each will get his own memory space and you will have 2 Storing things is memory usually is bad because your service is not stateless anymore, you can not scale it and what happens with data if service restarts - you lose your data. UPDATE: class GlobalTest:
__count = {}
@classmethod
def main(self, identifier: str):
if identifier not in self.__count:
self.__count[identifier] = 1
self.__count[identifier] += 1
return self.__count[identifier]
@app.get("/")
async def read_root(request: Request):
identifier = f"{request.client.host}:{request.client.port}"
return GlobalTest.main(identifier)
if __name__ == "__main__":
uvicorn.run(app) This will work for different browser windows, but not for different tabs because the tabs in the same browser have the same HOST and PORT |
Hey @JarroVGIT and @zoliknemet . Thanks for answering, let me give more insights. Is there a way to be in a totally stateless service? This situation would never ever be possible in a server with PHP / Apache. There is no way that one request is changing the state of a class and it will be kept in memory. It is totally stateless. The issue is caused by uvicorn workers being used for more than one request before they are recreated. Upon subsequent requests a single worker may have some memory state such as static or global variables carried over from previous requests (which is bad! its useless, and it make the service not anymore stateless). This could be avoided by setting uvicorn to one worker per request, but it would be too slow. Any suggestion? @zoliknemet your solution is not bad, but we have many programmers, and I would like to achieve a situation where there is no way that you can not be in a stateless situation. @JarroVGIT did you expect all request would create their own FastAPI()? Not really. I'm just expecting that the "static classes internal variables" if changed will remain isolated to the current request (as in any fully stateless service, such as PHP/Apache) ps: We have a more complicated code, this is just a dummy example. |
@tiangolo would you be able to look at it? the example code on the top should be self explanatory of the issue. |
@pvahabi from uuid import uuid4
import uvicorn as uvicorn
from fastapi import FastAPI, Header
app = FastAPI()
class GlobalTest:
__count = {}
@classmethod
def main(self, identifier: str):
if identifier not in self.__count:
self.__count[identifier] = 1
self.__count[identifier] += 1
return self.__count[identifier]
@app.post("/anonymous-register")
async def read_root():
return uuid4().hex
@app.get("/count")
async def read_root(unique_session_id: str = Header(...)):
return GlobalTest.main(unique_session_id)
if __name__ == "__main__":
uvicorn.run(app) Now each time CLIENT connects, goes to /anonymous-register and gets UUID which represents him. For each other request just send that UUID in header Try to give me more info about use case (why you need it) |
But you actually did expect it, as you didn't want to share state across requests. I don't know how to help to be honest, as this is normal behaviour for python based web services (both WSGI based such as Flask as ASGI based, such as Starlette and (by extension) FastAPI). PHP, once designed as an templating mechanic, works differently and is reinterpreted per request. Starlette, not so much so, it is "running" while talking 'events' with uvicorn. Each worker gets its own process (and by that, its own instance of the application) so local state of the application is not shared across workers but is shared by the requests. It's basically how you build ASGI based applications, and I don't see any way around it (other then; just not use it). It is not a FastAPI issue though, maybe you have more luck in the Starlette repo? They know this stuff better than I do for sure :) |
It's not a security concern. You are running your application in a python process, so it's normal that your counter goes up.
That's not true. It cannot be avoided. Those are python processes. The application is not stateless because you're coding it as non-stateless, it's not a web framework or server issue, it's just an implementation issue. I understand what you mean by this issue, but there's a misconception that the web framework (or server implementation) should prevent you from doing what you are able to do, but that's not the case. The worker is a python process, and objects are living there, if you change the value of an object, then that's it, the value will change. |
@Kludex hey Marcelo thanks for the answer, however this is still an issue. As an example look at this: https://stackoverflow.com/questions/67663970/optimal-way-to-initialize-heavy-services-only-once-in-fastapi Think about heavy objects you want to initialize once. Now add an internal variable to the heavy objects. Can you guarantee that multiple requests in parallel using the methods of this object will not have any interference between each others?
In this context, it seems to me the only way to be safe, its to create a new instance of any, literally any object per each request. Can you elaborate? Thanks |
I can try. :)
I cannot guarantee that. But that's more related to how Python works, than the web framework. There's nothing we can do to prevent the user to do that. Considering the scenario you proposed above, we can check this application: import asyncio
from fastapi import FastAPI
app = FastAPI()
class HeavyObject:
def __init__(self):
self.value = 0
obj = HeavyObject()
@app.get("/")
async def home():
task_id = id(asyncio.current_task())
print(task_id, obj.value)
obj.value += 1
await asyncio.sleep(3)
print(task_id, obj.value) If you call this server multiple times in the time frame of 3 seconds, you're going to see that all of them are going to finish with the same What we can do to prevent something like this from happening is teaching people that this is the expected behavior when you change a global object. If that's not enough, you can use some technique to make those objects immutable, and work with the values you're interested in: import asyncio
from dataclasses import dataclass
from fastapi import FastAPI
app = FastAPI()
@dataclass(frozen=True)
class HeavyObject:
value: int
obj = HeavyObject(value=0)
@app.get("/")
async def home():
value = obj.value
task_id = id(asyncio.current_task())
print(task_id, value)
value += 1
await asyncio.sleep(3)
print(task_id, value) |
I thank you for this, but there's no need to fight written violence with more of it. I'm grateful for all the people that asked questions here. Most of the questions could have been on StackOverflow, but we actually tell people to create issues for questions (it's on our documentation). The reason for it is to reward the people that help. There were many side effects in my life because of those questions: I learned a lot because I wanted to understand each question, and find the answer, even if they look easy now, they weren't when I started helping; and because of that, I grew my knowledge, and I was able to help other projects that help FastAPI, like Starlette and Uvicorn. Let's try to see things from a different light. 🙏 |
This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
First Check
Commit to Help
Example Code
Description
Open two browsers. Go to 127.0.0.1/docs and call this API a couple of times. You just got a counter over different requests. The static class is memorizing the value over different requests.
Is there any solution to this? Is there any other way to create static classes that will not persist over different requests? I think its definitely an issue, and potentially a security concern.
Thanks
Operating System
Linux
Operating System Details
No response
FastAPI Version
0.68.1
Python Version
3.9.7
Additional Context
No response
The text was updated successfully, but these errors were encountered: