Persisting memory state between requests in FastAPI

[pvahabi]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from fastapi import FastAPI

class GlobalTest():
    __count = 1

    @classmethod
    def main(self):
        self.__count += 1
        return self.__count

app = FastAPI()
@app.get("/")
async def read_root():
    return GlobalTest.main()

Description

Open two browsers. Go to 127.0.0.1/docs and call this API a couple of times. You just got a counter over different requests. The static class is memorizing the value over different requests.

Is there any solution to this? Is there any other way to create static classes that will not persist over different requests? I think its definitely an issue, and potentially a security concern.

Thanks

Operating System

Linux

Operating System Details

No response

FastAPI Version

0.68.1

Python Version

3.9.7

Additional Context

No response

[JarroVGIT]

I don’t understand the issue, this is very much by design. If you don’t want to share state of a class, then don’t use static methods? This is what they are supposed to do, and is certainly not a security concern (unless you don’t know what you are doing, but then everything is a security concern).

Maybe it helps if you explain what you are trying to achieve using a classmethod, and we can give you some pointers?

edit:
a validation question came to mind: did you expect all request would create their own FastAPI()?

[zoliknemet]

@pvahabi

This is probably not the thing you want to do.
If you need some kind of state to be saved you should use database or redis or something else.

Your example should work, but only with 1 worker, as soon as you get 2 workers each will get his own memory space and you will have 2 GlobalTest classes and 2 __count properties so you will have 2 states stored.

Storing things is memory usually is bad because your service is not stateless anymore, you can not scale it and what happens with data if service restarts - you lose your data.

UPDATE:
In your case you can do it like this to get different counters for different clients calling it:

class GlobalTest:
    __count = {}

    @classmethod
    def main(self, identifier: str):
        if identifier not in self.__count:
            self.__count[identifier] = 1
        self.__count[identifier] += 1
        return self.__count[identifier]


@app.get("/")
async def read_root(request: Request):
    identifier = f"{request.client.host}:{request.client.port}"
    return GlobalTest.main(identifier)


if __name__ == "__main__":
    uvicorn.run(app)

This will work for different browser windows, but not for different tabs because the tabs in the same browser have the same HOST and PORT

[pvahabi]

Hey @JarroVGIT and @zoliknemet . Thanks for answering, let me give more insights.

Is there a way to be in a totally stateless service? This situation would never ever be possible in a server with PHP / Apache. There is no way that one request is changing the state of a class and it will be kept in memory. It is totally stateless.

The issue is caused by uvicorn workers being used for more than one request before they are recreated. Upon subsequent requests a single worker may have some memory state such as static or global variables carried over from previous requests (which is bad! its useless, and it make the service not anymore stateless). This could be avoided by setting uvicorn to one worker per request, but it would be too slow.

Any suggestion?

@zoliknemet your solution is not bad, but we have many programmers, and I would like to achieve a situation where there is no way that you can not be in a stateless situation.

@JarroVGIT did you expect all request would create their own FastAPI()? Not really. I'm just expecting that the "static classes internal variables" if changed will remain isolated to the current request (as in any fully stateless service, such as PHP/Apache)

ps: We have a more complicated code, this is just a dummy example.

[pvahabi]

@tiangolo would you be able to look at it? the example code on the top should be self explanatory of the issue.

[zoliknemet]

@pvahabi
do you know who is using your app, do you have some kind of authentication?
If you don't can you generate some kind of ID per session, some UUID on first connect to your app, than you will know each tab that is unique. Look at example:

from uuid import uuid4
import uvicorn as uvicorn
from fastapi import FastAPI, Header


app = FastAPI()


class GlobalTest:
    __count = {}

    @classmethod
    def main(self, identifier: str):
        if identifier not in self.__count:
            self.__count[identifier] = 1
        self.__count[identifier] += 1
        return self.__count[identifier]


@app.post("/anonymous-register")
async def read_root():
    return uuid4().hex


@app.get("/count")
async def read_root(unique_session_id: str = Header(...)):
    return GlobalTest.main(unique_session_id)


if __name__ == "__main__":
    uvicorn.run(app)

Now each time CLIENT connects, goes to /anonymous-register and gets UUID which represents him. For each other request just send that UUID in header

Try to give me more info about use case (why you need it)
Do you have any Frontend or is it just API?

[JarroVGIT]

Not really. I'm just expecting that the "static classes internal variables" if changed will remain isolated to the current request (as in any fully stateless service, such as PHP/Apache)

But you actually did expect it, as you didn't want to share state across requests. I don't know how to help to be honest, as this is normal behaviour for python based web services (both WSGI based such as Flask as ASGI based, such as Starlette and (by extension) FastAPI).

PHP, once designed as an templating mechanic, works differently and is reinterpreted per request. Starlette, not so much so, it is "running" while talking 'events' with uvicorn. Each worker gets its own process (and by that, its own instance of the application) so local state of the application is not shared across workers but is shared by the requests. It's basically how you build ASGI based applications, and I don't see any way around it (other then; just not use it). It is not a FastAPI issue though, maybe you have more luck in the Starlette repo? They know this stuff better than I do for sure :)

[Kludex]

Is there any solution to this? Is there any other way to create static classes that will not persist over different requests? I think its definitely an issue, and potentially a security concern.

It's not a security concern. You are running your application in a python process, so it's normal that your counter goes up.

This could be avoided by setting uvicorn to one worker per request, but it would be too slow.

That's not true. It cannot be avoided. Those are python processes. The application is not stateless because you're coding it as non-stateless, it's not a web framework or server issue, it's just an implementation issue.

I understand what you mean by this issue, but there's a misconception that the web framework (or server implementation) should prevent you from doing what you are able to do, but that's not the case. The worker is a python process, and objects are living there, if you change the value of an object, then that's it, the value will change.

[pvahabi]

@Kludex hey Marcelo thanks for the answer, however this is still an issue.

As an example look at this: https://stackoverflow.com/questions/67663970/optimal-way-to-initialize-heavy-services-only-once-in-fastapi

Think about heavy objects you want to initialize once. Now add an internal variable to the heavy objects.

Can you guarantee that multiple requests in parallel using the methods of this object will not have any interference between each others?

• Req 1
  while still executing (not finished yet) is changing multiple time an internal variable of the heavy object.

• Req 2
  while still executing (not finished yet) is reading an internal variable of the heavy object.

In this context, it seems to me the only way to be safe, its to create a new instance of any, literally any object per each request. Can you elaborate?

Thanks

[Kludex]

Can you elaborate?

I can try. :)

Can you guarantee that multiple requests in parallel using the methods of this object will not have any interference between each others?

I cannot guarantee that. But that's more related to how Python works, than the web framework. There's nothing we can do to prevent the user to do that.

Considering the scenario you proposed above, we can check this application:

import asyncio

from fastapi import FastAPI


app = FastAPI()


class HeavyObject:
    def __init__(self):
        self.value = 0


obj = HeavyObject()


@app.get("/")
async def home():
    task_id = id(asyncio.current_task())
    print(task_id, obj.value)
    obj.value += 1
    await asyncio.sleep(3)
    print(task_id, obj.value)

If you call this server multiple times in the time frame of 3 seconds, you're going to see that all of them are going to finish with the same obj.value. That's fine, it's meant to be like this.

What we can do to prevent something like this from happening is teaching people that this is the expected behavior when you change a global object. If that's not enough, you can use some technique to make those objects immutable, and work with the values you're interested in:

import asyncio
from dataclasses import dataclass

from fastapi import FastAPI


app = FastAPI()


@dataclass(frozen=True)
class HeavyObject:
    value: int


obj = HeavyObject(value=0)


@app.get("/")
async def home():
    value = obj.value
    task_id = id(asyncio.current_task())
    print(task_id, value)
    value += 1
    await asyncio.sleep(3)
    print(task_id, value)

[Kludex]

Thank you for the framework and all the work along the way - you are awesome!

I thank you for this, but there's no need to fight written violence with more of it.

I'm grateful for all the people that asked questions here. Most of the questions could have been on StackOverflow, but we actually tell people to create issues for questions (it's on our documentation). The reason for it is to reward the people that help.

There were many side effects in my life because of those questions: I learned a lot because I wanted to understand each question, and find the answer, even if they look easy now, they weren't when I started helping; and because of that, I grew my knowledge, and I was able to help other projects that help FastAPI, like Starlette and Uvicorn.

Let's try to see things from a different light. 🙏