Different Routers with Dependency Conflicts for Scaling API

[brandonbiggs]

First Check

• I added a very descriptive title to this issue.
• I used the GitHub search to find a similar issue and didn't find it.
• I searched the FastAPI documentation, with the integrated search.
• I already searched in Google "How to X in FastAPI" and didn't find any information.
• I already read and followed all the tutorial in the docs and didn't find an answer.
• I already checked if it is not related to FastAPI but to Pydantic.
• I already checked if it is not related to FastAPI but to Swagger UI.
• I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

• I commit to help with one of those options 👆

Example Code

$ python3 example_1.py &
$ python3 example_2.py &
$ cat main.py
app = FastAPI()

from routers import example_1
app.include_router(example_1.router) # Is connecting to the example_1.py running in background

from routers import example_2
app.include_router(example_2.router) # Is connecting to the example_2.py running in background

Description

Hi all!

I have a question about scaling my API. The applications I'm trying to serve have very different needs. One app at /example_1 may need Tensorflow 1.4 but /example_2 needs Tensorflow 2.1. Unfortunately backwards compatibility won't work, especially as I add /example{3..n}. I am currently running into a big dependency issue (age old Python problem :D). So I have been exploring how to run each router in a separate "process" so they can access their own dependencies that don't conflict with each other. I'd like to run each out of a Docker container for ease of managing dependencies, but if I need to just use conda/pyenv that would work too.

I preferably don't want to run each router as it's own fastapi instance as I want to have all the docs, base urls, etc all out of the same place, but if I have to run each as their own fastapi instance I can. I don't know how to "scale" like this. I've tried googling/searching through the documentation, but I haven't been able to find anything helpful. I may also be looking in the wrong place, so any guidance would be greatly appreciated. Thanks.

Operating System

Linux, macOS

Operating System Details

No response

FastAPI Version

0.71.0

Python Version

3.8

Additional Context

No response

[jgould22]

How different is each version of this API?

Also how intermingled is the api, would you expect a client to be using both versions day to day or is one version strictly legacy and the other the new hotness?

If they are substantially different I would suggest just making your life easier and sticking a reverse proxy in front of it and hosting each application separately. Scaling is then just a matter of filtering on the path in your proxy and increasing the number of backends required.

[brandonbiggs]

How different is each version of this API?

Also how intermingled is the api, would you expect a client to be using both versions day to day or is one version strictly legacy and the other the new hotness?

Thanks for the reply! Sorry if I didn't phrase my question very well. So I'd like there to just be one API, but it would include different primary routes, each with different dependencies. So following along to your second question, I would expect a client to use both routes api.example.com/example1 and api.example.com/example2 as they have different purposes.

If they are substantially different I would suggest just making your life easier and sticking a reverse proxy in front of it and hosting each application separately. Scaling is then just a matter of filtering on the path in your proxy and increasing the number of backends required.

What do you mean with your last sentence here, increasing the number of backends?

I found Sub Applications - Mounts which is getting closer to what I want, but from my understanding this doesn't allow me to use different environments with different python dependencies and the docs aren't shared, I'd have to go to each mounted app's documentation link. The shared docs aren't absolutely necessary, just something I'd prefer.

Hopefully this adds some clarity. Thanks again.

[brandonbiggs]

This article seems to be achieving what I'm trying to do, so I'll probably look into this or a solution like this https://towardsdatascience.com/deploying-ml-models-in-production-with-fastapi-and-celery-7063e539a5db

[brandonbiggs]

I'm going to resolve my question, because I think I answered it myself, but I wanted to add one more link in case someone else comes across my question and doesn't quite know what to search for. FastAPI has Background Tasks which would solve some of my issues, but not quite all of them. But at the bottom of the page, FastAPI docs suggest using Celery too. So the answer was in the docs, I just didn't know how to find it since I didn't know exactly what I was looking for.