FastAPI with async background tasks + Prometheus + Grafana

This project provides a way to setup and run asynchronous (async/await) tasks using these libraries:

• Background
• ARQ
• SAQ
• FastStream
• Celery

This way can also be useful for testing your code with eager execution. You can see it in tests/conftest.py.

Finally, it shows how to monitor FastAPI app and execution of tasks with Prometheus + Grafana using these libraries:

• Prometheus-fastapi-instrumentator (web app)
• Prometheus-async (arq, saq, faststream)
• Flower (celery)

---

Installation

Clone the repository:

git clone https://github.com/ipakeev/fastapi-async-tasks

---

Usage

Create .env file and adjust it:

cp example.env .env

Build app image:

docker build -t fastapi-async-tasks .

Run all containers:

docker-compose up

---

Now you have access to these services:

• FastAPI: http://localhost:8000/docs
• Prometheus: http://localhost:9090/
• Grafana: http://localhost:3000/
• Flower: http://localhost:5555/
• SAQ Dashboard: http://localhost:8082/

Raw metrics are available here:

• FastAPI: http://localhost:8000/metrics
• ARQ: http://localhost:8001/metrics
• SAQ: http://localhost:8002/metrics
• FastStream: http://localhost:8003/metrics
• Celery: http://localhost:5555/metrics

---

Benchmark

You can test the performance of the application and run a benchmark using locust:

env endpoint={endpoint} locust --config locust/master.conf -f locust/benchmark.py -u 10 -i 1000

env endpoint={endpoint} locust --config locust/master.conf -f locust/background.py -u 10 -i 1000

Available {endpoint}s:

• io/simple
• io/sync
• io/thread
• cpu/simple
• cpu/process

---