sudo apt-get update
sudo apt-get install -y python3-pip nginx
sudo vim /etc/nginx/sites-enabled/fastapi_nginx
server {
listen 80;
server_name <EC2_IP_ADRESS>;
location / {
proxy_pass http://127.0.0.1:8000;
}
}
Then, save the file.
Next, edit nginx max body size with:
vim /etc/nginx/nginx.conf
http {
...
client_max_body_size 100M;
}
Restart the nginx server
sudo service nginx restart
If you see the following error:
Job for nginx.service failed because the control process exited with error code.
See "systemctl status nginx.service" and "journalctl -xeu nginx.service" for details.
You problaby did something wrong in nginx configurations
to get requirements in your project
pip freeze > requirements.txt
In EC2, do the setup:
pip install virtualenv
In project dir do:
virtualenv venv
virtualenv -p path_to_python venv
source venv/bin/activate
to install all the requirements:
$ pip install -r requirements.txt
if your process is killed, probably works:
$ pip install -r requirements.txt --no-cache-dir
and to check:
$ pip freeze
to endup your venv:
deactivate
To kill a port proccess:
sudo kill -9 `sudo lsof -t -i:8000`
FastApi - Respondendo com HTML: https://www.youtube.com/watch?v=ntOYtLoRgEQ&ab_channel=PSenna
Subindo um servidor FasAPI num EC2
Using FastAPI to deploy Machine Learning models
Why and How to make a Requirements.txt
Limit File Upload Size in NGINX
Build an AI-driven SaaS Application: FULLSTACK Tutorial with Python, React, and AWS
https://datatonic.com/insights/deploying-machine-learning-models-google-cloud/
Deploying models in Vertex AI What is vertex AI?
https://www.amplemarket.com/blog/how-to-deploy-machine-learning-microservice-to-google-cloud-run
https://www.youtube.com/watch?v=vieoHqt7pxo
Serving machine learning models with FastAPI: It’s not all about speed
FastAPI documentation about container
https://stackoverflow.com/questions/63169865/how-to-do-multiprocessing-in-fastapi
As we saw above, Whisper is fairly easy to install. However it requires advanced hardware. A GPU is recommended if you want to use the large version of the model.
If you use the whisper Python lib (see above) you will need around 10GB of RAM and 11GB of VRAM. It means that in practice you will need a 16GB GPU at least. It could be a NVIDIA Tesla T4 for example, or an NVIDIA A10.
On a Tesla T4, you will transcribe 30 seconds of audio in around 6 seconds. Performance Considerations
If you want to improve the default performance mentioned above, here are several strategies you can explore:
• Use a higher end GPU. For example you will get a better response time with GPUs using the Ampere platform like A10, A40, or A100.
• Work on batch inference in order to improve the throughput
• Leverage XLA compilation with Tensorflow or Jax
• Export the model to ONNX or TensorRT, and then serve it through the NVIDIA Triton Inference Server