Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add scalability instructions to readme #450

Merged
merged 1 commit into from
Dec 13, 2023
Merged

Add scalability instructions to readme #450

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
11 changes: 11 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,17 @@ const getUserInfoList = async () => {
## Common Customization Scenarios
Feel free to fork this repository and make your own modifications to the UX or backend logic. For example, you may want to change aspects of the chat display, or expose some of the settings in `app.py` in the UI for users to try out different behaviors.

### Scalability
For apps published with `az webapp up` or from the Azure AI Studio, you can increase your app's ability to handle concurrent requests from multiple users with the following steps:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, you could also add a gunicorn.conf.py in the App src itself to dynamically compute the optimal number of workers, it's what we often do for our App Service apps, i.e.

https://github.com/Azure-Samples/azure-django-postgres-flexible-appservice/blob/246b1e3ad98b3c11f74682a4f82e9a18d3b58911/src/gunicorn.conf.py#L7

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, I will look into this!

1. Upgrade your App Service plan tier to a higher tier, for example tiers with more than one vCPU.

2. Configure the following app settings on your App Service in the Azure Portal:
- `PYTHON_ENABLE_GUNICORN_MULTIWORKERS`: true
- `PYTHON_GUNICORN_CUSTOM_WORKER_NUM`: 5 (may be higher or lower depending on your App Service Plan tier)
- `PYTHON_GUNICORN_CUSTOM_THREAD_NUM`: 5 (may be higher or lower depending on your App Service Plan tier)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I checked the Oryx Repo and i found this pull request microsoft/Oryx#1707
with a great explanation how this parameters are processed.

So maybe you could add a comment that the PYTHON_GUNICORN_CUSTOM_WORKER_NUM and `PYTHON_GUNICORN_CUSTOM_THREAD_NUM' are NOT a must. And in case someone is unsure what is the correct setting for his underlaying App Service Plan he should not add this settings. So the web app will use the gunicorn recommendation => (2 * numCores) + 1 for the number of workers and threads;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point! I discussed that in this PR where I made this setting default for the azd appservice Bicep files: Azure/azure-dev#2571
(But then I forgot that Oryx did the right thing)
So yeah if you're not already overriding the gunicorn command, then the best practice is to just set PYTHON_ENABLE_GUNICORN_MULTIWORKERS to true, and no other customization.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the Oryx PR it looks like the default thread count will be 1 if not set, but workers will default to (2 + numCores) + 1. I'll make a follow up PR to add more detail in the suggestions here, thanks folks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Follow up #451


After adding the settings, be sure to save the configuration and then restart your app.

### Debugging your deployed app
First, add an environment variable on the app service resource called "DEBUG". Set this to "true".

Expand Down