From 52f2d049beee45bf853008b5fded34d233ea6109 Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Tue, 19 Nov 2024 15:36:40 +0100 Subject: [PATCH 1/3] docs(srv): update doc on container concurrency MTA-5283 --- .../containers-concurrency.mdx | 58 +++++++++++++++++-- 1 file changed, 54 insertions(+), 4 deletions(-) diff --git a/serverless/containers/reference-content/containers-concurrency.mdx b/serverless/containers/reference-content/containers-concurrency.mdx index 8a8ad7c9c5..1048f6ce89 100644 --- a/serverless/containers/reference-content/containers-concurrency.mdx +++ b/serverless/containers/reference-content/containers-concurrency.mdx @@ -16,18 +16,68 @@ categories: ## Concurrency overview -Concurrency determines the number of incoming requests a single instance of a container can process before scaling up. Serverless Containers autoscale according to the number of instances needed to handle the incoming workload. +Concurrency determines the number of incoming requests a single instance of a container can process before scaling up. Serverless Containers automatically scale according to the number of instances needed to handle the incoming workload. A higher number of instances processing requests at the same time implies a greater usage of memory and [vCPU](/serverless/containers/concepts/#vcpu), and consequently a higher cost. -## Maximum concurrent requests per instance +``` + Concurrency = 1 + + ┌─────────────┐ + │ ┌──────┐│ + │ │ │ ││ + └─────┼───► │ ││ Concurrency = 80 + │ │ ││ + │ └──────┘│ ┌─────────────┐ + │ ┌──────┐│ │ │ ┌──────┐│ + │ │ ││ └────┼───► │ ││ +3 requests ──────┼───► │ ││ 3 requests ─────┼───► │ ││ + │ │ ││ ┌────┼───► │ ││ + │ └──────┘│ │ │ └──────┘│ + │ ┌──────┐│ └─────────────┘ + │ │ ││ 1 service + ┌─────┼───► │ ││ 1 container instance + │ │ │ ││ + │ └──────┘│ + └─────────────┘ + 1 service + 3 container instances +``` -When [deploying a container](/serverless/containers/how-to/deploy-container/), Scaleway Serverless Containers allows you to edit the **Maximum concurrent requests per instance** parameter. +## Concurrency setting -Serverless Containers apply the highest possible value by default: **80 concurrent requests**. We recommend using the default value, as Serverless Containers are designed to efficiently function with it, but you can lower it to better fit specific requirements. +When [deploying a container](/serverless/containers/how-to/deploy-container/), Scaleway Serverless Containers allows you to configure the [concurrency](/serverless/containers/concepts/#concurrency), which is the maximum number of requests a single container instance can handle at the same time. The concurrency setting ranges from 1 to 1,000 simultaneous instances. + +By default, Serverless Containers allow **80 concurrent requests** per container instance. We recommend using the default value, as Serverless Containers are designed to efficiently function with it, but you can lower it to better fit specific requirements. + +## Impact of concurrency on container scaling + +If the number of incoming requests exceeds the set concurrency value for a container, Serverless Containers automatically creates new container instances to handle the additional load. + +For example, if a container is set to handle 10 concurrent requests, and 50 requests or events arrive at the same time, Serverless Containers will create at least 5 container instances to handle the incoming load. + +## Benefits of concurrency + +**Efficiency**: Setting a higher concurrency value allows a single container instance to handle more requests, which can reduce the total number of container instances, therefore reducing costs. + +**Responsiveness**: Setting a lower concurrency (down to 1 concurrent container instance) allows to reduce latency for individual requests, since the container focuses on fewer tasks at a time. However, + +## Implications for your service + +**Compute-bound workloads**: Workloads relying on heavy CPU or memory usage per request benefit from lower concurrency (e.g., 1 or 2) to avoid resource contention within the container. + +**I/O-bound workloads**: Workloads relying on smaller input/output operations, such as waiting for external APIs or databases benefit from higher concurrency, since the container can handle multiple requests while waiting on I/O. ## Examples +### Low concurrency + If your Serverless Container hosts a resource-intensive application that can only handle a small number of requests with the provisioned memory and vCPU, you can reduce the number of maximum concurrent requests per instance to scale quicker and avoid bottlenecks and queued requests. You can set the **Maximum concurrent requests per instance** of your container to `1` if it is designed to handle a single request at a time. However, a low maximum concurrency value may affect the performance of your deployment, as several instances of your container will start if it receives a high number of concurrent requests. + +### High concurrency + +If your Serverless Container hosts an application that receives a high number of requests that are easily processed with the provisioned memory and vCPU, you can set a greater number of maximum concurrent requests per instance. + +By setting a greater concurrency value than the average number of concurrent requests your container can handle, it will only scale up in the event of a spike in received requests. \ No newline at end of file From cdcc13fe6d0111a7f0a709cc26418a84150ee219 Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Thu, 21 Nov 2024 10:37:15 +0100 Subject: [PATCH 2/3] Update serverless/containers/reference-content/containers-concurrency.mdx Co-authored-by: nerda-codes <87707325+nerda-codes@users.noreply.github.com> --- .../containers/reference-content/containers-concurrency.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/serverless/containers/reference-content/containers-concurrency.mdx b/serverless/containers/reference-content/containers-concurrency.mdx index 1048f6ce89..4f9891324e 100644 --- a/serverless/containers/reference-content/containers-concurrency.mdx +++ b/serverless/containers/reference-content/containers-concurrency.mdx @@ -60,7 +60,7 @@ For example, if a container is set to handle 10 concurrent requests, and 50 requ **Efficiency**: Setting a higher concurrency value allows a single container instance to handle more requests, which can reduce the total number of container instances, therefore reducing costs. -**Responsiveness**: Setting a lower concurrency (down to 1 concurrent container instance) allows to reduce latency for individual requests, since the container focuses on fewer tasks at a time. However, +**Responsiveness**: Setting a lower concurrency (down to 1 concurrent container instance) allows to reduce latency for individual requests, since the container focuses on fewer tasks at a time. ## Implications for your service From 004d4d60544330fd5c37d32eae867cd05dedf21b Mon Sep 17 00:00:00 2001 From: SamyOubouaziz Date: Thu, 21 Nov 2024 10:40:09 +0100 Subject: [PATCH 3/3] fix(srv): update --- serverless/containers/concepts.mdx | 2 ++ 1 file changed, 2 insertions(+) diff --git a/serverless/containers/concepts.mdx b/serverless/containers/concepts.mdx index ae3f95ae0b..5ec78489bc 100644 --- a/serverless/containers/concepts.mdx +++ b/serverless/containers/concepts.mdx @@ -27,6 +27,8 @@ Startup process steps are: Concurrency defines the number of simultaneous requests a single instance of your container can handle at the same time. Once the number of incoming requests exceeds this value, your container scales according to your parameters. +Refer to the [dedicated documentation](/serverless/containers/reference-content/containers-concurrency/) for more information on container concurrency. + ## Container A container is a package of software that includes all dependencies: code, runtime, configuration, and system libraries so that it can run on any host system. Scaleway provides custom Docker images that are entirely handled for you in the cloud. With Containers, you can rely on your favorite technologies such as Django, or Ruby on Rails.