Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions serverless/containers/concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,8 @@ Startup process steps are:

Concurrency defines the number of simultaneous requests a single instance of your container can handle at the same time. Once the number of incoming requests exceeds this value, your container scales according to your parameters.

Refer to the [dedicated documentation](/serverless/containers/reference-content/containers-concurrency/) for more information on container concurrency.

## Container

A container is a package of software that includes all dependencies: code, runtime, configuration, and system libraries so that it can run on any host system. Scaleway provides custom Docker images that are entirely handled for you in the cloud. With Containers, you can rely on your favorite technologies such as Django, or Ruby on Rails.
Expand Down
58 changes: 54 additions & 4 deletions serverless/containers/reference-content/containers-concurrency.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,68 @@ categories:

## Concurrency overview

Concurrency determines the number of incoming requests a single instance of a container can process before scaling up. Serverless Containers autoscale according to the number of instances needed to handle the incoming workload.
Concurrency determines the number of incoming requests a single instance of a container can process before scaling up. Serverless Containers automatically scale according to the number of instances needed to handle the incoming workload.

A higher number of instances processing requests at the same time implies a greater usage of memory and [vCPU](/serverless/containers/concepts/#vcpu), and consequently a higher cost.

## Maximum concurrent requests per instance
```
Concurrency = 1

┌─────────────┐
│ ┌──────┐│
│ │ │ ││
└─────┼───► │ ││ Concurrency = 80
│ │ ││
│ └──────┘│ ┌─────────────┐
│ ┌──────┐│ │ │ ┌──────┐│
│ │ ││ └────┼───► │ ││
3 requests ──────┼───► │ ││ 3 requests ─────┼───► │ ││
│ │ ││ ┌────┼───► │ ││
│ └──────┘│ │ │ └──────┘│
│ ┌──────┐│ └─────────────┘
│ │ ││ 1 service
┌─────┼───► │ ││ 1 container instance
│ │ │ ││
│ └──────┘│
└─────────────┘
1 service
3 container instances
```

When [deploying a container](/serverless/containers/how-to/deploy-container/), Scaleway Serverless Containers allows you to edit the **Maximum concurrent requests per instance** parameter.
## Concurrency setting

Serverless Containers apply the highest possible value by default: **80 concurrent requests**. We recommend using the default value, as Serverless Containers are designed to efficiently function with it, but you can lower it to better fit specific requirements.
When [deploying a container](/serverless/containers/how-to/deploy-container/), Scaleway Serverless Containers allows you to configure the [concurrency](/serverless/containers/concepts/#concurrency), which is the maximum number of requests a single container instance can handle at the same time. The concurrency setting ranges from 1 to 1,000 simultaneous instances.

By default, Serverless Containers allow **80 concurrent requests** per container instance. We recommend using the default value, as Serverless Containers are designed to efficiently function with it, but you can lower it to better fit specific requirements.

## Impact of concurrency on container scaling

If the number of incoming requests exceeds the set concurrency value for a container, Serverless Containers automatically creates new container instances to handle the additional load.

For example, if a container is set to handle 10 concurrent requests, and 50 requests or events arrive at the same time, Serverless Containers will create at least 5 container instances to handle the incoming load.

## Benefits of concurrency

**Efficiency**: Setting a higher concurrency value allows a single container instance to handle more requests, which can reduce the total number of container instances, therefore reducing costs.

**Responsiveness**: Setting a lower concurrency (down to 1 concurrent container instance) allows to reduce latency for individual requests, since the container focuses on fewer tasks at a time.

## Implications for your service

**Compute-bound workloads**: Workloads relying on heavy CPU or memory usage per request benefit from lower concurrency (e.g., 1 or 2) to avoid resource contention within the container.

**I/O-bound workloads**: Workloads relying on smaller input/output operations, such as waiting for external APIs or databases benefit from higher concurrency, since the container can handle multiple requests while waiting on I/O.

## Examples

### Low concurrency

If your Serverless Container hosts a resource-intensive application that can only handle a small number of requests with the provisioned memory and vCPU, you can reduce the number of maximum concurrent requests per instance to scale quicker and avoid bottlenecks and queued requests.

You can set the **Maximum concurrent requests per instance** of your container to `1` if it is designed to handle a single request at a time. However, a low maximum concurrency value may affect the performance of your deployment, as several instances of your container will start if it receives a high number of concurrent requests.

### High concurrency

If your Serverless Container hosts an application that receives a high number of requests that are easily processed with the provisioned memory and vCPU, you can set a greater number of maximum concurrent requests per instance.

By setting a greater concurrency value than the average number of concurrent requests your container can handle, it will only scale up in the event of a spike in received requests.