[Fargate] [request]: offer high-performance network options

[lifeofguenter]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
It would be great if it were possible to opt-in for an explicit high-performance network option.

Which service(s) is this request for?
Fargate

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Certain services require a high networking through-put to internal services (s3/elasticache/etc.) - currently it does not seem to be possible to enforce a high network capacity.

Are you currently working around this issue?
No current solution known to us.

Additional context

The following reported feature requests are in the similar range as they demand more performance out of Fargate:

• [Fargate] [request]: compute optimized options #1030 (compute optimized options)
• [Fargate] [request]: Add higher vCPU / Memory options #164 (higher cpu/mem options)
• [Fargate] [request]: Allow to increase container disk space #384 (larger container storage)
• [ECS] Add support for GPU with Docker 19.03 #457 (gpu option)
• AWS Fargate GPU Support: When is GPU support coming to fargate? #88 (gpu option)

Additionally the following blog post goes so far into actually measuring the networking performance: https://stormforger.com/blog/aws-fargate-network-performance/

Attachments

[prameshbajra]

A much needed feature for FarGate instances.

[pawel-przybyla]

This option is required to run time-critical , high RPS services using Fargate launch type. We have an internal .NET Core service. The C# code (including call to PostgreSQL) is executed in 2-3 ms (95 percentile). For almost 2 years it was successfully running, the average Target Response Time metric of ALB was ~4-5 ms. This week after new deployment this metric was increased to ~40-50 ms. Service C# code still executed in ~3 ms. We deployed few instances in two Regions and behavior is consistent. Target Response Time metric of ALB is ~40 ms. Based on CloudWatch metric we believe the CPU and Memory are not an issue. Our assumption is that the currently being selected host has worse network

[ravishtiwari]

It would be really nice to have this feature where users are able to choose or customize different options. It would make it more suitable for additional workloads that can't consistently run on Fargate atm.
👍

[KeenanLawrence]

Any feedback on this? Would definitely strengthen Fargate's stance vs EC2 if it had a high-performance network option.

[zachcasper]

Hey folks, Zach here from the Fargate PM team. I'm working on Fargate's direction around performance. I'm very interested in knowing a few more details here. A few questions:

1. What are some use cases for a high-performance networking option? What types of workloads specifically?
2. Do these workloads need increased bandwidth or decreased latency?
3. Would a more consistent network bandwidth at the current performance level solve this?
4. Or no, and you need more than today—what minimum/baseline bandwidth would you need in Gbps?

Thanks!

[lifeofguenter]

Hi @zachcasper - thanks for coming by :)

What are some use cases for a high-performance networking option? What types of workloads specifically?

S3 uploads: improve transfer speeds, by design I don't think you can scale-out (within a single request) so the task handling the upload should be able to upload as fast as possible.

DynamoDB, Memcached/Redis: usually this can be scaled-out, but with high-performing micro services it can become more difficult to scale out if cpu is not the limiting factor but bandwidth.

Do these workloads need increased bandwidth or decreased latency?

In our experience latency was not an issue but available/consistent bandwidth

Would a more consistent network bandwidth at the current performance level solve this?

Maybe, but then the base-line should at least be on-par with t3/t4g.

[brignolij]

Hey folks, Zach here from the Fargate PM team. I'm working on Fargate's direction around performance. I'm very interested in knowing a few more details here. A few questions:

1. What are some use cases for a high-performance networking option?  What types of workloads specifically?

2. Do these workloads need increased bandwidth or decreased latency?

3. Would a more consistent network bandwidth at the current performance level solve this?

4. Or no, and you need more than today—what minimum/baseline bandwidth would you need in Gbps?

Thanks!

Hi thanks for the questions, here some suggestion from my side.

In our case , we have some issue with network with our .net 6 task running on arm64 runtime on ecs fargate.

1. Me and my team are facing issue when bursting a container with requests who required our application to call another service. in our case, for each of theses requests , our application make a call to redis (elastic cache) or mssql(ec2) or external api.
   We have small container (1024 cpu, 3gbram). the cpu and ram are still not at more than 30%, but we have like 5% of our call to external services (redis, mssql, api) receive an timeout / http 0 when calling. From my understand, the network controller is saturated.
   fargate task with more network connection allowed whould be great. Also, we have no clue of network performance according cpu & ram configuration.

Regards

[jnicholls]

Thanks @zachcasper for offering attention to this request. I would like to parrot the answers from @lifeofguenter, namely that my team's use cases are similarly challenged by a lack of consistent throughput, low throughput capacity, and not really knowing what that capacity is in practice. And with a lot of the same services in mind (S3 TX/RX, DynamoDB, Redis, etc.)

I too have leaned on this experiment to understand what to expect out of Fargate. For I/O bound workloads, it would help our Fargate capacity planning to at least know what the current network capacity is at various vCPU/Mem combinations. From there, it would be great to perhaps explore allowing network performance to be a separate task capacity dimension.

To expand upon @lifeofguenter's S3 upload use case: it was mentioned that it was not possible to scale-out a request. However, it is in fact possible to scale out an object upload across available CPUs using S3's multipart upload support. The same can be accomplished for downloads with S3's Range support. One particular use case I have is reading in a lot of S3 objects, and concatenating them together into larger, fewer S3 objects. This process is strictly I/O bound, and I am having to instantiate enough Fargate tasks to accomplish the necessary I/O volume needed, leaving a lot of CPU on the table. The idea here would be to enable high-throughput networking in order to move a Fargate task from being I/O bound to being CPU bound, if I/O is its primary dimension. Right now, we're considering making more use of these tasks to fill the gap in CPU usage, but architecturally I see value in having deterministic network capacity as an option for planning Fargate capacity for various workloads.

[TreeKat71]

One of reasons I would like to have better network performance is related to another issue: Fargate can not cache the image, so better network performance can definitely save some money.

[fbuinosquy1985]

same as @brignolij , we have a lot of .net services that consumes rabbitmq, redis ..

[brignolij]

@fbuinosquy1985 we solved a lot of issue by doing some application code optimization:
by optimization of http usage.
check : https://learn.microsoft.com/en-us/aspnet/core/fundamentals/http-requests?view=aspnetcore-7.0 or
https://restsharp.dev/v107/#restclient-and-options
in one word, make your network service (http, redis) as singleton. it will avoid having too many network connections open.
In our case, we fixed all our issues.