-
Notifications
You must be signed in to change notification settings - Fork 321
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Fargate] [request]: Allow to increase container disk space #384
Comments
I second that! I really enjoy using ECS with Fargate. Not having to manage the underlying infrastructure is really fantastic. Tell us about your request Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard? Are you currently working around this issue? |
Other requested ways to increase Fargate storage include mounting EFS or mounting S3 storage |
Yes, but for our use case ability to increase disk space directly would be preferred because (a) it is less components to provision/manage/delete and (b) it is unclear what would read/write speed of the attached EFS/S3 volume look like. |
We would also be interested in this. Our use case would be deploying and running a large bundled application (15GB+) via our Jenkins CI on ECS/Fargate. Currently we are too limited on space to achieve this. |
Yes, we too have a use case for this where we are currently performing some ingest tasks in fargate on new media files as they are uploaded to an s3 bucket and the existing (small) storage limit is restricting the sizes of files we can process. Our image file is quite small. We basically just need slightly larger (ephemeral) storage to download the file to and output derivative thumbnails to before saving them to s3. Pretty much exactly the same scenario as the OP (Xantrul). The ability to choose from predefined storage options (just like cpu and memory is for fargate) would be a perfect implementation for this. We already choose task memory allocation based on the input file size and it would be trivial to also spec the storage allocation in the same logic. |
Thumbs up for this request. We are using Fargate and love it. We would like to be able to increase storage over 10Gb or even map task storage to S3. |
I was linked to this issue by AWS' support staff when I asked them how to increase the volume size for a Fargate container. Needless to say, I was equally disappointed that this isn't already an option. In my case I was writing a Step Functions state machine that transforms data through multiple phases (it launches Glue Scripts, Lambda functions, and yes - Fargate containers). I ran this pipeline today and quickly bumped into the disk space limit for one of the Fargate containers. We have a running EKS cluster for some related work as well, so I could write it as a Kubernetes job and then use a Lambda to launch it, and make use of the Step Functions SDK to monitor progress. Or, I could write a bunch of Terraform to convert my ECS cluster into a managed one instead of relying on the defaults and ease of use that Fargate provides. There are probably other ways to go about it, too. But regardless of which alternative path I choose, it's adding several days to my development cycle, when I was expecting to just be able to set a property in my task definition and be done with it. I am very surprised this isn't supported. |
We are also running into this issue. Very surprised and annoyed that there is a hard limit here. |
We have this problem and it is quite blocking. Could we know if you're working on it please ? |
Same thing here. Fat container image on EKS. Fargate node can't pull. |
Same with us, we have containers that needs to run machine learning models and would need storage more than 15GB. Currently tasks are failing with error |
+1 |
Probably still not enough for most of use cases but fargate platform version 1.4.0 offers now 20GB storage space: https://aws.amazon.com/blogs/containers/aws-fargate-launches-platform-version-1-4/ |
And it's now possible to mount EFS volumes for additional storage: https://aws.amazon.com/about-aws/whats-new/2020/04/amazon-ecs-aws-fargate-support-amazon-efs-filesystems-generally-available/ |
Can we extend this to EC2 type also? |
We'd like a configurable amount of disk space above 20gb on Fargate on EKS. |
Looks like this is launching soon! https://github.com/aws/copilot-cli/releases/tag/v1.6.0 |
https://aws.amazon.com/about-aws/whats-new/2021/04/amazon-ecs-aws-fargate-configure-size-ephemeral-storage-tasks/ This feature is available on ECS now. |
Is there any expected date to add support to this on CDK? |
@alanprot For now, you can do this: ephemeralStorageHack(taskDef: ecs.FargateTaskDefinition): void {
const props = taskDef.node.defaultChild as ecs.CfnTaskDefinition;
props.addPropertyOverride("EphemeralStorage", {
SizeInGiB: 100,
});
} |
@markusa380 your solution works, but after a deployment with CodePipeline/ECSDeployAction the EphemeralStorage override is lost. Have you experienced this? |
@iwarshak No sorry, I didn't use CodePipeline however. |
After some time trying to get this into my task definition json, I found that terraform supports it. Here's the correlating portion of the task definition for 200GiB ephemeral storage (FARGATE): {
"targetId": null,
"targetType": null,
"value": null,
"name": "ecs.capability.extensible-ephemeral-storage"
} |
Hi, Your link is only valid for ECS on Fargate, not EKS on Fargate. Thanks |
emm.. Didn't it any progress in THREE years on EKS Fargate? @tabern |
@vibhav-ag if I would like to request an increase in Fargate ECS task size beyond the new 200 GiB limit, is this a good ticket to reopen or shall I create a new one? |
@gj I would recommend opening another one with some detail on what is the driver for the ask. Thanks! |
Tell us about your request
It looks like disk volume one gets in Fargate is ~10 GB. It would be helpful to be able to configure the size it in Task/Container configuration. In our case, we would like to go as high as 150 - 200 GB.
(link to doc confirming this limit: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/service_limits.html)
Which service(s) is this request for?
Fargate
Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
We have a service with a fairly simple flow: It downloads a zip file, extracts useful information from the file and publishes it into external database. The problem is - those zip files can be up to 100 GB in size.
The full flow we tested is: messages come into SQS queue -> Lambda reads from SQS and invokes Fargate task -> Fargate does the job
The workload is very spiky - sometimes we have dozens of files coming in at the same time and sometimes we have nothing for a day or two. So Lambda+Fargate combination works perfectly for us because we process jobs as soon as we get them without any wait, but at the same time don't pay for any resources when we have nothing to process. And nothing to manage too!
Are you currently working around this issue?
Old version of our service runs on multiple EC2 instances. For the new version we're currently evaluating options and since Fargate cannot be used looking into EKS.
Additional context
From other requests in this area it looks like you're already working on the option to attach EBS/EFS volumes, but in our case we don't need persistent storage - we just need more space which is created and destroyed with the container.
The text was updated successfully, but these errors were encountered: