Fargate Agent configuration options enhancements #1614
Conversation
|
This is really great @mhmcdonald! I'm excited to get these extra options into the Fargate Agent! mypy is failing for two minor reasons: |
src/prefect/agent/fargate/agent.py
Outdated
| if not self.execution_role_arn: | ||
| self.logger.exception( | ||
| "Fargate requires task definition to have execution role ARN to support ECR images" | ||
| ) | ||
| raise Exception("Fargate task execution role required") |
There was a problem hiding this comment.
I don't think this Exception needs to be raised however a warning could still be logged. The reason being (for example) in my set up I am using images based in DockerHub and this execution_role_arn wasn't required.
There was a problem hiding this comment.
Sounds good @joshmeek - I'm happy to remove the Exception and just log the warning. The reason we did this was because without the execution_role_arn, the iam role for the agent wasn't being set properly and this fixed it. In your aws environment is an iam role being used for the fargate tasks? Is it somehow defaulting to a iam role? (Just curious). I will update shortly.
There was a problem hiding this comment.
For mine I'm am actually not using an IAM role so using one may actually warrant the need of the execution_role_arn I will test various set ups / configurations and see if that will affect it. Thanks for the feedback 👍
UI: orion-design release 0.1.71
Thanks for contributing to Prefect!
Please describe your work and make sure your PR:
CHANGELOG.md(if appropriate)docs/outline.tomlfor API reference docs (if appropriate)Note that your PR will not be reviewed unless all three boxes are checked.
What does this PR change?
This PR makes a few changes to the Fargate Agent. The first change is that it allows subnets and security groups to be set based on environment variables. The second change is that it adds two AWS configuration options to the Fargate Agent:
task_role_arnandexecution_role_arn.Why is this PR important?
This PR is important because Fargate requires the task definition to have execution role ARN to support ECR images. These configurations are necessary for the IAM role of the agent to be properly set.