Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EcsRegisterTaskDefinitionOperator params are incorrectly invalidated #29637

Closed
1 of 2 tasks
janrito opened this issue Feb 20, 2023 · 7 comments
Closed
1 of 2 tasks

EcsRegisterTaskDefinitionOperator params are incorrectly invalidated #29637

janrito opened this issue Feb 20, 2023 · 7 comments
Labels

Comments

@janrito
Copy link

janrito commented Feb 20, 2023

Apache Airflow Provider(s)

amazon

Versions of Apache Airflow Providers

https://raw.githubusercontent.com/apache/airflow/constraints-2.4.3/constraints-3.10.txt

Apache Airflow version

2.4.3

Operating System

https://github.com/aws/aws-mwaa-local-runner / on macos 12.6

Deployment

Docker-Compose

Deployment details

local

What happened

ECS task definition cpu and memory params are incorrectly invalidated

What you think should happen instead

parameters are correct, it should register a task. I can run the same bit of code using boto3.client.ecs.register_task_definition`

How to reproduce

Try and define a task using:

EcsRegisterTaskDefinitionOperator(
        task_id="register_task",
        family=TASK_FAMILY_NAME,
        container_definitions=[
            {
                "name": CONTAINER_NAME,
                "image": "ubuntu",
                "workingDirectory": "/usr/bin",
                "entryPoint": ["sh", "-c"],
                "command": ["ls"],
            }
        ],
        register_task_kwargs={
            "requiresCompatibilities": ["FARGATE"],
            "cpu": "256",
            "memory": "512",
            "networkMode": "awsvpc",
        },
    )
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter cpu, value: 256, type: <class 'int'>, valid types: <class 'str'>
Invalid type for parameter memory, value: 512, type: <class 'int'>, valid types: <class 'str'>

Could this error have something to do with JSON serialising/deserialising where the parameters are autodetected as ints, rather than deserialised into strings

Anything else

No response

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@janrito janrito added area:providers kind:bug This is a clearly a bug needs-triage label for new issues that we didn't triage yet labels Feb 20, 2023
@boring-cyborg
Copy link

boring-cyborg bot commented Feb 20, 2023

Thanks for opening your first issue here! Be sure to follow the issue template!

@Taragolis
Copy link
Contributor

Not a bug Airflow provider, the validation happen inside botocore / boto3, see available options and types in ECS.Client.register_task_definition

@Taragolis Taragolis closed this as not planned Won't fix, can't repro, duplicate, stale Feb 20, 2023
@Taragolis Taragolis added invalid provider:amazon-aws AWS/Amazon - related issues and removed needs-triage label for new issues that we didn't triage yet invalid labels Feb 20, 2023
@Taragolis Taragolis reopened this Feb 20, 2023
@Taragolis
Copy link
Contributor

Nevermind, I have a look on wrong parameters initially.
Did you use specific values such as render_template_as_native_obj=True in DAG?

@janrito
Copy link
Author

janrito commented Feb 20, 2023

Ah! yes. I did. Why is that clashing?

@Taragolis Taragolis added area:core and removed provider:amazon-aws AWS/Amazon - related issues area:providers labels Feb 20, 2023
@Taragolis
Copy link
Contributor

register_task_kwargs it is templated field which evaluate thought jinja and I with this settings NativeEnvironment transform "512" to 512 even if it not not contain jinja blocks

Copy link

This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author.

Copy link

This issue has been closed because it has not received response from the issue author.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Mar 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants