Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(aws-lambda): Ability to specify CPU architecture for building image #20907

Open
1 of 2 tasks
SamStephens opened this issue Jun 28, 2022 · 5 comments
Open
1 of 2 tasks
Labels
@aws-cdk/aws-lambda Related to AWS Lambda effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2

Comments

@SamStephens
Copy link
Contributor

SamStephens commented Jun 28, 2022

Describe the feature

I'd like the ability to specify usage of a bundling image for a particular CPU architecture. As an example, for a Java build, rather than saying:

        builder_options = BundlingOptions(
            command=packaging_instructions,
            image=aws_lambda.Runtime.JAVA_11.bundling_image,
            user="root",
            output_type=BundlingOutput.ARCHIVED,
        )

Which uses the architecture of the machine I'm building on, I'd like to be able to say something like:

        builder_options = BundlingOptions(
            command=packaging_instructions,
            image=aws_lambda.Runtime.JAVA_11.bundling_image_for(aws_lambda.Architecture.X86_64),
            user="root",
            output_type=BundlingOutput.ARCHIVED,
        )

Use Case

We use a Java library that doesn't work on ARM. We deploy to x86_64 Lambda, so this shouldn't be a problem. But in order to ensure builds on M1/M2 Mac work, we need to force the Docker image we use for Java to be an x86_64 image.

I'm currently using

        builder_options = BundlingOptions(
            command=packaging_instructions,
            image=aws_lambda.Runtime('java11:latest-x86_64', aws_lambda.RuntimeFamily.JAVA).bundling_image,
            user="root",
            output_type=BundlingOutput.ARCHIVED,
        )

As a workaround, but it'd be nice to be able to use the aws_lambda.Runtime and aws_lambda.Architecture constants.

I learned about this workaround from #18696 (comment)

Proposed Solution

No response

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.26.0, Python

Environment details (OS name and version, etc.)

Ubuntu 20.04 on WSL 2 within Windows 10.0.19044

@SamStephens SamStephens added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Jun 28, 2022
@github-actions github-actions bot added the @aws-cdk/aws-lambda Related to AWS Lambda label Jun 28, 2022
@SamStephens
Copy link
Contributor Author

Actually, my suggestion

aws_lambda.Runtime.JAVA_11.bundling_image_for(aws_lambda.Architecture.X86_64)

Is based on a Runtime being architecture independent. But the workaround

aws_lambda.Runtime('java11:latest-x86_64', aws_lambda.RuntimeFamily.JAVA).bundling_image

Implies that a Runtime can have an Architecture. Which is a little odd.

@peterwoodworth peterwoodworth added p1 and removed needs-triage This issue or PR still needs to be triaged. labels Jul 1, 2022
mergify bot pushed a commit that referenced this issue Aug 3, 2022
Based on [this](#16770) PR
Add the missing part to add platform support when using lambda `fromImageAsset`

As we are not allowed to specify `platform` flag for `DockerImageAsset`, users deploying cdk on x86_64 platform will not be able to bundle lambda.DockerImageFunction for the new arm64 architecture. Similarly, users deploying cdk on arm64 architecture like Mac M1 will not be able to bundle images for AWS Fargate, which is x86_64 only.

# builder experience with aws-lambda
For x86_64 users deploying Lambda functions with container runtime on Lambda Graviton2(arm64) from local container image assets with multi-architecture docker base images. Specify the platform to ensure the image would be built for ARM64 architecture.

```
 new DockerImageFunction(stack, 'Lambda', {
      code: DockerImageCode.fromImageAsset(path.join(__dirname, 'docker-arm64-handler')),
      architecture: Architecture.ARM_64,
    });
```

Fixes: #12472, #20907
----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [x] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [x] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@kaizencc kaizencc added the effort/small Small work item – less than a day of effort label Aug 24, 2022
@kaizencc kaizencc removed their assignment Aug 24, 2022
josephedward pushed a commit to josephedward/aws-cdk that referenced this issue Aug 30, 2022
Based on [this](aws#16770) PR
Add the missing part to add platform support when using lambda `fromImageAsset`

As we are not allowed to specify `platform` flag for `DockerImageAsset`, users deploying cdk on x86_64 platform will not be able to bundle lambda.DockerImageFunction for the new arm64 architecture. Similarly, users deploying cdk on arm64 architecture like Mac M1 will not be able to bundle images for AWS Fargate, which is x86_64 only.

# builder experience with aws-lambda
For x86_64 users deploying Lambda functions with container runtime on Lambda Graviton2(arm64) from local container image assets with multi-architecture docker base images. Specify the platform to ensure the image would be built for ARM64 architecture.

```
 new DockerImageFunction(stack, 'Lambda', {
      code: DockerImageCode.fromImageAsset(path.join(__dirname, 'docker-arm64-handler')),
      architecture: Architecture.ARM_64,
    });
```

Fixes: aws#12472, aws#20907
----

### All Submissions:

* [x] Have you followed the guidelines in our [Contributing guide?](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md)

### Adding new Unconventional Dependencies:

* [ ] This PR adds new unconventional dependencies following the process described [here](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md/#adding-new-unconventional-dependencies)

### New Features

* [x] Have you added the new feature to an [integration test](https://github.com/aws/aws-cdk/blob/main/INTEGRATION_TESTS.md)?
	* [x] Did you use `yarn integ` to deploy the infrastructure and generate the snapshot (i.e. `yarn integ` without `--dry-run`)?

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@SamStephens SamStephens changed the title (aws-lambda): Ability to specify CPU architecturefor building image (aws-lambda): Ability to specify CPU architecture for building image Oct 16, 2022
@colifran colifran added p1.5 and removed p1 labels May 15, 2023
@otaviomacedo otaviomacedo added p2 and removed p1.5 labels May 22, 2023
@HeskethGD
Copy link

HeskethGD commented Nov 28, 2023

Is it possible to specify the architecture on the bundling image in cdk? Think i'm hitting into this problem. Pydantic core is throwing errors in my deployment as it seems to not be matched up for the architecture and I think it may be related to this. I did swap everything to arm64 and had it working when using cdk deploy from my mac but now I'm deploying from github actions rather than my local machine it is breaking and i'm wondering if it is doing something different there that may be related to this. It suggests here that it is possible to specify architecture in bundling but I don't see it when i go to the actual BundlingOptions page https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.aws_lambda_python_alpha/PythonLayerVersion.html

@tolikkostin
Copy link

@HeskethGD have you solved your issue? Seems I faced the same)

@SamStephens
Copy link
Contributor Author

@HeskethGD @tolikkostin the workaround to specify an architecture is right in the original issue description. Rather than using an image constant like aws_lambda.Runtime.PYTHON_3_11.bundling_image, explicitly specify that you want the image for x86_64. In your case, something like

bundling_options = aws_lambda_python_alpha.BundlingOptions(
    image=aws_lambda.Runtime('python3.11:latest-x86_64', aws_lambda.RuntimeFamily.PYTHON).bundling_image,
)

Then use those bundling options as a parameter to PythonLayerVersion.

@tolikkostin
Copy link

@SamStephens Thanks that works for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-lambda Related to AWS Lambda effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2
Projects
None yet
Development

No branches or pull requests

7 participants