Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: SAM Transformer does not support all the parameter mapping for "SourceAccessConfiguration" property for EventSourceMapping resource. #3339

Closed
cool-raj opened this issue Sep 5, 2023 · 3 comments
Assignees
Labels
stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at.

Comments

@cool-raj
Copy link

cool-raj commented Sep 5, 2023

I have found the following issue where requestor has asked for the new selfmangedkafka event mapping , However my issue is with the mapping of parameters as some of that are not supported in the selfmangedkafka event source by the SAM transformer .

Description:

I am using the Sam template to deploy the selfmanagedkafka event source mapping with my lambda.
Seems like some of the parameter are not mappable (SASL_SCRAM_256_AUTH and SERVER_ROOT_CA_CERTIFICATE)in the serverless template .

Steps to reproduce:

use a sam template to integrate a selfmanaged kafka cluster which offers SASL_SCRAM_256_AUTH auth mechanism and CA certificate for TLS .

eg :
AWSTemplateFormatVersion: '2010-09-09' Transform: AWS::Serverless-2016-10-31 Resources: AlphaConsumerRole: Metadata: aws:copilot:description: An IAM role for the worker lambda SamResourceId: AlphaConsumerRole Type: AWS::IAM::Role Properties: AssumeRolePolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: Service: - lambda.amazonaws.com Action: - sts:AssumeRole Path: / ManagedPolicyArns: - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole - arn:aws:iam::aws:policy/service-role/AWSLambdaVPCAccessExecutionRole - arn:aws:iam::aws:policy/service-role/AWSLambdaMSKExecutionRole - arn:aws:iam::123456789123:policy/my-kafka-lambda MSKToS3Lambda: Type: AWS::Serverless::Function Properties: CodeUri: s3://mytest-samclisourcebucket-1af44wx6aksiq/kafka-lambda/4407ff87d25b8715424938a6c4ded55a Handler: com.amazonaws.kafka.samples.HandlerMSK Runtime: java17 Timeout: 900 MemorySize: 512 Role: Fn::GetAtt: - AlphaConsumerRole - Arn VpcConfig: SecurityGroupIds: - sg-0670ae80c15dad260 SubnetIds: - subnet-0b5ff34efbffceea4 - subnet-0a8810ee02cf7aa39 - subnet-0592c949cd17d2f80 Events: KafkaEventSource: Type: SelfManagedKafka Properties: BatchSize: 10 Enabled: true ConsumerGroupId: test-grp KafkaBootstrapServers: - development-kafka-sd.awscp.pke.fhm.de:9292 StartingPosition: LATEST Topics: - mytesttopic SourceAccessConfigurations: - Type: VPC_SUBNET URI: subnet-0b5ff34efbffceea4 - Type: VPC_SUBNET URI: subnet-0a8810ee02cf7aa39 - Type: VPC_SUBNET URI: subnet-0592c949cd17d2f80 - Type: VPC_SECURITY_GROUP URI: sg-06b9ae80c15dab257 - Type: SASL_SCRAM_256_AUTH URI: arn:aws:secretsmanager:eu-central-1:123456789123:secret:test-sasl-certificate-ghijk - Type: SERVER_ROOT_CA_CERTIFICATE URI: arn:aws:secretsmanager:eu-central-1:123456789123:secret:test-root-certificate-abcdef Metadata: SamResourceId: MSKToS3Lambda

Observed result:

E0001 Error transforming template: Resource with id [MSKToS3Lambda] is invalid. Event with id [KafkaEventSource] is invalid. Multiple auth mechanism properties specified in SourceAccessConfigurations for self managed kafka event.

Expected result:

I would expect the transformer should be able to map both the authentication mechanism to the corresponsing supported parameter in cloudfomation .

https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-property-function-selfmanagedkafka.html#sam-function-selfmanagedkafka-sourceaccessconfigurations

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-eventsourcemapping-sourceaccessconfiguration.html

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS:
  2. sam --version:
  3. AWS region:
# Paste the output of `sam --info` here

Add --debug flag to command you are running

@cool-raj cool-raj added the stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at. label Sep 5, 2023
@cool-raj cool-raj changed the title Bug: SAM Transformer does not support all the parameter mapping for "SourceAccessConfiguration" property Bug: SAM Transformer does not support all the parameter mapping for "SourceAccessConfiguration" property for EventSourceMapping resource. Sep 5, 2023
@SimonCMoore SimonCMoore transferred this issue from aws/aws-sam-cli Sep 6, 2023
@ConnorRobertson ConnorRobertson self-assigned this Sep 7, 2023
@ConnorRobertson
Copy link
Contributor

This fix for this will roll out in the coming weeks!

@cool-raj
Copy link
Author

cool-raj commented Sep 12, 2023

Hey @ConnorRobertson thanks for taking this issue :) , What does this mean in regards to SAM-CLI . Will this be part of SAM-CLI Release ?

@ConnorRobertson
Copy link
Contributor

The feature will work for you once the change reaches the regions you are using to deploy and should not explicitly need a SAM CLI change. SAM CLI, which includes some SAM code receives automated updates and is usually released soon after SAM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stage/needs-triage Automatically applied to new issues and PRs, indicating they haven't been looked at.
Projects
None yet
Development

No branches or pull requests

2 participants