Cross region SNS trigger #3676

matthisk · 2017-05-24T15:23:45Z

This is a (Bug Report)

Description

When trying to deploy a lambda with an SNS trigger in a different region the cloudformation script fails.

What went wrong?

Cloudformation fails to create SNS event subscription for lambda with error:

invalid parameter: TopicArn

What did you expect should have happened?

Expected the lambda to deploy (via cloudformation) with a trigger on an SNS topic in a different region.

What was the config you used?

service: testservice
provider:
  name: aws
  runtime: nodejs6.10
  region: us-west-1

functions:
  hello:
    handler: handler.run
    events: 
      - sns: arn:aws:sns:eu-west-1:xxxxxxxxxxx:stresstest

What stacktrace or error message from your provider did you see?

Serverless: Packaging service...
Serverless: Uploading CloudFormation file to S3...
Serverless: Uploading artifacts...
Serverless: Uploading service .zip file to S3 (2.41 MB)...
Serverless: Updating Stack...
Serverless: Checking Stack update progress...
CloudFormation - UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - stresser-dev
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - HelloLogGroup
CloudFormation - CREATE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::Logs::LogGroup - HelloLogGroup
CloudFormation - CREATE_COMPLETE - AWS::Logs::LogGroup - HelloLogGroup
CloudFormation - CREATE_COMPLETE - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - HelloLambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Function - HelloLambdaFunction
CloudFormation - CREATE_COMPLETE - AWS::Lambda::Function - HelloLambdaFunction
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Permission - HelloLambdaPermissionStresstestSNS
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Version - HelloLambdaVersionOVTh8yMAX1zsFvyQcL4UekM3pyWBiYTwE71y8M
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Permission - HelloLambdaPermissionStresstestSNS
CloudFormation - CREATE_IN_PROGRESS - AWS::SNS::Subscription - HelloSnsSubscriptionStresstest
CloudFormation - CREATE_IN_PROGRESS - AWS::Lambda::Version - HelloLambdaVersionOVTh8yMAX1zsFvyQcL4UekM3pyWBiYTwE71y8M
CloudFormation - CREATE_COMPLETE - AWS::Lambda::Version - HelloLambdaVersionOVTh8yMAX1zsFvyQcL4UekM3pyWBiYTwE71y8M
CloudFormation - CREATE_FAILED - AWS::SNS::Subscription - HelloSnsSubscriptionStresstest
CloudFormation - CREATE_FAILED - AWS::Lambda::Permission - HelloLambdaPermissionStresstestSNS
CloudFormation - UPDATE_ROLLBACK_IN_PROGRESS - AWS::CloudFormation::Stack - stresser-dev
CloudFormation - UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS - AWS::CloudFormation::Stack - stresser-dev
CloudFormation - DELETE_IN_PROGRESS - AWS::Lambda::Permission - HelloLambdaPermissionStresstestSNS
CloudFormation - DELETE_COMPLETE - AWS::SNS::Subscription - HelloSnsSubscriptionStresstest
CloudFormation - DELETE_COMPLETE - AWS::Lambda::Version - HelloLambdaVersionOVTh8yMAX1zsFvyQcL4UekM3pyWBiYTwE71y8M
CloudFormation - DELETE_COMPLETE - AWS::Lambda::Permission - HelloLambdaPermissionStresstestSNS
CloudFormation - DELETE_IN_PROGRESS - AWS::Lambda::Function - HelloLambdaFunction
CloudFormation - DELETE_COMPLETE - AWS::Lambda::Function - HelloLambdaFunction
CloudFormation - DELETE_IN_PROGRESS - AWS::Logs::LogGroup - HelloLogGroup
CloudFormation - DELETE_IN_PROGRESS - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - DELETE_COMPLETE - AWS::Logs::LogGroup - HelloLogGroup
CloudFormation - DELETE_COMPLETE - AWS::IAM::Role - IamRoleLambdaExecution
CloudFormation - UPDATE_ROLLBACK_COMPLETE - AWS::CloudFormation::Stack - stresser-dev

Similar or dependent issues:

SNS event compiler tries to use an existing ARN as a topic name #2183

Additional Data

  Your Environment Information -----------------------------
     OS:                 darwin
     Node Version:       7.10.0
     Serverless Version: 1.14.0

The text was updated successfully, but these errors were encountered:

pmuens · 2017-05-24T18:33:58Z

Thanks for opening @matthisk 👍

🤔 right now Serverless assumes that everything will be deployed into one region. @eahefnawy do you have any idea if there's a possible solution / workaround available for this?

orasis · 2017-05-31T02:49:05Z

I have the same problem. I need to subscribe to an us-east-1 ARN for the AWS marketplace but my whole infrastructure runs in us-west-2

jthomerson · 2017-07-10T19:12:21Z

@pmuens I don't think this is actually a Serverless issue. I looked into this once before and IIRC it is purely a CloudFormation issue. That seems to also be indicated by the error coming back from CloudFormation. I think the issue will need to be raised with AWS directly.

jthomerson · 2017-07-10T19:17:47Z

Interestingly, even when I manually subscribed a Lambda function in a different region than the SNS topic, I had weird behavior. Here's what happened:

$ aws sns subscribe \
   --topic-arn arn:aws:sns:us-east-1:123456789012:TOPIC_NAME \
   --protocol lambda \
   --notification-endpoint arn:aws:lambda:us-west-2:987654321098:function:function-name

# That worked:
{
    "SubscriptionArn": "arn:aws:sns:us-east-1:123456789012:TOPIC_NAME:c9b553d2-3d77-4a99-ad22-52bdb744bc06"
}

When I went to the SNS topic in the web console, I could see the cross-account, cross-region subscription.

But then when I went to the Lambda function in the web console, it showed me this:

Do we even know for sure if AWS supports cross-region SNS->Lambda subscriptions? I could not find any documentation saying they did.

pmuens · 2017-07-11T06:08:01Z

Interesting 🤔 Thanks for the update and investigation on this @jthomerson 👍

Do we even know for sure if AWS supports cross-region SNS->Lambda subscriptions?

Not entirely sure about that. The first time I encountered it was this issue.

ghost · 2017-08-15T17:38:22Z

@jthomerson
Q: Do my AWS Lambda functions need to be in the same region as my Amazon SNS usage?
You can subscribe your AWS Lambda functions to an Amazon SNS topic in any region.
https://aws.amazon.com/sns/faqs/

We need this on our project. What are the next steps for follow up with AWS? I'm sure you guys can pull more strings than we can to get this worked out with CloudFormation

jthomerson · 2017-08-15T17:42:06Z

@srg-avai Yes, you can subscribe cross-region like the FAQ says. However, you can not do it (or could not at the time that I wrote that post) via CloudFormation. Thus, I ended up having to make a custom resource that did the cross-region subscription, and using that custom CloudFormation resource rather than using the built-in CF SNS subscription resource. Obviously, that's not an option for the SLS team (since we'd all have to deploy custom resources to use SLS), but maybe their connections / weight with AWS could get the issue noticed if someone on their team reported it to AWS.

ghost · 2017-08-15T17:45:56Z

@jthomerson thank you, do you mind providing some more detail? i'm interested in the custom resource approach... could you share an example to head start me (and anyone in the future with this prob) on this?

ghost · 2017-08-15T23:07:36Z

fwiw anyone else reading this... the approach jthomerson describes above is as follows:

implement a custom resource - use the resources example on the serverless docs and then search AWS docs for custom resources
the custom resource is a lambda which cloud formation calls mid-deployment, you do what you need to do in the lambda (e.g. subscribe a lambda to a topic in another region), and then you post to an endpoint (provided by cloudfront on the request) when you're done - with a success or failure property (all described in AWS custom resources docs)

jscattergood · 2017-10-13T19:22:38Z

I have created a Gist showing how to implement the custom resource with SLS:

https://gist.github.com/jscattergood/00a2ea6a80fe41a74c5c7efed1b238b4

AdilHindistan · 2018-10-12T18:11:57Z

This is now supported in cloudformation, however, you have to add target region within cfn.
Example:
` SNSSampleSubscription:
Type: AWS::SNS::Subscription

Properties:
  Endpoint: !Sub 'arn:aws:lambda:us-east-2:2222222:function:sample_lambda'
  Protocol: lambda
  TopicArn: !Sub 'arn:aws:sns:us-east-1:1111111:osbf-pull-ami'
  Region: !Sub 'us-east-1'`

In above example, lambda is in us-east-2, and it is subscribing to an sns in us-east-1. The "Region" allows you to do that.

Imran99 · 2018-10-16T01:10:38Z

This is now supported in cloudformation, however, you have to add target region within cfn.

I was able to modify my serverless.yml with the Region parameter and I can see that the subscription is created successfully against the correct topic.

serverless.yml

resources:
  Resources
    SubNameFromGeneratedCfn:
      Properties:
        Region: eu-west-1

I also had to ensure that serverless-psuedo-parameters didn't auto replace the region so just a heads up if you are using that:

custom:
  pseudoParameters:
    skipRegionReplace: true

Fzomerdijk · 2018-11-08T09:30:59Z

Hi,
I can confirm the "Region" option is working, although I have not seen this option in the documentation.

My "LambdaFunction" is created in a EU region and I am subscribing tot the "AmazonIpSpaceChanged" SNS notification in "us-east-1"

See CFN json sniped below.

Regards,
Frank

`
"LambdaAmazonIpSpaceChangedSubscription" : {

  "Type" : "AWS::SNS::Subscription",

  "Properties" : {

    "Endpoint" : {"Fn::GetAtt" : ["LambdaFunction", "Arn"] },

    "Protocol" : "lambda",

    "TopicArn" : "arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged",

    "Region": "us-east-1"

  }

}

`

Imran99 · 2018-11-09T06:28:37Z

Whilst there is now a workaround available it would be great if there was first class support for this on the sns event in serverless.

prernab · 2019-01-04T07:03:34Z

The sns topic is in the region eu-west-1 (not created using cloud formation) so permission is added manually in topic policy and lambda is in different region us-west-2 and using cloud formation. How to provide permission in this case? As I am getting same error "invalid parameter: TopicArn".

{
"Sid": "Subscribe-to-topic",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::xxxxx:root"
},
"Action": "sns:Subscribe",
"Resource": "arn:aws:sns:eu-west-1:xxxxx:publish-notifier"
}

ZhangVic · 2019-04-11T03:58:46Z

@Imran99 Could you please share you serverless.yml ?

patrusso2 · 2019-05-22T13:53:28Z

Hey all - sorry if this is just me being dense, but this took me a while to figure out and I want to add a couple clarifications in case they help out others who run into this issue.

My lambda functions are in us-east-2, but I wanted to use a SNS queue in us-east-1 as a trigger.

I originally tried this in my serverless.yml file, but I got the "invalid parameter: TopicArn" error (like the original poster).

handleBounceOrComplaintEmailSNS:
  handler: server.handleBounceOrComplaintEmailSNS
  events:
    - sns:
        arn: arn:aws:sns:us-east-1:0005555555555:email-bounces-complaints

I didn't know enough about serverless or cloudformation to know what @Imran99 meant by SubNameFromGeneratedCfn, so it took a lot of head scratching and searching.
After staring at my .serverless/cloudformation-template-update-stack.json for a while (and several failed attempts), I finally sorted it out. By searching for my SNS's ARN in .serverless/cloudformation-template-update-stack.json, I found that the SubNameFromGeneratedCfn Imran99 was referring to was HandleBounceOrComplaintEmailSNSSnsSubscriptionemailbouncescomplaints (in my template). By using that name under resources, I was able get it working! So these two components of my serverless.yml file are:

handleBounceOrComplaintEmailSNS:
  handler: server.handleBounceOrComplaintEmailSNS
  events:
    - sns:
        arn: arn:aws:sns:us-east-1:0005555555555:email-bounces-complaints

resources:  
  Resources:
    HandleBounceOrComplaintEmailSNSSnsSubscriptionemailbouncescomplaints:
      Type: AWS::SNS::Subscription
      Properties:
        Region: us-east-1

Bottom line: use Imran99's workaround, but note you'll need to search .serverless/cloudformation-template-update-stack.json to figure out SubNameFromGeneratedCfn. Thank you all for the workaround!

mithun · 2019-07-09T17:42:10Z

Whilst there is now a workaround available it would be great if there was first class support for this on the sns event in serverless.

@dschep why was this issue closed? I do not see any messages from maintainers as to whether this issue will be addressed. There isn't a pending PR either.

herebebogans · 2019-07-12T01:45:18Z

I've raised PR #6366 that should add this functionality.

isaacList · 2019-07-30T07:23:42Z

Hey all - sorry if this is just me being dense, but this took me a while to figure out and I want to add a couple clarifications in case they help out others who run into this issue.

My lambda functions are in us-east-2, but I wanted to use a SNS queue in us-east-1 as a trigger.

I originally tried this in my serverless.yml file, but I got the "invalid parameter: TopicArn" error (like the original poster).
handleBounceOrComplaintEmailSNS:
  handler: server.handleBounceOrComplaintEmailSNS
  events:
    - sns:
        arn: arn:aws:sns:us-east-1:0005555555555:email-bounces-complaints
I didn't know enough about serverless or cloudformation to know what @Imran99 meant by SubNameFromGeneratedCfn, so it took a lot of head scratching and searching.
After staring at my .serverless/cloudformation-template-update-stack.json for a while (and several failed attempts), I finally sorted it out. By searching for my SNS's ARN in .serverless/cloudformation-template-update-stack.json, I found that the SubNameFromGeneratedCfn Imran99 was referring to was HandleBounceOrComplaintEmailSNSSnsSubscriptionemailbouncescomplaints (in my template). By using that name under resources, I was able get it working! So these two components of my serverless.yml file are:
handleBounceOrComplaintEmailSNS:
  handler: server.handleBounceOrComplaintEmailSNS
  events:
    - sns:
        arn: arn:aws:sns:us-east-1:0005555555555:email-bounces-complaints

resources:  
  Resources:
    HandleBounceOrComplaintEmailSNSSnsSubscriptionemailbouncescomplaints:
      Type: AWS::SNS::Subscription
      Properties:
        Region: us-east-1
Bottom line: use Imran99's workaround, but note you'll need to search .serverless/cloudformation-template-update-stack.json to figure out SubNameFromGeneratedCfn. Thank you all for the workaround!

Why would I get the following error even if I follow your example? My lambda function is deployed to us-west-1

  Serverless Error ---------------------------------------
 
  An error occurred: HandleEmailBounceSnsSubscriptionsesbouncestopic - Property Protocol cannot be empty..

functions:
  handleEmailBounce:
    handler: handler.handleEmailBounce
    # notificationArn: ${self:provider.TopicArn}
    events:
      - sns: ${self:provider.TopicArn}  # arn:aws:sns:us-east-1:${self:provider.awsAccountId}:ses-bounces-topic

resources:
  Resources:
    HandleEmailBounceSnsSubscriptionsesbouncestopic:
      Type: AWS::SNS::Subscription
      Properties:
        Region: us-east-1
       # TopicArn: arn of lambda
       # Protocol: lambda

Even if you add TopicArn and Protocol properties, I'm still getting errors....

herebebogans · 2019-07-30T09:08:42Z

@isaacList You no longer need the cloudformation override (I added this in my PR above). Can you try again with the latest version of serverless without the resources override.

isaacList · 2019-07-30T10:15:48Z

@herebebogans Yeah I have updated to the latest version. I can successfully deploy the application without any errors, but I can't find SNS trigger when I go to AWS lambda trigger or find Subscription in the SNS console of us-east-1 region..... The AWS account is the same

herebebogans · 2019-07-30T12:01:57Z

Try the longer form for the event

    events:
      - sns: 
          arn: ${self:provider.TopicArn}

I'll look into why the short form is not adding it.

isaacList · 2019-07-30T13:11:11Z

@herebebogans I am exactly this longer form... But I can't see SNS trigger in Lambda console...

herebebogans · 2019-07-30T23:56:47Z

@isaacList Can you post the relevant snippets from serverless.yml .. eg your provider section + the function definition?

isaacList · 2019-07-31T02:12:48Z

@herebebogans Here you are....

service:
  name: email-bounce-receiver

provider:
  name: aws
  config: ${file(serverless/${env:NODE_ENV}.yml)}
  vpcSettings: ${file(vpc_settings.yml)}
  vpc: ${self:provider.vpcSettings.vpc} #securityGroupIds: [xxx] subnetIds: [xxx]
  runtime: provided
  stage: ${env:NODE_ENV} #dev
  awsAccountId: ${self:provider.config.awsAccountId} 
  region: ${self:provider.config.region} # us-west-1
  stack: ${self:provider.config.stack} #somestring
  TopicArn: arn:aws:sns:us-east-1:${self:provider.awsAccountId}:ses-bounces-topic

package:
  artifact: ./artifact.zip

functions:
  handleEmailBounce:
    handler: handler.handleEmailBounce
    events:
      - sns:
        arn: ${self:provider.TopicArn}

    layers:
      - arn:aws:lambda:${self:provider.region}:${self:provider.awsAccountId}:layer:nodejs-10:1

    environment:
      NODE_ENV: ${env:NODE_ENV}
      NODE_APP_INSTANCE: ${env:NODE_APP_INSTANCE}

    vpc: ${self:provider.vpc}
    reservedConcurrency: 1

herebebogans · 2019-07-31T02:27:58Z

Indent arn in the event two more spaces :-)

isaacList · 2019-07-31T03:22:24Z

Ok You have corrected the sample code. It works perfectly now. Thanks a lot. 😆

Try the longer form for the event
    events:
      - sns: 
        arn: ${self:provider.TopicArn}
I'll look into why the short form is not adding it.

haiyangjiajian · 2019-11-20T10:11:28Z

Service	Account	Region
Lambda	Account1	us-west-1
SNS	Account2	us-east-1

In Account1 we subscribe SNS in region us-west-1, it will fail with error "Error code: InvalidParameter - Error message: Invalid parameter: TopicArn"

We need to switch Account1 to region us-east-1 to do the subscription, it will success.

eric-dorsey · 2023-03-16T14:24:32Z

Seems like this other workaround of adding a topicName (and the topicName doesn't even have to match the actual topic name), which was used for a different problem (resources with the same name - though in our use case we did have SNS topics with the same name across different regions), also resolves not being able to subscribe to SNS topics in other regions (ie, always getting the invalid parameter: TopicArn output):

Repeating working snippet from #4742 (comment) here in case it helps anyone since this issue #3676 seems to come up when searching this problem:

functions:
  SomeFunction:
    handler: main.lambda_handler
    events:
      - sns:
          arn: arn:aws:sns:us-east-1:123456789012:SomeSNSTopic
          topicName: us-east-1
      - sns:
          arn: arn:aws:sns:us-east-2:123456789012:SomeSNSTopic
          topicName: us-east-2

pmuens added question status/more-info-needed and removed question status/more-info-needed labels May 24, 2017

jthomerson mentioned this issue Jul 10, 2017

Support external sns event silvermine/serverless-plugin-external-sns-events#1

Closed

holyjak mentioned this issue Feb 15, 2018

Resources with the same name in different accounts override each other => include account in names #4742

Open

dschep closed this as completed Nov 8, 2018

herebebogans mentioned this issue Jul 12, 2019

Cross region SNS Trigger #6366

Merged

8 tasks

zvymazal mentioned this issue Jul 25, 2019

Cross account SNS Trigger #6444

Closed

konstantinj mentioned this issue Jan 9, 2020

Lambda cannot subscribe to SNS Topic in different region aws/aws-cdk#5734

Closed

2 tasks

OliverHofkens mentioned this issue Feb 15, 2021

Cross-region SNS event triggers not working using CF intrinsic functions #8944

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Cross region SNS trigger #3676

Cross region SNS trigger #3676

matthisk commented May 24, 2017

pmuens commented May 24, 2017 •

edited

Loading

orasis commented May 31, 2017

jthomerson commented Jul 10, 2017

jthomerson commented Jul 10, 2017

pmuens commented Jul 11, 2017 •

edited

Loading

ghost commented Aug 15, 2017

jthomerson commented Aug 15, 2017

ghost commented Aug 15, 2017

ghost commented Aug 15, 2017

jscattergood commented Oct 13, 2017

AdilHindistan commented Oct 12, 2018 •

edited

Loading

Imran99 commented Oct 16, 2018 •

edited

Loading

Fzomerdijk commented Nov 8, 2018

Imran99 commented Nov 9, 2018

prernab commented Jan 4, 2019

ZhangVic commented Apr 11, 2019

patrusso2 commented May 22, 2019

mithun commented Jul 9, 2019

herebebogans commented Jul 12, 2019

isaacList commented Jul 30, 2019 •

edited

Loading

herebebogans commented Jul 30, 2019

isaacList commented Jul 30, 2019 •

edited

Loading

herebebogans commented Jul 30, 2019 •

edited

Loading

isaacList commented Jul 30, 2019 •

edited

Loading

herebebogans commented Jul 30, 2019

isaacList commented Jul 31, 2019 •

edited

Loading

herebebogans commented Jul 31, 2019

isaacList commented Jul 31, 2019 •

edited

Loading

haiyangjiajian commented Nov 20, 2019

eric-dorsey commented Mar 16, 2023 •

edited

Loading

Cross region SNS trigger #3676

Cross region SNS trigger #3676

Comments

matthisk commented May 24, 2017

This is a (Bug Report)

Description

Additional Data

pmuens commented May 24, 2017 • edited Loading

orasis commented May 31, 2017

jthomerson commented Jul 10, 2017

jthomerson commented Jul 10, 2017

pmuens commented Jul 11, 2017 • edited Loading

ghost commented Aug 15, 2017

jthomerson commented Aug 15, 2017

ghost commented Aug 15, 2017

ghost commented Aug 15, 2017

jscattergood commented Oct 13, 2017

AdilHindistan commented Oct 12, 2018 • edited Loading

Imran99 commented Oct 16, 2018 • edited Loading

Fzomerdijk commented Nov 8, 2018

Imran99 commented Nov 9, 2018

prernab commented Jan 4, 2019

ZhangVic commented Apr 11, 2019

patrusso2 commented May 22, 2019

mithun commented Jul 9, 2019

herebebogans commented Jul 12, 2019

isaacList commented Jul 30, 2019 • edited Loading

herebebogans commented Jul 30, 2019

isaacList commented Jul 30, 2019 • edited Loading

herebebogans commented Jul 30, 2019 • edited Loading

isaacList commented Jul 30, 2019 • edited Loading

herebebogans commented Jul 30, 2019

isaacList commented Jul 31, 2019 • edited Loading

herebebogans commented Jul 31, 2019

isaacList commented Jul 31, 2019 • edited Loading

haiyangjiajian commented Nov 20, 2019

eric-dorsey commented Mar 16, 2023 • edited Loading

pmuens commented May 24, 2017 •

edited

Loading

pmuens commented Jul 11, 2017 •

edited

Loading

AdilHindistan commented Oct 12, 2018 •

edited

Loading

Imran99 commented Oct 16, 2018 •

edited

Loading

isaacList commented Jul 30, 2019 •

edited

Loading

isaacList commented Jul 30, 2019 •

edited

Loading

herebebogans commented Jul 30, 2019 •

edited

Loading

isaacList commented Jul 30, 2019 •

edited

Loading

isaacList commented Jul 31, 2019 •

edited

Loading

isaacList commented Jul 31, 2019 •

edited

Loading

eric-dorsey commented Mar 16, 2023 •

edited

Loading