DeadLetterConfig support

[pmuens]

What did you implement:

Closes #2982

Add config variable for function-based DeadLetterConfig support.

How did you implement it:

Add the onError config variable for functions.

How can we verify it:

Deploy this service. Make your Lambda error out.

service: service

provider:
  name: aws
  runtime: nodejs6.10

functions:
  hello:
    handler: handler.hello
    onError: arn:aws:sns:us-east-1:XXXXX:test # SNS

Todos:

• Write tests
• Write documentation
• Fix linting errors
• Make sure code coverage hasn't dropped
• Provide verification config / commands / resources
• Update the messages below

Is this ready for review?: YES
Is it a breaking change?: NO

[pmuens]

Just a quick update on this one.

Debugged it today and it seems like the IamRoleLambdaExectuion which includes the inline policies for SNS and SQS is not updated by CloudFormation when deploying.

The initial deployment (from scratch) will suceed, however changing an existing stack errors out since the IAM Role is not updated.

The weird part is that the generated CloudFormation template is correct and the CloudFormation console tells you that everything was updated correctly. But if you look into the policies for the role in the IAM console you can see that it was not updated. 🤔

Don't know what's wrong here. A DependsOn is already in place for the Lambda function resource so that the function is created / updated after the IamRoleLambdaExecution is created / updated.

@eahefnawy do you have any idea what might be missing here?

---

Step-by-step list to reproduce the behavior above (:top:)

[0. Make sure that you use a brand new service]

1. Create a new SNS topic (save the arn since you need to use it later)
2. Create a new SQS topic (save the arn since you need to use it later)
3. Add the onError config (just ONE) with the arn as described in the PR description (it doesn't matter if you use sns or sqs)
4. Run serverless deploy
5. Change the onError config so that it uses the sns arn instead of the sqs arn (or the other way around)
6. Run serverless deploy
7. Deployment will fail with an error message that the permissions are missing
8. Inspect the update template in the .serverless directory to see that the inline policy is set correctly
9. Look into the CloudFormation AWS console to see that the IamRoleLamdaExecution resource was updated before the Lambda function was updated
10. Look into the inline policies in the IAM AWS console for the IamRoleLambdaExecution to see that it was not updated

[pmuens]

I did another deep dive into this.

IMHO this is definitely smth. on AWS end. The weird thing is that the update of the IamRoleLambdaExecution only takes a few seconds (~2 seconds in my case) and won't update the inline policy. The Lambda function is always updated adter the IamRoleLambdaExecution is updated, so the DependsOn works quite fine.

Another indicator for a problem on AWS end is that you'll see the correct inline-policy if you look into the compiled CloudFormation template which is uploaded to the deployment bucket and used during the stack update process.

Not quite sure how we should proceed here...

We could use permission resources we merge into the template. I tested this but it's also not working flawlessly on my machine and it looks like it's not the best practice when dealing with DLQ setup through CloudFormation (see: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-lambda-function-deadletterconfig.html).

Do you have an idea how we can make this work @eahefnawy ?

[HyperBrain]

Couldn't it just be that we set a fixed name for the inline policy? Did you try to change the inline policy name to have some unique appendix? IMO the update might work if the inline policy gets a different name on each build.

The inline policy name is set here:
https://github.com/serverless/serverless/blob/master/lib/plugins/aws/lib/naming.js#L94-L106

[pmuens]

@HyperBrain that's a good hint. Thanks for that! 👍

The only thing which makes me curious is that we also add other inline policies (e.g. for stream events or when merging iamRoleStatements --> https://github.com/pmuens/serverless-crud/blob/master/serverless.yml#L8-L18) and everything works fine in that case. 🤔

[pmuens]

Just updated the code so that only SNS topic arns are supported.

Tested everything again and it's working fine. Will merge after the build passed

[neowulf]

Couldn't find the ticket related to supporting SQS. Could that ticket be kindly link here for tracking?

What's the current status on supporting SQS as a dead letter queue? Is it recommended to implement the corresponding cloudformation instructions?

Thanks!

[neowulf]

@pmuens - silly question - was the code generating the cloudformation template by referring to the return value of the SQS resource?

I ran into similar symptoms as described earlier in the ticket. The return value of the SQS resource is the http endpoint! 🙄 One would have to refer to the Arn attribute of the SQS resource.

Simply adding the following to my serverless.yml, I was able to configure a Dead Letter Queue for my lambda function using Cloudformation within serverless.yml:

provider:
  iamRoleStatements:
    - Sid: LambdaDLQPermissions
      Effect: Allow
      Action:
        - "sqs:SendMessage*"
      Resource:
        - "Fn::GetAtt":
          - LambdaFunctionDeadLetterQueue
          - Arn

functions:
  MyFunction:
    handler: handler.XXXXXX

resources:
  Resources:
    MyFunctionLambdaFunction:
      Type: "AWS::Lambda::Function"
      Properties:
        DeadLetterConfig:
          TargetArn:
            "Fn::GetAtt":
              - LambdaFunctionDeadLetterQueue
              - Arn

    LambdaFunctionDeadLetterQueue:
      Type: "AWS::SQS::Queue"
      Properties:
        QueueName: "MyFunctionDLQ"
        MessageRetentionPeriod: 1209600 # 14 days in seconds

[kalinchernev]

@neowulf thank you for sharing your solution, it worked well for me! Even removed serverless-plugin-lambda-dead-letter totally from the stack, no need.

This part from the iamRoleStatements was also not needed. I believe it's stayed from a previous set of attempts as this section is "CloudFormation-aware".

      Resource:
        - "Fn::GetAtt":
          - LambdaFunctionDeadLetterQueue
          - Arn