[first cut] update rollback failed lambda #93
Conversation
|
Apologies for the linting errors, looks like I didn't have that set up ! Will push changes with the right liniting. |
| name: 'updateRollbackFailed', | ||
| sourcePath: 'rules/updateRollbackFailed.js', | ||
| parameters: { | ||
| includeResources: { |
There was a problem hiding this comment.
Could this parameter be called otherStacks ? @aarthykc do you think this would be more intuitive?
| { | ||
| Effect: 'Allow', | ||
| Action: [ | ||
| 'cloudformation:*' |
There was a problem hiding this comment.
@aarthykc this should be more specific, so that it adheres to least privilege. ie. i think this only needs to be cloudformation:DescribeStacks
| if (err) return callback(err); | ||
| else { | ||
| data[0].forEach(function(i) { | ||
| if(i.name.indexOf('production') > -1 && i.status.indexOf('UPDATE_COMPLETE') > -1) { |
There was a problem hiding this comment.
@aarthykc if I am reading this correctly, this will also catch stacks that are in any of these other status states shown in the list under Stack Status Codes here http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-describing-stacks.html
Is there a reason not to check for the exact match of UPDATE_ROLLBACK_FAILED?
|
|
||
| function describeStacks(params, callback) { | ||
| var q1 = queue(1); | ||
| var cloudformation = new AWS.CloudFormation({region: 'us-east-1'}); |
There was a problem hiding this comment.
@aarthykc right now this just checks stacks in us-east-1. It should check for this state on stacks in all regions.
There are times when stacks are caught in
UPDATE_ROLLBACK_FAILEDand it's good to be alerted when a stack is in such a state. Since there are no cloudwatch events that currently track this, the easiest way was to create a lambda function that would do that looks at which productions are currently in theUPDATE_ROLLBACK_FAILEDstate and alert in case that is the case.I'm currently testing for whether the stack is in
UPDATE_COMPLETEto see if it sends a notification will change this back toUPDATE_ROLLBACK_FAILEDwhen I confirm this works.cc @ianshward, @emilymcafee - does this look okay ?
Next actions: