A simple flow for generating CloudFormation Lambda-Backed Custom Resource handlers in node.js. The scope of this module is to structure the way developers author simple Lambda-Backed resources into simple functional definitions of Create, Update, Delete.
Also supports:
- Automatic expansion of
__default__Propertiesvalues into any tree or subtree of any Custom Resource utilizingcfn-lambdafor implementation - Validation of
'ResourceProperties'- Using inline JSONSchema objects as
Schema - Using a
SchemaPathto JSONSchema file - Using a custom
Validatecallback
- Using inline JSONSchema objects as
- Optional
NoUpdatecallback, which runs as a READ function for whenUpdateshould be made due to all parameters being identical - because some resources still need to return attributes forFn::GetAttcalls. - Convenience
Environmentvalues- Lambda ARN
- Lambda Name
- AWS Account ID for the Lambda
- Region for the Lambda
- Array of String
TriggersReplacementforResource.Propertieskey strings that force delegation to resourceCreatefor seamless full replacement without downtime in many cases, and forcingUPDATE_COMPLETE_CLEANUP_IN_PROGRESS. - An
SDKAliasfunction generator that structures and greatly simplifies the development of custom resources that are supported by the Node.jsaws-sdkbut not supported by CloudFormation.
This package on NPM
This package on GitHub
- Stable
Custom::ApiGatewayRestApi(GitHub / NPM) - Stable
Custom::ApiGatewayMethod(GitHub / NPM) - Stable
Custom::ApiGatewayMethodResponse(GitHub / NPM) - Beta
Custom::ApiGatewayDeployment(GitHub / NPM) - Alpha
Custom::ApiGatewayStage(GitHub / NPM)
Any custom resource using this tool as a dependency can run this deploy script from the root of the custom resource project to deploy Lambdas to all regions. Add this line to the "scripts" section of your package.json inside your repository using this module as a direct dependency:
"cfn-lambda-deploy": "node ./node_modules/cfn-lambda/deploy.js"
You can also deploy the Lambdas programmatically like so:
(cfn-module) (default region) (regions) (callback)
node -e "require('cfn-lambda').deploy('cfn-yourmodule', 'us-east-1', ['us-east-1'], null);"
You must also set up:
- Add
<reporoot>/execution-policy.jsonto define the abilities the Lambda should have. - Have AWS credentials configured in your environment, via one of:
$AWS_PROFILEin your environment- a credentials file
$AWS_ACCESS_KEY_IDand$AWS_SECRET_ACCESS_KEYin your environment.
You then run this from within the repository directly depending on cfn-lambda:
$ npm run cfn-lambda-deploy
It should look like this:
This is a contrived example call to fully demonstrate the way to interface with the creation API.
You can manually define these properties, or use SDKAlias for Create, Update and/or Delete.
var CfnLambda = require('cfn-lambda');
exports.handler = CfnLambda({
Create: Create, // Required function
Update: Update, // Required function
Delete: Delete, // Required function
// Any of following to validate resource Properties
// If you do not include any, the Lambda assumes any Properties are valid.
// If you define more than one, the system uses all of them in this order.
Validate: Validate, // Function
Schema: Schema, // JSONSchema v4 Object
SchemaPath: SchemaPath, // Array path to JSONSchema v4 JSON file
// end list
NoUpdate: NoUpdate, // Optional
TriggersReplacement: TriggersReplacement, // Array<String> of properties forcing Replacement
LongRunning: <see Long Running below> // Optional. Configure a lambda to last beyond 5 minutes.
});
Provides convenience Environment values.:
var CfnLambda = require('cfn-lambda');
// After receiving `event` and `context`...
console.log(CfnLambda.Environment);
/*
{
`LambdaArn`: 'foo bar', // Full ARN for the current Lambda
`Region`: 'us-east-1', // Region in which current Lambda resides
`AccountId`: '012345678910', // The account associated with the Lambda
`LambdaName`: 'LambdaName' // Name for the current Lambda
}
*/
Only works after the generated CfnLambda function has been called by Lambda.
Called when CloudFormation issues a 'CREATE' command.
Accepts the CfnRequestParams Properties object, and the reply callback.
function Create(CfnRequestParams, reply) {
// code...
if (err) {
// Will fail the create.
// err should be informative for Cfn template developer.
return reply(err);
}
// Will pass the create.
// physicalResourceId defaults to the request's `[StackId, LogicalResourceId, RequestId].join('/')`.
// FnGetAttrsDataObj is optional.
reply(null, physicalResourceId, FnGetAttrsDataObj);
}
Called when CloudFormation issues an 'UPDATE' command.
Accepts the RequestPhysicalId String, CfnRequestParams Properties object, the OldCfnRequestParams Properties object, and the reply callback.
function Update(RequestPhysicalID, CfnRequestParams, OldCfnRequestParams, reply) {
// code...
if (err) {
// Will fail the update.
// err should be informative for Cfn template developer.
return reply(err);
}
// Will pass the update.
// physicalResourceId defaults to pre-update value.
// FnGetAttrsDataObj is optional.
reply(null, physicalResourceId, FnGetAttrsDataObj);
}
Called when CloudFormation issues a 'DELETE' command.
Accepts the RequestPhysicalId String, CfnRequestParams Properties object, and the reply callback.
function Delete(RequestPhysicalID, CfnRequestParams, reply) {
// code...
if (err) {
// Will fail the delete (or rollback).
// USE CAUTION - failing aggressively will lock template,
// because DELETE is used during ROLLBACK phases.
// err should be informative for Cfn template developer.
return reply(err);
}
// Will pass the delete.
// physicalResourceId defaults to pre-delete value.
// FnGetAttrsDataObj is optional.
reply(null, physicalResourceId, FnGetAttrsDataObj);
}
Used before 'CREATE', 'UPDATE', or 'DELETE' method handlers. The CloudFormation request will automatically fail if any truthy values are returned, and any String values returned are displayed to the template developer, to assist with resource Properties object correction.
Important: To prevent ROLLBACK lockage, the 'DELETE' will be short circuited if this check fails. If this check fails, CloudFormation will be told that everything went fine, but no actual further actions will occur. This is because CloudFormation will immediately issue a 'DELETE' after a failure in a 'CREATE' or an 'UPDATE'. Since these failures themselves will have resulted from a validation method failure if the subsequent 'DELETE' fails, this is safe.
May be a:
- Custom validation function as
Validatecallback - JSONSchema v4
Schema - JSONSchema v4 file path as
SchemaPath
The truthy String return value will cause a 'FAILURE', and the String value is used as the CloudFormation 'REASON'.
// Example using a custom function
// CfnRequestParams are all resource `Properties`,
// except for the required system `ServiceToken`.
function Validate(CfnRequestParams) {
// code...
if (unmetParamCondition) {
return 'You must blah blah include a parameter... etc'
}
if (someOtherCondition) {
return 'Informative message to CFN template developer goes here.'
}
// Returning a falsey value will allow the action to proceed.
// DO NOT return truthy if the request params are valid.
}
Using a JSONSchema Schema property value will automatically generate the String invalidation return values for you when validating against the parameters - simply provide the template and the validation and error messging is taken care of for you.
If you choose to use a JSONSchema template, the service will also use the JSONSchema metaschema to ensure the provided JSONSchema is a valid schema itself.
// Example using a custom JSONSchema Version 4 template
// This might be in a file you manually load like `schema.json`, or a JS object.
var Schema = {
type: 'object',
required: [
'foo'
],
properties: {
foo: {
type: 'string'
},
selectable: {
type: 'string',
enum: ['list', 'of', 'valid', 'values']
}
},
additionalProperties: false
};
A convenient way to get the benefits of Schema object validation, but keeping your code clean and segregated nicely.
The path is defined as an Array so that we can use the path module.
var SchemaPath = [__dirname, 'src', 'mytemplate.json'];
Optional. Triggered by deep JSON object equality of the old and new parameters, if defined.
Even when short-circuiting an Update is a good idea, a resource provider may still need to return a set of properties for use with Fn::GetAtt in CloudFormation templates. This NoUpdate handler triggers in the special case where no settings on the resource change, allowing the developer to simultaneously skip manipulation logic while doing read operations on resources to generate the attribute sets Fn::GetAtt will need.
// Using a custom NoUpdate for READ to supply properties
// for Fn::GetAtt to access in CloudFormation templates
function NoUpdate(PhysicalResourceId, CfnResourceProperties, reply) {
// code that should be read-only if you're sane...
if (errorAccessingInformation) {
return reply('with an informative message');
}
// Can have many keys on the object, though I only show one here
reply(null, PhysicalResourceId, {Accessible: 'Attrs object in CFN template'});
}
This is very advanced Lambda self replication.
The inner workings of this feature are a lot to take in. I strongly suggest you just read the source code for cfn-elasticsearch-domain to see how the index.js file utilizes the LongRunning feature, as the concrete example code is much more understandable than abstract definitions of parameters and options.
cfn-elasticsearch-domain/index.js GitHub
If you have the appetite for it... Read on...
Some resources will take a considerable amount of time to complete, like an Elasticsearch Domain. In order to utilize Lambda-Backed Custom Resources within CloudFormation while avoiding the hard 300 second / 5 minute Lambda timeout for resources that will take more than 5 minutes to finish, cfn-lambda allows resource developers to leverage bundled Lambda self-replication logic. Developers can configure the LongRunning property on the lambda definition options object with a few settings to tell cfn-lambda to simply run some action initialization code (such as initiating an Elasticsearch Domain Create), then periodically self-replicate to check the status of the long-running process. The majority of cases where AWS APIs or SDKs return statusCode === 202 will use this technique to avoid Lambda death at 5 minutes.
The self-replication strategy will trigger if the developer configures the following on the LongRunning property object: PingInSeconds, MaxPings, LambdaApi, Methods.METHOD_NAME.
The duration a Lambda will wait between spawning self-replication calls and triggering the next LongRunning.Methods.METHOD_NAME call. This value should not exceed 240 (4 minutes), because we need to leave enough time before the 5 minute hard process death is triggered by AWS.
After this time, the lambda will spawn a new lambda, which will call the LongRunning.Method.METHOD_NAME, where METHOD_NAME is Create, Delete, Update, depending on which are configured and the lifecycle phase the resource is moving through.
The maximum number of self-respawn and check cycles the Lambda will go through. After exceeding this number, the Lambda will circuit break and send a Failed to Stabilize response to the CloudFormation stack.
cfn-lambda uses this namespace to invoke the Lambda. Allows the Custom Resource developer using cfn-lambda to specify a Lambda API version, or stub the value out for testing.
In most cases, just pass new AWS.Lambda({apiVersion: '2015-03-31'}) as the API namespace.
Most of the LongRunning logic happens here. At its most configured, this subobject will have 3 properties corresponding to the normal actions: Create, Update, and Delete.
When you configure one of these properties, the flow of that CloudFormation action type changes - within the reply callback function in the corresponding normal/top-level callback you defined for the resource, reply-ing with success just tells cfn-lambda that you correctly initialized the Create/Delete/Update for the resource, and to start using the corresponding LongRunning.Methods.METHOD to ping to final completion. That is, the resource will not COMPLETE the action until the function you define finalizes the SUCCESS.
Read below to see how to define each LongRunning.Methods.METHOD...
All three LongRunning.Methods receive a special object as their first parameter. The LongRunningContext object carries useful state across all spawned lambda ping cycles.
LongRunningContext.RawResponse: carries the original intercepted call that your first Create initialization call tried to send to CloudFormation. Used internally for state manipulation. DO NOT ALTER THIS VALUE unless you really know what you're doing, as tampering can cause Lambda recursion to spiral out of control!LongRunningContext.PhysicalResourceId: Carries the originalPhysicalResourceIdintercepted call that your first Create initialization call tried to send to CloudFormation. Useful if your check functions need this value and cannot recompute it from theResourcePropertiessent by CloudFormation.LongRunningContext.Data: Carries the original data hash, if present, intercepted from your call toreplywithin the initializer method. Useful if your check functions need these data value(s) and cannot recompute them from theResourcePropertiessent by CloudFormation. Will not be present if you did not pass a third parameter toreplyin the initializer, since theGetAtt-usableDatahash is optional incfn-lambda.LongRunningContext.PassedPings: The number of ping spawns before this current run that have occurred. DO NOT ALTER THIS NUMBER! Subtracting from this number will make your Lambdas infinitely self-replicate, very very bad!
Will be called during Lambda pingspawn cycles. Here, CheckCreate is an example of a check function definition for LongRunning.Methods.Create.
function CheckCreate(LongRunningContext, params, reply, notDone) {
// LongRunningContext is object type specified above
// params are Properties straight from CloudFomation
// reply is callback just like in normal Create,
// call it with reply(errMsg) or reply(null, physicalId, AttrHash)
// notDone takes no parameters, use this to tell
// cfn-lambda to use another ping/spawn cycle and check again later
}
Will be called during Lambda pingspawn cycles. Here, CheckUpdate is an example of a check function definition for LongRunning.Methods.Update.
function CheckUpdate(LongRunningContext, physcialId, params, oldParams, reply, notDone) {
// LongRunningContext is object type specified above
// physicalId is PhysicalResourceId from pre-Update resource state
// params are Properties straight from CloudFomation
// oldParams are Properties from CloudFormation for before the Update began
// reply is callback just like in normal Update,
// call it with reply(errMsg) or reply(null, physicalId, AttrHash)
// to finalize the transition and notify CloudFormation.
// notDone takes no parameters, use this to denote no errors and tell
// cfn-lambda to use another ping/spawn cycle and check again later
}
Will be called during Lambda pingspawn cycles. Here, CheckDelete is an example of a check function definition for LongRunning.Methods.Delete.
function CheckDelete(LongRunningContext, physcialId, params, reply, notDone) {
// LongRunningContext is object type specified above
// physicalId is PhysicalResourceId from pre-Delete resource state
// params are Properties straight from CloudFomation
// reply is callback just like in normal Delete,
// call it with reply(errMsg) or reply(null, physicalId, AttrHash)
// to finalize the transition and notify CloudFormation.
// notDone takes no parameters, use this to denote no errors and tell
// cfn-lambda to use another ping/spawn cycle and check again later
}
Optional. Tells cfn-lambda to divert the 'Update' call from CloudFormation to the Create handler the developer assigns to the Lambda. This technique results in the most seamless resource replacement possible, by causing the new resource to be created before the old one is deleted. This Delete cleanup process occurs in the UPDATE_COMPLETE_CLEANUP_IN_PROGRESS phase after all new resources are created. This property facilitates triggering that said phase.
exports.handler = CfnLambda({
// other properties
TriggersReplacement: ['Foo', 'Bar'],
// other properties
});
Now, if the Lambda above ever detects a change in the value of Foo or Bar resource Properties on Update, the Lambda will delegate to a two-phase Create-new-then-Delete-old resource replacement cycle. It will use the Create handler provided to the same CfnLambda, then subsequently the prodvided Delete if and only if the Create handler sends a PhysicalResourceId different from the original to the reply callback in the handler.
Structures and accelerates development of resources supported by the aws-sdk (or your custom SDK) by offering declarative tools to ingest events and proxy them to AWS services.
Will automatically correctly ignore ServiceToken from CloudFormation Properties. All settings are optional, except api and method.
var AWS = require('aws-sdk');
var AnAWSApi = new AWS.SomeNamespace();
var CfnLambda = require('cfn-lambda');
// Then used as the Create property as defined in Usage above
var MyAliasActionName = CfnLambda.SDKAlias({ // Like Create, Update, Delete
returnPhysicalId: 'KeyFromSDKReturn' || function(data) { return 'customValue'; },
downcase: boolean, // Downcase first letter of all top-level params from CloudFormation
api: AnAWSApi, // REQUIRED
method: 'methodNameInSDK', // REQUIRED
mapKeys: {
KeyNameInCfn: 'KeyNameForSDK'
},
forceBools: [ // CloudFormation doesn't allow Lambdas to recieve true booleans.
'PathToCfnPropertyParam', // This will coerce the parameter at this path.
'Also.Supports.Wildcards.*',
'But',
'only.at.path.end'
],
keys: [ // Defaults to including ALL keys from CloudFormation, minus ServiceToken
'KeysFrom',
'CloudFormationProperties',
'ToPassTo',
'TheSDKMethod',
'**UsedBeforeMapKeys**'
],
returnKeys: [
'KeysFrom',
'SDKReturnValue',
'ToUseWithCfn',
'Fn::GetAttr'
],
ignoreErrorCodes: [IntegerCodeToIgnore, ExWouldBe404ForDeleteOps],
physicalIdAs: 'UsePhysicalIdAsThisKeyInSDKCall',
});
// Then...
exports.handler = CfnLambda({
Create: MyAliasActionName, // Doesn't have to be Create, can be Update or Delete
// ...
});
Sometimes it is advantageous to be able to reuse JSON objects or fragments of JSON objects in Properties of Custom Resources, like when you need to build similar complex/large resources frequently that differ by only a few properties.
Any module using cfn-lambda supports __default__ property expansion. __default__ can be added anywhere in the Properties object for a resource, with __default__ containing an arbitrary JSON/String/Array/null/Number value serialized using toBase64(JSON.stringify(anyObject)). cfn-lambda will expand these properties before hitting any validation checks, by running JSON.parse(fromBase64(encodedDefault)) recursively, and overwriting any values in the __default__ tree with those actually set on the Properties object.
The best example of this is the cfn-variable module's example.template.json, wherein a very large RestApi is created with over a large repeated subtree of Resource objects. cfn-variable is a custom resource that takes any value and serializes it using toBase64(JSON.stringify(anyValue)), making it a perfect fit for this behavior.
In the example in cfn-variable, this technique is used to create 120 Resource objects in under 15 seconds (this example uses less):
// This is cfn-variable storing the serialized object:
"MySubtreeVariable": {
"Type": "Custom::Variable",
"Properties": {
"ServiceToken": {
"Fn::Join": [
":",
[
"arn",
"aws",
"lambda",
{
"Ref": "AWS::Region"
},
{
"Ref": "AWS::AccountId"
},
"function",
{
"Ref": "VariableCustomResourceName"
}
]
]
},
"VariableValue": {
"ChildResources": [
{
"PathPart": "a",
"ChildResources": [
{
"PathPart": "aa",
"ChildResources": [
{
"PathPart": "aaa"
},
{
"PathPart": "aab"
},
{
"PathPart": "aac"
}
]
},
{
"PathPart": "ab",
"ChildResources": [
{
"PathPart": "aba"
},
{
"PathPart": "abb"
},
{
"PathPart": "abc"
}
]
},
{
"PathPart": "ac",
"ChildResources": [
{
"PathPart": "aca"
},
{
"PathPart": "acb"
},
{
"PathPart": "acc"
}
]
}
]
}
]
}
}
},
// Then this will make the tree 3x because you used a variable with __default__
"ExpandedResourceTree": {
"DependsOn": [
"MyRestApi",
"MyVariable"
],
"Type": "Custom::ApiGatewayResourceTree",
"Properties": {
"ServiceToken": "<the token>",
"RestApiId": {
"Ref": "MyRestApi"
},
"ParentId": {
"Fn::GetAtt": [
"MyRestApi",
"RootResourceId"
]
},
"ChildResources": [
{
"PathPart": "alpha",
"__default__": {
"Fn::GetAtt": [
"MySubtreeVariable",
"Value"
]
}
},
{
"PathPart": "beta",
"__default__": {
"Fn::GetAtt": [
"MySubtreeVariable",
"Value"
]
}
},
{
"PathPart": "gamma",
"__default__": {
"Fn::GetAtt": [
"MySubtreeVariable",
"Value"
]
}
}
]
}
}