Sub Minute Lambda Executor

When building applications in AWS, developers may be required to poll external systems such as HTTP, REST, or TCP endpoints. Polling the external system every few seconds may be required along with modifying the polling interval, such as polling faster during the day and slower at night. For example, tracking municipal bus predictions, vehicle positions, and alerts by querying restful endpoints. Typically, a developer would use a cron style job scheduler to perform this type of scheduling task. Using AWS serverless services there is no way to create a scheduled task that is less than one minute. Using Amazon CloudWatch, you are able to produce events at intervals of one minute or above, but does not allow for scheduling less that a minute. This solution will demonstrate how to use a set of services to produce scheduled events at a rate of one second or more in a highly accurate manor. The solution will show how to invoke an AWS Lambda function to poll the external systems at scheduled intervals which can be adjusted over time.

Architecture

• AWS Step Functions are used to invoke a Lambda function which controls the final Lambda execution. The AWS Step Function will restart itself once the Lambda completes.
• AWS Lambda is used to control the timing and final asynchronous execution of the worker Lambda which will perform the operation required for the external system.
• Amazon DynamoDB is used to store control flags. The "running" field determines if the system should continue to run. The "waitseconds" field is used to determine the period between worker Lambda invocations.

Setup

There are 2 methods to install the sample. Either via the pre-generated CloudFormation template with inline Lambda functions or via CDK.

CloudFormation

1. A CloudFormation template (SubMinuteLambdaExecutor.yaml) has been pre-generated and can be used directly
2. Follow the Creating a stack on the AWS CloudFormation console documentation
3. During the Selecting a stack template step upload the SubMinuteLambdaExecutor.yaml file provided
4. During the Specifying stack name and parameters step please update the "waitseconds" to the timeout required
5. Complete the remaining steps from the documentation and wait for the stack to deploy
6. The outputs tab of the stack information will display name and arn for the DynamoDB table, Lambda functions, and Step Function.

CDK

1. AWS CDK Toolkit is required
2. From a terminal window at the root directory of this project do cdk deploy
3. When complete, a list of outputs will display name and arn for the DynamoDB table, Lambda functions, and Step Function.

Running

1. To start the system, start execution of the Step Function that was created.
2. The Step Function does not require any inputs

Stopping Execution

1. To stop execution, update the DynamoDB table column value "running" to false

Resource Cleanup

CloudFormation

1. Delete the CloudFormation stack

CDK

1. From a terminal window at the root directory of this project do cdk destroy

Security

See CONTRIBUTING for more information.

License

This library is licensed under the MIT-0 License. See the LICENSE file.