Mongo Rust Lambda Demo

Provides a demo of an AWS Lambda function written in Rust which uses MongoDB's Rust Driver to connect to and insert records into a remote MongoDB database.

AWS provides specific language Lambda runtimes for some programming languages and a custom Lambda runtime for other languages (including Rust). This demo uses an AWS custom runtime based on Amazon Linux 2. AWS provides an open source runtime API for Rust: the aws-lambda-rust-runtime crate. This crate manages the instantiation of the deployed Rust Lambda function (deployed as a compiled executable called bootstrap), and the subsequent invocation of this Lambda function's core logic (called a handler) for each request/event dispatched to it. It also provides an API for the Lambda function's main logic to extract a request payload, query the context it is running in and generate a response each time it is invoked. There's no inherent performance impact when using a custom runtime versus a 'standard' runtime. Indeed, the sorts of programming languages that demand a custom Lambda runtime (e.g. C++, Rust) tend to be more efficient anyway.

AWS provides a documented example for deploying a generic Rust Lambda function. The project here serves a more specific purpose of giving an example for creating and deploying a Rust Lambda function to interact with a MongoDB database. Furthermore, whether developing a Lambda function to interact with a MongoDB database in Rust or any other programming language, you should ensure you follow MongoDB's Best Practices Connecting from AWS Lambda.

For a description of what the example Rust Lambda function code does, see section Lambda Rust Code Description at the base of this page.

How To Deploy, Test And Monitor

Prerequisites

1. Ensure you have a remote MongoDB cluster (self-managed or hosted in the Atlas DBaaS - which can be the free tier) which is network accessible from your client workstation.

2. Ensure the MongoDB cluster you are connecting to has a database user available with at least write privileges to the database called test. If you are using an Atlas hosted MongoDB database, you will additionally need to follow the steps in section Restrict network access to your Atlas cluster of the best practices to enable your subsequently deployed Lambda function to be able to access the database.

3. Install the AWS Command Line Interface (AWS CLI) (version 2) including its prerequisites and then test you have configured the AWS CLI correctly on your workstation by attempting to run the following command from a terminal to list the currently deployed AWS Lambda functions in your AWS account region (there may be none):

   aws lambda list-functions

4. Install the latest version of the Rust development environment, if it isn't already installed. Then run the following command from a terminal to install the Rust toolchain for the target OS environment the Lambda function will execute in (i.e. x86-64 Linux):

   rustup target add x86_64-unknown-linux-gnu

5. Via the AWS console for your AWS account, create a new lambda execution IAM role in the IAM section of the console using the following steps:

   • Choose Create role and under Common use cases, choose Lambda.
   • Choose Next: Permissions and under Attach permissions policies, choose the AWS managed policies AWSLambdaBasicExecutionRole and AWSXRayDaemonWriteAccess.
   • Choose Next: Tags, choose Next: Review and for Role name, enter a new role name with any value, e.g. "jdoe-lambda-role".
   • Choose Create role and once created make a copy the ARN for the role for use later.

Compilation

• From a terminal, in this Github project's root folder, run commands to 1) build the project's executable for the target environment, and 2) rename the executable to bootstrap and bundle it into a zip file ready for deployment (as required by AWS Lambda):

cargo build --release --target x86_64-unknown-linux-gnu
rm -f mongo-rust-lambda-demo.zip && cp ./target/x86_64-unknown-linux-gnu/release/mongo-rust-lambda-demo ./bootstrap && zip mongo-rust-lambda-demo.zip bootstrap && rm -f bootstrap

Deployment

• Run the following AWS CLI command to deploy the Lambda function zipped executable, first changing the values of the argument for --role to match the role ARN you copied in the Prerequisites, and the MongoDB URL part of the-environment argument to match the URL of your own MongoDB database including username and password (i.e. replace mongodb+srv://myuser:mypassword@mycluster.a123z.mongodb.net/):

aws lambda create-function --function-name mongo-rust-lambda-demo \
  --handler doesnt.matter \
  --zip-file fileb://./mongo-rust-lambda-demo.zip \
  --runtime provided.al2 \
  --role arn:aws:iam::637263836326:role/jdoe-lambda-role \
  --tracing-config Mode=Active \
  --environment Variables="{MONGODB_URL=mongodb+srv://myuser:mypassword@mycluster.a123z.mongodb.net/,RUST_BACKTRACE=1,RUST_LOG='error,warn,info'}"

Testing

1. Run the following AWS CLI command to invoke your deployed AWS Lambda function and display its response:

aws lambda invoke --function-name mongo-rust-lambda-demo \
  --payload '{"message": "Hi from Jane"}' \
  --cli-binary-format raw-in-base64-out \
  output.json && cat output.json

    NOTE 1: The response from the Rust executable includes an invocation_count field which shows how many times the specific instance of the Lambda function has been invoked (there could be more than one instance when under load). If you run the test command repeatedly in a short space of time, you should see this number increment each time. Suppose you wait more than roughly 15 minutes before invoking the test again. In that case, you will likely see the count reset to one because the AWS Lambda runtime will have destroyed the existing Lambda function instance, having been idle, and will have instantiated a new instance upon receiving this later request.

    NOTE 2: In real-world environments, you wouldn't be using the AWS CLI to invoke your Lambda function, and instead, you might be triggering it synchronously via an HTTP API endpoint or asynchronously via an AWS S3 or SNS event for example.

2. Use the MongoDB Shell (mongosh) to inspect the document inserted each time the Lambda function was invoked with the previous command by running the following (first change the MongoDB URL argument to match the URL of your own MongoDB database including username and password):

mongosh "mongodb+srv://myuser:mypassword@mycluster.a123z.mongodb.net/"

use test
db.lambdalogs.find()

Monitoring

There are a few options for monitoring your deployed Lambda function, including:

1. From the Functions page of the Lambda console you can use the Monitor tab to view when your Lambda function was invoked, its output and other statistics.

2. Run the following AWS CLI command to tail the emitted log events for the Lambda function (then invoke the AWS CLI test again to see logged output):

aws logs tail /aws/lambda/mongo-rust-lambda-demo --follow

OPTIONAL: Redeploying

If you make any changes/enhancements to the Rust code, you can rebuild and deploy the new version with the following commands:

cargo build --release --target x86_64-unknown-linux-gnu
rm -f mongo-rust-lambda-demo.zip && cp ./target/x86_64-unknown-linux-gnu/release/mongo-rust-lambda-demo ./bootstrap && zip mongo-rust-lambda-demo.zip bootstrap && rm -f bootstrap

aws lambda update-function-code --function-name mongo-rust-lambda-demo \
  --zip-file fileb://./mongo-rust-lambda-demo.zip

Building From Scratch

For reference, below are some of the commands that can be run to check, compile, build and test the Rust code outside of the AWS Lambda runtime:

# Compile the Rust application to an execetuable runnable on the host workstation
cargo build

# Run the unit tests for the Rust application
cargo test

# Run the application (inside an integration test) on the local worstation (this enables the 
#  majority of the code to be executed outside of the AWS Lambda runtime for rapid testing) 
# IMPORTANT: First change the MongoDB URL listed below to match the URL of your database
RUST_LOG="error,warn,info" MONGODB_URL="mongodb+srv://myuser:mypassword@mycluster.a123z.mongodb.net/" cargo test -- --ignored --show-output

# Run Rust's lint checks to catch common mistakes and suggest where code can be improved:
cargo clippy

# Run Rust's layout format checks to ensure consistent code formatting is used:
cargo fmt -- --check

Lambda Rust Code Description

This project contains a single Rust source file:  main.rs

Notes about the Rust code:

• The two key functions are:
  • main() - The Lambda function initialisation code in this demo instantiates a static reference to a new instance of a MongoDB Driver's client for communicating with the remote database (after first reading the database's URL from the environment variable MONGODB_URL). Environment variables are the standard Lambda way to provide context metadata to your Lambda function code. This metadata was declared when the AWS CLI command aws lambda create-function was used earlier. Finally, the main function declares the handler function (see next sub-bullet) to the Lambda runtime.
  • handler() - The Lambda function handler code is invoked every time the AWS Lambda receives a request. This uses the AWS Lambda API to read the request JSON payload and some other context data about the Lambda function instance. It then invokes some code to insert a log record into a MongoDB database by using the MongoDB Client instantiated earlier in main(). Finally, it returns a new JSON payload to the caller.
• The code declares a Rust structure called DBLogRecord used in the function db_insert_record() to populate with data ready to be inserted into the MongoDB database collection. The call to the MongoDB Driver's collection.insert_one() API automatically transforms the data structure into a MongoDB BSON document to be inserted. The driver transparently uses the Rust serialisation/deserialisation library called serde to covert the data structure to a BSON document.
• Most of the logic for this Lambda function is delegated to a function named process_work() and functions that it then calls. This enables the bulk of the code to be executed outside of the Lambda runtime, directly on your workstation, for rapid prototyping. Specifically, the integration test function integration_test_execute_full_flow(), near the end of the source file, invokes the process_work() function to execute the main logic end-to-end.
• The source file also contains a set of unit tests.