Wrapper to simplify writing AWS Lambda functions in Rust (using the Python execution environment)
Switch branches/tags
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
assets
examples Merge pull request #44 from jzuber4/upgradecpython Oct 2, 2018
src
test
.gitignore
.travis.yml Add Travis CI Support and Badge Mar 28, 2018
CODE_OF_CONDUCT.md
Cargo.toml
LICENSE-APACHE
LICENSE-MIT Update copyright year Oct 27, 2017
README.md
Vagrantfile
ansible.cfg
requirements.yml Add Vagrant Development Environment Apr 9, 2018
vagrant.yml

README.md

rust-crowbar

Build Status crates.io docs.rs

logo

crowbar makes it easy to write AWS Lambda functions in Rust. It wraps native Rust functions into CPython modules that handle converting Python objects into Rust objects and back again.

Usage

Add both crowbar and cpython to your Cargo.toml:

[dependencies]
crowbar = "0.3"
cpython = "0.2"

Use macros from both crates:

#[macro_use(lambda)]
extern crate crowbar;
#[macro_use]
extern crate cpython;

And write your function using the lambda! macro:

lambda!(|event, context| {
    println!("hi cloudwatch logs, this is {}", context.function_name());
    // return the event without doing anything with it
    Ok(event)
});

Building Lambda Functions

For your code to be usable in AWS Lambda's Python execution environment, you need to compile to a dynamic library with the necessary functions for CPython to run. The lambda! macro does most of this for you, but Cargo still needs to know what to do.

You can configure Cargo to build a dynamic library with the following. If you're using the lambda! macro as above, you need to use lambda for the library name (see the documentation for lambda! if you want to use something else).

[lib]
name = "lambda"
crate-type = ["cdylib"]

cargo build will now build a liblambda.so. Put this in a zip file and upload it to an AWS Lambda function. Use the Python 3.6 execution environment with the handler configured as liblambda.handler.

Build Environment

It is notoriously difficult to build a properly-linked shared library against the Lambda execution environment. Using either an Amazon Linux AMI or the Amazon Linux Docker image, even if using the exact version as Lambda, any package installations will likely upgrade OpenSSL from 1.0.1k to 1.0.2k, causing a linker exception at runtime. @naftulikay lost a ridiculous amount of time trying to statically compile, pin package versions, and try linker hacks.

The best solution available is to use the lambci/lambda:build-python3.6 Docker image which is built from an exact filesystem replica via tarballing the filesystem at runtime from a Python 3.6 runtime Lambda function. The authors went to extensive lengths to pin packages and replicate the environment as accurately as possible, and experience has shown this is the best way to build Python 3.6 shared libraries.

@naftulikay created a sample Rust build environment based on the upstream lambci/lambda:build-python3.6 image at naftulikay/crowbar. Previously, naftulikay/circleci-amazonlinux-rust was used and the aforementioned issues were encountered. Despite CircleCI being used in the name, the image is a fairly generic Rust build environment and should be fairly portable and resuable. For Travis CI and CircleCI examples, please look in the examples/ci directory.

Because you're building a dynamic library, other libraries that you're dynamically linking against need to also be in the Lambda execution environment. By using the lambci/lambda:build-python3.6 image, the build environment will be consistent with the runtime environment.

As described here, the Lambda execution environment uses a runtime library path equivalent to:

LD_LIBRARY_PATH=/lib64:/usr/lib64:$LAMBDA_TASK_ROOT:$LAMBDA_TASK_ROOT/lib

@naftulikay wrote a fairly naïve Python script which will recursively copy linked libraries into the deployment package under lib/. This ensures that any non-standard libraries will be available on the library path at runtime. See the examples/ci/{travis,circle} directories for examples on how to use this, and see naftulikay/docker-crowbar for more information.

Deployment

Serverless framework

Serverless framework is an extemely popular workflow tool for developing and deploying serverless applications and comes with in depth guides for AWS lambda. Serverless framework is surrounded by a strong ecosystem of plugins including a plugin for deploying Rust applications. A serverless rust template for quickly bootstraping, building, and deploying crowbar applications can be found here.

Contributing

crowbar welcomes your contributions:

  • Let us know if you use crowbar in production
  • If you have a bug report or an idea, submit an issue
  • If you want something to work on, check the issues list
  • Please submit non-trivial changes as an issue first; send a pull request when the implementation is agreed on

crowbar follows a code of conduct; please read it.