Skip to content

Commit

Permalink
Update lambda docs with infos about the new provider (#417)
Browse files Browse the repository at this point in the history
Co-authored-by: Daniel Fangl <daniel.fangl@gmail.com>
  • Loading branch information
dominikschubert and dfangl committed Dec 23, 2022
1 parent a0468e8 commit 6a26da2
Show file tree
Hide file tree
Showing 4 changed files with 83 additions and 3 deletions.
18 changes: 17 additions & 1 deletion content/en/references/configuration.md
Expand Up @@ -139,7 +139,23 @@ While the ElasticSearch API is actively maintained, the configuration variables
| `KINESIS_LATENCY` | `500` (default), `0` (to disable)| Integer value of milliseconds, causing the Kinesis API to delay returning a response in order to mimic latency from a live AWS call. |
| `KINESIS_INITIALIZE_STREAMS` | `"my-first-stream:1,my-other-stream:2:us-west-2,my-last-stream:1"` | A comma-delimited string of stream names, its corresponding shard count and an optional region to initialize during startup. If the region is not provided, the default region is used. Only works with the `kinesis-mock` `KINESIS_PROVIDER`. |

### Lambda
### Lambda (New / `asf`)

| Variable| Example Values | Description |
| - | - | - |
| `LAMBDA_TRUNCATE_STDOUT` | `2000` (default) | Allows increasing the default char limit for truncation of lambda log lines when printed in the console.|
| `BUCKET_MARKER_LOCAL` | `hot-reload` (default) | Optional bucket name for running lambdas locally.|
| `LAMBDA_DOCKER_NETWORK` | | Optional Docker network for the container running your lambda function. This configuration value also applies to ECS containers. Needs to be set to the network the LocalStack container is connected to if not default bridge network. |
| `LAMBDA_DOCKER_FLAGS` | `-e KEY=VALUE`, `-v host:container`, `-p host:container`, `--add-host domain:ip` | Additional flags passed to Lambda Docker `run`\|`create` commands (e.g., useful for specifying custom volume mounts). Does only support environment, volume, port and add-host flags |
| `LAMBDA_REMOVE_CONTAINERS` | `1` (default) | Whether to remove containers after Lambdas being inactive for 10 minutes. Only applicable when using docker-reuse executor. |
| `LAMBDA_RUNTIME_EXECUTOR` | `docker` (default) | Where Lambdas will be executed |
| | `kubernetes` | PRO-only. Execute lambdas in a Kubernetes cluster |
| `LAMBDA_RUNTIME_ENVIRONMENT_TIMEOUT` | `10` (default) | How many seconds Lambda will wait for the runtime environment to start up |
| `LAMBDA_RUNTIME_IMAGE_MAPPING` | | Allows two options to customize the resolution of Lambda runtime: <br> 1. pattern with `<runtime>` placeholder, e.g. `custom-repo/lambda-<runtime>:2022` <br> 2. json dict mapping the <runtime> to an image, e.g. `{"python3.9": "custom-repo/lambda-py:thon3.9"}` |
| `LAMBDA_K8S_IMAGE_PREFIX` | `amazon/aws-lambda-` (default) | Prefix for images that will be used to execute Lambda functions in kubernetes. |


### Lambda (Legacy)

| Variable| Example Values | Description |
| - | - | - |
Expand Down
46 changes: 46 additions & 0 deletions content/en/references/lambda-asf-provider.md
@@ -0,0 +1,46 @@
---
title: "Lambda Provider Behavioral Changes"
weight: 5
description: >
Behavioral changes to become default with the new lambda provider
---

## Overview
The new lambda provider has some significant changes in its behavior.
This includes both its API parity, the execution and the container environments themselves.

## API
Parity with AWS has been increased across all Lambda API methods.
Another improvement is the stricter input validation, so expect more invalid input to fail.
One example of this, are Lambda roles. While don't have to actually exist, now have to be at least in a valid ARN format. Using arbitrary strings like `r1` is no longer supported.

## Execution
Lambda will now use the official AWS provided lambda images, instead of lambci images.
This leads to an increased parity with the AWS Lambda environments.
Another effect is that lambda now supports ARM containers, which will be executed depending on your host machine architecture (only available for runtimes based on amazon linux 2).

Lambda will now, by default, reuse containers for each lambda version.
This leads to significantly decreased execution times (from ~800 - 1000ms from the old `docker` executor to around 10ms for a simple echo lambda).
Initialization / global state of your lambdas will stay the same between invocations, as long as it is executed in the same environment.

Multiple concurrent executions will cause LocalStack to scale up and start additional environments for your function, to allow parallel executions.

Timeouts are now properly supported, and will, as in AWS, not kill the environment itself (filesystem stays the same) but only restart the internal infrastructure (your code), as do function errors.


## Changes in Hot Swapping
The magic key for hot reloading (or swapping) buckets has changed from `__local__` to `hot-reload`.
While the former is still supported in the old provider, the new one (`asf`) will only support the latter.
The configuration variable `BUCKET_MARKER_LOCAL` is still respected. Use this if you want to customize its name.
Since the new Lambda provider does not restart the containers after each invocation, even for hot reloading, the filesystem will stay the same.

Since the runtime inside the container will only be restarted after a change, any initialization you do, will only be reset for invocations after a change in the specified code folders.
This leads to faster invocation times if no changes have occurred.

Please keep in mind that any changes to the filesystem (for example in /tmp) will not be deleted if the function code changed.

{{< alert title="Keep in mind" color="warning">}}
It can take up to 700ms for the lambda to detect the changes you did in the code folder. Until that time has passed, any invocations will still be executed on the former code.

If using Docker Desktop, you might need to allow file sharing for your target folders: [https://docs.docker.com/desktop/settings/mac/#file-sharing](https://docs.docker.com/desktop/settings/mac/#file-sharing)
{{< /alert >}}
15 changes: 13 additions & 2 deletions content/en/user-guide/aws/lambda/index.md
Expand Up @@ -8,15 +8,26 @@ aliases:
- /aws/lambda/
---

{{< alert title="Switching to the new implementation" color="warning" >}}
A new implementation of the Lambda service is available since LocalStack 1.3.
Set `PROVIDER_OVERRIDE_LAMBDA=asf` when starting LocalStack and let us know if you experience any issues!

Starting with LocalStack 2.0 the current Lambda implementation will be deprecated in favor of this new provider.
For more information about behavioral changes, please consult the [Lambda Behavioral Changes]({{< ref "references/lambda-asf-provider" >}}) page.
{{< /alert >}}


AWS Lambda is a Serverless Function as a Service (FaaS) system that allows you to write code in your favorite programming language and run it on the AWS ecosystem. Unlike deploying your code on a server, you can now break down your application into many independent functions and deploy them as a singular units. With the help of AWS Lambda, you can strive for more modular code that can be tested and debugged while integrated with the AWS infrastructure and your core system.

LocalStack allows you to execute your Lambda functions locally, without the need to deploy them to AWS. This is a great way to test your code, and to learn more about how your Lambda functions work, before deploying them to AWS. LocalStack allows you to execute your Lambda functions, in various execution modes, which is detailed on our [Lambda execution modes]({{< ref "lambda-executors" >}}) page.

## Lambda Providers

LocalStack's Lambda support is available via two providers: `old` and `asf`. For users, switching between the two providers has a lot of impacts. Using the `PROVIDER_OVERRIDE_lambda`, you can switch between the two providers. The `old` provider is the default provider, and the `asf` provider is ASF, our new and more stable provider. The `old` provider is loaded by default, and you need to set `PROVIDER_OVERRIDE_lambda=asf` to use the ASF provider.
LocalStack's Lambda support is available via two providers, the old one and our new and more stable provider (labeled `asf` after our new internal service framework). Set the `PROVIDER_OVERRIDE_LAMBDA` variable to switch between the two providers, e.g. `PROVIDER_OVERRIDE_LAMBDA=asf`.

With v2.0, the default will be changed to ASF, but the old provider will still be available (using the feature flag with the value `legacy`), though it will be removed in further releases. Currently, `asf_pro`, meant for licensed users for LocalStack, isn't available.
By default LocalStack still uses the old provider.
With LocalStack v2.0, this will be changed to ASF, but the old provider will still be available, in case you need some additional time to migrate.
This fallback option will then be removed in further releases.

## Special tooling for Lambdas

Expand Down
Expand Up @@ -21,6 +21,7 @@ This way, any saved change inside your source file directly affects the already

## Covered Topics

[Using the new Lambda provider](#using-the-new-lambda-provider):
[Application Configuration Examples](#application-configuration-examples):
* [Code hot-swapping for JVM Lambdas](#code-hot-swapping-for-jvm-lambdas)
* [Code hot-swapping for Python Lambdas](#code-hot-swapping-for-python-lambdas)
Expand All @@ -33,6 +34,12 @@ This way, any saved change inside your source file directly affects the already

[Useful Links](#useful-links)

## Using the new Lambda provider
When using the new Lambda provider, hot-reloading does not conflict with fast execution times, and it is not needed to spawn a new container for every invocation.
Also, hot reloading for Lambda layers (Pro) is now supported, it works identical to functions, but only 1 layer is allowed per function if hot-reloading is active in the layer.

For more information about behavioral changes, please consult the [Lambda Behavioral Changes]({{< ref "references/lambda-asf-provider" >}}) page.

## Application Configuration Examples

### Code hot-swapping for JVM Lambdas
Expand Down

0 comments on commit 6a26da2

Please sign in to comment.