Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document using Martin as a AWS Lambda using PMTiles on S3, or another cloud provider #1102

Open
nyurik opened this issue Dec 25, 2023 · 2 comments

Comments

@nyurik
Copy link
Member

nyurik commented Dec 25, 2023

We urgently need anyone with the AWS Lambda experience to write a short documentation on how to use Martin with Amazon Lambda functions, or similar tech with other cloud providers. The tiles data should be placed on an AWS bucket, with public access. Martin does not support S3 authentication yet, PRs welcome. Note that the bucket URL should not be publicized - rather it should simply be used by the Lambda function, which provides adequate protection from malicious users who may want to download large data directly, thus raising hosting costs.

The markdown doc file would go into https://github.com/maplibre/martin/tree/main/docs/src

nyurik added a commit that referenced this issue Feb 1, 2024
This adds the lambda-web crate to adapt the actix App to speak to Lambda
by way of the lambda_runtime crate.

AWS Lambda has native support for scripting languages to
execute a function directly; compiled languages must embed a runtime to
fetch incoming events from Lambda and post the responses. This detects
the environment variables to start up in Lambda mode instead of the
normal HTTP server, and is added as an optional feature.

Lambda has five (!) distinct ways of routing HTTP requests to a
function; this supports some of them. (Specifically, the most obvious
way to do this is with a Function URL, which is newest and simplest, and
perhaps with CloudFront, which speaks to the Function URL and not Lambda
directly.)

The error handling could probably be refined, I was just trying to get
this to compile.

(Supported: API Gateway HTTP API with payload format version 2.0; API
Gateway REST API; Lambda function URLs / Not supported: API Gateway HTTP
API with payload format version 1.0; Application Load Balancer)

Necessary for #1102 to be able to run the released packages directly,
and only having to configure the appropriate environment.

---------

Co-authored-by: Yuri Astrakhan <yuriastrakhan@gmail.com>
@nyurik
Copy link
Member Author

nyurik commented Feb 2, 2024

There is now documentation page at https://maplibre.org/martin/run-with-lambda.html, thx to @jleedev - but it only applies to the v0.14 of Martin (not yet released). I did a pass at rewording it a bit in bdce8bb, but it may need some more love:

  • add images with AWS user interface to explain how to do all these steps
  • elaborate why zip deployment should use application model
  • ...?

@jleedev
Copy link
Collaborator

jleedev commented Feb 3, 2024

Some context for the choice of how to talk to AWS (as a human): No real reason except that it's a useful way to ensure that the steps are reproducible. (And AWS has a million ways to do everything.)

Certainly possible to document the other method if that's simpler to follow, e.g: Create an empty directory, download the martin binary to bin/martin, write the appropriate script to bootstrap, zip the contents (not inside a directory, e.g. zip martin.zip bootstrap bin/martin), upload the layer in the console, create the function in the console.

Which, perhaps, could be more instructive than "paste this yaml and a bunch of stuff happens", to explore all the objects and settings involved.

For the container deployment, actually, it's possible to use more SAM and not have to create an ECR by hand, which might be an improvement:

AWSTemplateFormatVersion: 2010-09-09
Transform: 'AWS::Serverless-2016-10-31'
Resources:
  demotiles:
    Type: 'AWS::Serverless::Function'
    Properties:
      Architectures:
        - arm64
      PackageType: Image
      ImageConfig:
        Command:
          - '<tile url here>'
      FunctionUrlConfig:
        AuthType: NONE
    Metadata:
      Dockerfile: Dockerfile
      DockerContext: .

And the Dockerfile contains only the line FROM ghcr.io/maplibre/martin:main; and it could also embed a yaml configuration file too. SAM builds the docker image locally, provisions an ECR, and pushes it.

The other advantage of CloudFormation (and SAM/CDK) is that cleanup is easier, you delete the stack and the stack deletes all the stuff.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants