User-friendly deployment and management tool for AWS Lambda function
Python JavaScript

README.md

Lamvery

Build Status Coverage Scrutinizer Code Quality Version

Description

User-friendly deploy and management tool for AWS Lambda function.

Why user-friendly?

The format of the configuration file is YAML(with Jinja2)

  • YAML is user-friendly than JSON
  • We can avoid some redundant contents by Jinja2 template engine

Additional features that are not in the standard Lambda functions

  • Passing the environment variables
  • Passing the confidential information use KMS encryption
  • Passing the confidential files use KMS encryption
  • And more

More useful features for deploying and invoking our functions

  • We can build, configure, deploy, rollback and invoke with a single command
  • We can rollback correctly to the previous version of the alias
  • Deply(build) hooks
  • And more

More useful features for using and managing the related services

  • API Gateway (Swagger)
  • CloudWatch Events
  • CloudWatch Logs
  • And more

Requirements

  • Python2.7
  • pip

Recommends

  • virtualenv
    Automatically collect the lightweighted and compiled libraries in the virtualenv environment.

Installation

PyPI

pip install lamvery

Apt

echo "deb https://dl.bintray.com/willyworks/deb trusty main" | sudo tee -a /etc/apt/sources.list
sudo apt-get update
sudo apt-get install lamvery
export PATH=/opt/lamvery/bin:$PATH

Yum

echo "
[bintraybintray-willyworks-rpm]
name=bintray-willyworks-rpm
baseurl=https://dl.bintray.com/willyworks/rpm/centos/\$releaserver/\$basearch/
gpgcheck=0
enabled=1
" | sudo tee -a /etc/yum.repos.d/bintray-willyworks-rpm.repo
sudo yum install lamvery
export PATH=/opt/lamvery/bin:$PATH

Setup and configuration

First,

lamvery init

And then edit your .lamvery.yml like so.
The configuration is written in YAML syntax with jinja2 template.
Environment variables are stored in the env variable.

General settings (default: .lamvery.yml)

profile: private
region: us-east-1
versioning: true
default_alias: test
clean_build: false
configuration:
  name: lamvery-test
  runtime: python2.7
  role: {{ env['AWS_LAMBDA_ROLE'] }}
  handler: lambda_function.lambda_handler
  description: This is sample lambda function.
  timeout: 10
  memory_size: 128
  environment_variables:
    MYENV: foobar
  vpc_config:
    subnets:
    - subnet-cadf2993
    security_groups:
    - sg-4d095028

profile

The name of a profile to use. If not given, it depends on boto3.

region

The region name of your environment.
If you doesn't set this option, it depends on boto3.

versioning

Enable the function versioning.

default_alias

The alias when it has not been specified in the -a or --alias option.

clean_build

Build the archive(zip) in the temporary area.

configuration

  • name
    The name of your function.

  • runtime
    The runtime environment for the Lambda function you are uploading.
    Currently, lamvery supports python2.7 and nodejs.

  • role
    The Amazon Resource Name (ARN) of the IAM role for your function.

  • handler
    The function within your code that Lambda calls to begin execution.

  • description
    The description of your function.

  • timeout
    The function execution time(seconds) at which Lambda should terminate the function.

  • memory_size
    The amount of memory for your function environment.

  • environment_variables
    Environment variables.

  • vpc_config
    The VPC configurations for the function to access resources in your VPC.

    • subnets
      The Subnet ids in your VPC.
    • security_groups
      The SecurityGroup ids in your VPC.

CloudWatch Events settings (default: .lamvery.event.yml)

rules:
- name: foo
  description: bar
  schedule: 'rate(5 minutes)'
  targets:
  - id: test-target-id
    input:
      this:
      - is: a
      - sample: input

rules

CloudWatch Event Rules.

  • NAME
    The name of CloudWatch Event Rule.

  • description
    The description of CloudWatch Event Rule.

  • schedule
    The schedule expression of CloudWatch Event Rule.

  • disabled
    When this setting is true, change the state of CloudWatch Event Rule to DISABLED.
    default: false

  • targets
    The targets of CloudWatch Event Rule.

    • id
      The unique target assignment ID.
    • input
      Arguments passed to the target.
    • input_path
      The value of the JSONPath that is used for extracting part of the matched event when passing it to the target.

input and input_path are mutually-exclusive and optional parameters of a target.

Secret informations (default: .lamvery.secret.yml)

key_id: {{ env['AWS_KMS_KEY_ID'] }}
cipher_texts:
  foo: CiC4xW9lg7HaxaueeN+d9yJMyY1uw1i7tYVvQz9I8+e2UBKXAQEBAgB4uMVvZYOx2sWrnnjfnfciTMmNbsNYu7WFb0M/SPPntlAAAABuMGwGCSqGSIb3DQEHBqBfMF0CAQAwWAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAzWTJWk/69T8NTBquoCARCAK2Hg2de71hzwjiMKkfMSG2G1Olj1EjxajS+3PsFVTPZ91Oi/AjR1aMqAI8U=

key_id

The ID of your encryption key on KMS.

cipher_texts

The name and cipher texts to pass to the lambda function.

secret_files

The filename and the encrypted contents to pass to the lambda function.

Excluded patterns from the archive (default: .lamvery.exclude.yml)

- ^\.lamvery\.yml$
- ^\.lamvery\.event\.yml$
- ^\.lamvery\.secret\.yml$
- ^\.lamvery\.exclude\.yml$

Exclude files or directories using regular expression.

Action hooks (default: .lamvery.hook.yml)

build:
  pre:
  - pip install -r requirements.txt -t ./
  post: []

build

The hooks for build(and deploy)

  • pre
    The commands that run before building.

  • post
    The commands that run after building.

API Gateway integration (default: .lamvery.api.yml)

api_id: myipugal74
stage: dev
cors:
  origin: '*'
  methods:
  - GET
  - OPTION
  headers:
  - Content-Type
  - X-Amz-Date
  - Authorization
  - X-Api-Key
configuration:
  swagger: '2.0'
  info:
    title: Sample API
  schemes:
  - https
  paths:
    /:
      get:
        produces:
        - application/json
        parameters:
        - name: sample
          in: query
          required: false
          type: string
        responses:
          '200':
            description: 200 response
            schema:
              $ref: '#/definitions/Sample'
  definitions:
    Sample:
      type: object

api_id

The id of your REST API.
This is written on automatically when you deployed your API with the -w option.

stage

The name of the stage in API Gateway.

cors

CORS (Cross-Origin Resource Sharing) options.
If you did not set the x-amazon-apigateway-integration option, these are set automatically.

  • origin
    For the response's header that named "Access-Control-Allow-Origin".
  • headers
    For the response's header that named "Access-Control-Allow-Headers".
  • methods
    For the response's header that named "Access-Control-Allow-Methods".

configuration

The settings of your APIs written in the Swagger format.
If you did not set the x-amazon-apigateway-integration option, these are set automatically.

Commands

build

  • Build and archive your code and libraries to <your-function-name>.zip
  • Store the secret informations to the archive
lamvery build [-e <env-name>=<env-value>]

deploy

  • Build and deploy your code and libraries
  • Store the secret informations to the archive
  • Update configuration of the function
  • Set alias to a version of the function
lamvery deploy [-e <env-name>=<env-value>] [-a <alias>]

rollback

  • Rollback to the previous version of the function

You must do one of the following to use this command.

  • Deploy with publish(-p,--publish) and alias(-a, --alias) options.
  • Turn on(true) versioning and set a value to default_alias in the configuration file.
lamvery rollback [-a <alias>]

set-alias

  • Set alias to a version of the function
lamvery set-alias -a <alias> -v <alias-version>

encrypt

  • Encrypt a text value using KMS
lamvery encrypt [-s] -n <secret-name> <secret-value>

encrypt-file

  • Encrypt a file using KMS
lamvery encrypt [-s] -n <filename> <local-file-path>

decrypt

  • Decrypt the secret information using KMS
lamvery decrypt -n <secret-name>

events

  • Apply CloudWatch Events settings
lamvery events [-k] [-a <alias>]

invoke

  • Invoke the function and output result
lamvery invoke [-a <alias>] [-v <version>] '{"foo": "bar"}'

or

lamvery invoke [-a <alias>] [-v <version>] path/to/input.json

logs

  • Watch the log events of the function
lamvery logs [-f] [-F <filter>] [-s <start-time-string>] [-i <interval-seconds>]

api

  • Manage your APIs
lamvery api [-n] [-r] [-s <stage-name>] [-w]

generate

  • Generate skeleton files
lamvery generate -k <kind-of-file>

Options

-a or --alias

This option is needed by the deploy,set-alias,invoke,rollback,events,api commands.
Alias for a version of the function.

-c or --conf-file

This option needed by all commands.
Specify the configuration file.
default: .lamvery.yml

-d or --dry-run

This option is needed by the deploy,alias,rollback,events,api commands.
Output the difference of configuration and the alias without updating.

-s or --single-file

This option is needed by the archive and deploy command.
Archive only the lambda function file, so you can inline edit in the AWS Management Console.

-l or --no-libs

This option is needed by the archive and deploy command.
Archive without all libraries.

-n or --secret-name

This option is needed by the encrypt and decrypt commands.
The name of the secret value.

-p or --publish

This option is only needed by the deploy command. Publish the version as an atomic operation.

-k or --keep-empty-events

This option is only needed by the events command. Keep the empty CloudWatch Event Rule that does not have CloudWatch Event Target.

-s or --store

This option is only needed by the encrypt command.
Store encripted value to configuration file (default: .lamvery.yml).
Requires the -n or --secret-name option.

-v or --version

This option is needed by the set-alias,invoke,rollback commands.
Version of the function.

-f or --follow

This option is only needed by the logs command.
Watch the log events and updates the display (like tail -f).

-F or --filter

This option is only needed by the logs command.
Filtering pattern for the log messages.

-i or --interval

This option is only needed by the logs command.
Intervals(seconds) to watch the log events.

-s or --start

This option is only needed by the logs command.
Time to start the log events watching.
Examples: yesterday,"-1 h", "2016-01-01", "2016-01-01 10:20:30"

-t or --target

This option is only needed by the set-alias command.
The alias of the version that is targeted for setting alias.

-e or --env

This option is needed by the archive and deploy commands.
Environment variables that pass to the function.
This option can be used repeatedly to pass multiple variables.
Examples: FOO=BAR

-r or --remove

This option is only needed by the api command.
Remove your APIs.

-s or --stage

This option is only needed by the api command.
The name of the stage in API Gateway.

-w or --write-id

This option is only needed by the api command.
Write the id of your API to the configuration file (default: .lamvery.api.yml)

-n or --no-integrate

This option is only needed by the api command.
Without automatic integration.

-k or --kind

This option is only needed by the generate command.
The kind of the file.
Allowed values: function

-n or --name

This option is only needed by the encrypt-file command.
The name of the secret file.

How to use the confidential information in the lambda function

1. Create key on KMS

See: https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

2. Create IAM role for lambda function

Policy example:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt"
            ],
            "Resource": [
                "arn:aws:kms:us-east-1:<your-account-number>:key/<your-key-id>"
            ]
        }
    ]
}

3. Set the key-id to your configuration file

Configuration example:

  • .lamvery.yml
profile: default
region: us-east-1
versioning: false
default_alias: null
configuration:
  name: sample_lambda_function
  runtime: python2.7 # or nodejs
  role: arn:aws:iam::000000000000:role/lambda_basic_execution
  handler: lambda_function.lambda_handler
  description: This is sample lambda function.
  timeout: 10
  memory_size: 128
  • .lamvery.secret.yml
key_id: xxxx-yyyy-zzzz # <-here!
cipher_texts: {}
secret_files: {}

4. Encrypt and store the confidential information to your configuration file

Command example:

lamvery encrypt -s -n foo "This is a secret"

5. Write your function

Generate the skeleton function by this command.

lamvery generate -k function

Code example:

  • Python
  import lamvery

  def lambda_handler(event, context):
      print(lamvery.secret.get('foo'))
  • Node.js
var lamvery = require('./lamvery.js');

exports.lambda_handler = function(event, context) {
    lamvery.secret.get('foo', function(err, data) {
        console.log(data);
    });
}

6. Deploy your function

Command example:

lamvery deploy

7. Invoke your function

Command example:

lamvery invoke {}

Result example:

START RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d Version: $LATEST
This is a secret
END RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d

How to use the confidential file in the lambda function

1. Create key on KMS

See: https://docs.aws.amazon.com/kms/latest/developerguide/create-keys.html

2. Create IAM role for lambda function

Policy example:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "kms:Decrypt"
            ],
            "Resource": [
                "arn:aws:kms:us-east-1:<your-account-number>:key/<your-key-id>"
            ]
        }
    ]
}

3. Set the key-id to your configuration file

Configuration example:

  • .lamvery.yml
profile: default
region: us-east-1
versioning: false
default_alias: null
configuration:
  name: sample_lambda_function
  runtime: python2.7 # or nodejs
  role: arn:aws:iam::000000000000:role/lambda_basic_execution
  handler: lambda_function.lambda_handler
  description: This is sample lambda function.
  timeout: 10
  memory_size: 128
  • .lamvery.secret.yml
key_id: xxxx-yyyy-zzzz # <-here!
cipher_texts: {}
secret_files: {}

4. Encrypt and store the confidential file to your configuration file

Command example:

lamvery encrypt-file -s -n foo.txt /path/to/local/confidential/file

5. Write your function

Generate the skeleton function by this command.

lamvery generate -k function

Code example:

  • Python
  import lamvery

  def lambda_handler(event, context):
      print(open(lamvery.secret.file('foo.txt'), 'r').read())
  • Node.js
var lamvery = require('./lamvery.js');

exports.lambda_handler = function(event, context) {
    lamvery.secret.file('foo.txt', function(err, path) {
        fs.readFile(path, 'utf-8', function(err, txt) {
            console.log(txt);
        });
    });
}

6. Deploy your function

Command example:

lamvery deploy

7. Invoke your function

Command example:

lamvery invoke {}

Result example:

START RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d Version: $LATEST
This is a secret file
END RequestId: 13829c9c-9f13-11e5-921b-6f048cff3c2d

Development

Pull requests are very welcome! Make sure your patches are well tested. Ideally create a topic branch for every separate change you make. For example:

  1. Fork the repo
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Added some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request

Authors

Created and maintained by Masashi Terui (marcy9114@gmail.com)

License

MIT License (see LICENSE)