title	description	author	publishedDate	date
API Gateway and Lambda Integration Basic	show a very basic integration of api gateway and lambda	haimtran	06/23/2022	2022-07-24

Introduction

GitHub this demonstrates a very basic example of integrating api gateway with a lambda function.

• lambda handler header for proxy apigw
• apigw and lambda integration
• simple test to see lambda scale concurrency
• add deployment stages
• add access log
• protect api using waf
• apigw data model and transformation, request and response mapping template

Architecture

Lambda Handler

add header to work with api gw proxy integration

import datetime
import time
import json

def handler(event, context) -> json:
    """
    simple lambda function
    """

    # time stamp
    now = datetime.datetime.now()
    time_stamp = now.strftime("%Y/%m/%d %H:%M:%S.%f")

    # sleep
    time.sleep(2)

    return {
        'statusCode': 200,
        'headers': {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Headers": "Content-Type",
            "Access-Control-Allow-Methods": "OPTIONS,GET"
        },
        'body': json.dumps({
            'message': f'lambda {time_stamp} {event}'
        })
    }

Cdk Stack

lambda inline function

// lambda function
const func = new cdk.aws_lambda.Function(this, "HelloLambdaTest", {
  functionName: "HelloLambdaTest",
  code: cdk.aws_lambda.Code.fromInline(
    fs.readFileSync(path.resolve(__dirname, "./../lambda/index.py"), {
      encoding: "utf-8",
    })
  ),
  runtime: cdk.aws_lambda.Runtime.PYTHON_3_8,
  memorySize: 512,
  timeout: Duration.seconds(10),
  handler: "index.handler",
});

api gateway and integration with lambda

// apigatway
const apigw = new cdk.aws_apigateway.RestApi(this, "ApiGwDemo", {
  restApiName: "ApiGwDemo",
});

// resource
const resource = apigw.root.addResource("order");

// method and lambda integration
resource.addMethod("GET", new cdk.aws_apigateway.LambdaIntegration(func));

Concurrency

Send concurrent requests and see how lambda scale

import time
from concurrent.futures import ThreadPoolExecutor
import boto3

# function name
FUNCTION_NAME = "HelloLambdaTest"

# lambda client
lambda_client = boto3.client("lambda")

# number of concurrent request
NUM_CONCUR_REQUEST = 100


def invoke_lambda(id: int) -> str:
    """
    invoke lambda
    """
    res = lambda_client.invoke(
        FunctionName=FUNCTION_NAME
    )

    print(f'lamda {id} {res["Payload"].read()}')
    print("\n")
    return res['Payload'].read()


def test_scale_lambda() -> None:
    """
    Test how lambda scale
    """
    with ThreadPoolExecutor(max_workers=NUM_CONCUR_REQUEST) as executor:
        for k in range(1, NUM_CONCUR_REQUEST):
            executor.submit(invoke_lambda, k)


if __name__ == "__main__":
    while True:
        test_scale_lambda()
        time.sleep(5)

Enable Access Log

Log access to api gw to a cloudwatch loggroup. First, create a iam role which assumed by the api so that it can put event logs to a cloudwatch log group.

const role = new aws_iam.Role(this, "RoleForApiGwInvokeLambda", {
  roleName: "ApiGwInvokeLambda",
  assumedBy: new aws_iam.ServicePrincipal("apigateway.amazonaws.com"),
});

role.addToPolicy(
  new aws_iam.PolicyStatement({
    effect: Effect.ALLOW,
    actions: ["lambda:InvokeFunction"],
    resources: [func.functionArn],
  })
);

role.addToPolicy(
  new aws_iam.PolicyStatement({
    effect: Effect.ALLOW,
    actions: [
      "logs:CreateLogGroup",
      "logs:CreateLogStream",
      "logs:DescribeLogGroups",
      "logs:DescribeLogStreams",
      "logs:PutLogEvents",
      "logs:GetLogEvents",
      "logs:FilterLogEvents",
    ],
    resources: ["*"],
  })
);

create a log group to store the log

// access log group - prod stage
const prodLogGroup = new aws_logs.LogGroup(this, "ProdLogGroup", {
  logGroupName: "ProdLogGroupAccessLog",
  removalPolicy: RemovalPolicy.DESTROY,
});

Deployment and Stage

Disable the default deployment and add two stages later on

const apiGw = new aws_apigateway.RestApi(this, "HelloApiGw", {
  restApiName: "HelloApiGw",
  deploy: false,
});

const book = apiGw.root.addResource("book");

add prod stage

book.addMethod(
  "GET",
  new aws_apigateway.LambdaIntegration(func, {
    proxy: true,
    allowTestInvoke: false,
    credentialsRole: role,
    integrationResponses: [
      {
        statusCode: "200",
      },
    ],
  }),
  // method options
  {
    // required for non-proxy
    methodResponses: [{ statusCode: "200" }],
  }
);

create a cloudwatch loggroup

const devLogGroup = new aws_logs.LogGroup(this, "ApiAccessLogGroup", {
  logGroupName: "DevLogGroupAccessLog",
  removalPolicy: RemovalPolicy.DESTROY,
});

create a deployment

const deployment = new aws_apigateway.Deployment(this, "Deployment", {
  api: apiGw,
});

deploy the dev stage

const devStage = new aws_apigateway.Stage(this, "DevStage", {
  stageName: "dev",
  deployment,
  dataTraceEnabled: true,
  accessLogDestination: new aws_apigateway.LogGroupLogDestination(devLogGroup),
  accessLogFormat: aws_apigateway.AccessLogFormat.jsonWithStandardFields(),
});

deploy the prod stage

const prodStage = new aws_apigateway.Stage(this, "ProdStage", {
  stageName: "prod",
  deployment,
  dataTraceEnabled: true,
  accessLogDestination: new aws_apigateway.LogGroupLogDestination(prodLogGroup),
  accessLogFormat: aws_apigateway.AccessLogFormat.jsonWithStandardFields(),
});

store and publish stage arn so we can attach waf to these arn later on inside the waf stack

this.apiArns.push(prodStage.stageArn);

Protect API using WAF

create three waf rules to protect the api. First rule is an AWS managed rule which block IP addresses typically associated with bots from Amazon internal threat intelligence

const awsMangedRuleIPReputationList: aws_wafv2.CfnWebACL.RuleProperty = {
  name: "AWSManagedRulesCommonRuleSet",
  priority: 10,
  statement: {
    managedRuleGroupStatement: {
      name: "AWSManagedRulesCommonRuleSet",
      vendorName: "AWS",
    },
  },
  overrideAction: { none: {} },
  visibilityConfig: {
    sampledRequestsEnabled: true,
    cloudWatchMetricsEnabled: true,
    metricName: "AWSIPReputationList",
  },
};

second waf rule geo restrict block from a list of countries

const ruleGeoRestrict: aws_wafv2.CfnWebACL.RuleProperty = {
  name: "RuleGeoRestrict",
  priority: 2,
  action: {
    block: {},
  },
  statement: {
    geoMatchStatement: {
      countryCodes: ["SG"],
    },
  },
  visibilityConfig: {
    sampledRequestsEnabled: true,
    cloudWatchMetricsEnabled: true,
    metricName: "GeoMatch",
  },
};

third rule is ip rate based which block if more than 2000 request per second from an IP address

const ruleLimiteRequestsThreshold: aws_wafv2.CfnWebACL.RuleProperty = {
  name: "LimiteRequestsThreshold",
  priority: 1,
  action: {
    block: {},
  },
  statement: {
    // 2000 requests within 5 minutes
    rateBasedStatement: {
      limit: 2000,
      aggregateKeyType: "IP",
    },
  },
  visibilityConfig: {
    sampledRequestsEnabled: true,
    cloudWatchMetricsEnabled: true,
    metricName: "LimitRequestsThreshold",
  },
};

API WAF Test

install npm and nodejs 16 for Amazon linux 2

wget https://nodejs.org/dist/v16.0.0/
tar -xvf node-v16.0.0-linux-x64.tar.gz
export PATH=/home/ec2-user/node-v16.0.0-linux-x64/bin:$PATH

install artillery

npm install -g artillery

using Artillery, send more than 2000 requests from client using 10 threads

artillery quick -n 2100 --count 10 ENDPOINT

then the IP will be blocked and received 403 (fobiden error) in the following requests

Api Gateway Model and Transform

first example is to create a GET request and using both request, and response mapping template. Request mapping template in TVL

{
  "id": "$input.params('userid')",
  "name": "$input.params('username')"
}

and response mapping template

#set($inputRoot = $input.path('$'))
{
    "id": "$inputRoot.body.id",
    "message": "$inputRoot.body.message"
}

testing GET with request query parameter, print event or echo the event in lambda to see what Lambda receives from api gateway

userid=111&username=haimtran

expected response as the response mapping template

{
  "id": "",
  "message": ""
}

Update

setup apigw to push log to cloudwatch log group

const apiGw = new aws_apigateway.RestApi(this, "HelloApiGw", {
  restApiName: "HelloApiGw",
  cloudWatchRole: true, 
  deploy: false,
  // deployOptions: {
  //   stageName: "prod",
  //   accessLogDestination: new aws_apigateway.LogGroupLogDestination(
  //     prodLogGroup
  //   ),
  //   accessLogFormat:
  //     aws_apigateway.AccessLogFormat.jsonWithStandardFields(),
  // },
});

Reference

• Lambda optmization
• Lambda re-invent 2021
• WAF protect API
• Apigw integration timeout
• Apigw mapping template