Introduction#

GitHub this demonstrates a very basic example of integrating api gateway with a lambda function.

  • lambda handler header for proxy apigw
  • apigw and lambda integration
  • simple test to see lambda scale concurrency
  • add deployment stages
  • add access log
  • protect api using waf
  • apigw data model and transformation, request and response mapping template
apigw lambda basic

Architecture#

Untitled Diagram drawio

Lambda Handler#

add header to work with api gw proxy integration

import datetime
import time
import json


def handler(event, context) -> json:
    """
    simple lambda function
    """


    # time stamp
    now = datetime.datetime.now()
    time_stamp = now.strftime("%Y/%m/%d %H:%M:%S.%f")


    # sleep
    time.sleep(2)


    return {
        'statusCode': 200,
        'headers': {
            "Access-Control-Allow-Origin": "*",
            "Access-Control-Allow-Headers": "Content-Type",
            "Access-Control-Allow-Methods": "OPTIONS,GET"
        },
        'body': json.dumps({
            'message': f'lambda {time_stamp} {event}'
        })
    }

Cdk Stack#

lambda inline function

// lambda function
const func = new cdk.aws_lambda.Function(this, 'HelloLambdaTest', {
  functionName: 'HelloLambdaTest',
  code: cdk.aws_lambda.Code.fromInline(
    fs.readFileSync(path.resolve(__dirname, './../lambda/index.py'), {
      encoding: 'utf-8'
    })
  ),
  runtime: cdk.aws_lambda.Runtime.PYTHON_3_8,
  memorySize: 512,
  timeout: Duration.seconds(10),
  handler: 'index.handler'
})

api gateway and integration with lambda

// apigatway
const apigw = new cdk.aws_apigateway.RestApi(this, 'ApiGwDemo', {
  restApiName: 'ApiGwDemo'
})


// resource
const resource = apigw.root.addResource('order')


// method and lambda integration
resource.addMethod('GET', new cdk.aws_apigateway.LambdaIntegration(func))

Concurrency#

Send concurrent requests and see how lambda scale

import time
from concurrent.futures import ThreadPoolExecutor
import boto3


# function name
FUNCTION_NAME = "HelloLambdaTest"


# lambda client
lambda_client = boto3.client("lambda")


# number of concurrent request
NUM_CONCUR_REQUEST = 100




def invoke_lambda(id: int) -> str:
    """
    invoke lambda
    """
    res = lambda_client.invoke(
        FunctionName=FUNCTION_NAME
    )


    print(f'lamda {id} {res["Payload"].read()}')
    print("\n")
    return res['Payload'].read()




def test_scale_lambda() -> None:
    """
    Test how lambda scale
    """
    with ThreadPoolExecutor(max_workers=NUM_CONCUR_REQUEST) as executor:
        for k in range(1, NUM_CONCUR_REQUEST):
            executor.submit(invoke_lambda, k)




if __name__ == "__main__":
    while True:
        test_scale_lambda()
        time.sleep(5)

Enable Access Log#

Log access to api gw to a cloudwatch loggroup. First, create a iam role which assumed by the api so that it can put event logs to a cloudwatch log group.

const role = new aws_iam.Role(this, 'RoleForApiGwInvokeLambda', {
  roleName: 'ApiGwInvokeLambda',
  assumedBy: new aws_iam.ServicePrincipal('apigateway.amazonaws.com')
})


role.addToPolicy(
  new aws_iam.PolicyStatement({
    effect: Effect.ALLOW,
    actions: ['lambda:InvokeFunction'],
    resources: [func.functionArn]
  })
)


role.addToPolicy(
  new aws_iam.PolicyStatement({
    effect: Effect.ALLOW,
    actions: [
      'logs:CreateLogGroup',
      'logs:CreateLogStream',
      'logs:DescribeLogGroups',
      'logs:DescribeLogStreams',
      'logs:PutLogEvents',
      'logs:GetLogEvents',
      'logs:FilterLogEvents'
    ],
    resources: ['*']
  })
)

create a log group to store the log

// access log group - prod stage
const prodLogGroup = new aws_logs.LogGroup(this, 'ProdLogGroup', {
  logGroupName: 'ProdLogGroupAccessLog',
  removalPolicy: RemovalPolicy.DESTROY
})

Deployment and Stage#

Disable the default deployment and add two stages later on

const apiGw = new aws_apigateway.RestApi(this, 'HelloApiGw', {
  restApiName: 'HelloApiGw',
  deploy: false
})


const book = apiGw.root.addResource('book')

add prod stage

book.addMethod(
  'GET',
  new aws_apigateway.LambdaIntegration(func, {
    proxy: true,
    allowTestInvoke: false,
    credentialsRole: role,
    integrationResponses: [
      {
        statusCode: '200'
      }
    ]
  }),
  // method options
  {
    // required for non-proxy
    methodResponses: [{ statusCode: '200' }]
  }
)

create a cloudwatch loggroup

const devLogGroup = new aws_logs.LogGroup(this, 'ApiAccessLogGroup', {
  logGroupName: 'DevLogGroupAccessLog',
  removalPolicy: RemovalPolicy.DESTROY
})

create a deployment

const deployment = new aws_apigateway.Deployment(this, 'Deployment', {
  api: apiGw
})

deploy the dev stage

const devStage = new aws_apigateway.Stage(this, 'DevStage', {
  stageName: 'dev',
  deployment,
  dataTraceEnabled: true,
  accessLogDestination: new aws_apigateway.LogGroupLogDestination(devLogGroup),
  accessLogFormat: aws_apigateway.AccessLogFormat.jsonWithStandardFields()
})

deploy the prod stage

const prodStage = new aws_apigateway.Stage(this, 'ProdStage', {
  stageName: 'prod',
  deployment,
  dataTraceEnabled: true,
  accessLogDestination: new aws_apigateway.LogGroupLogDestination(prodLogGroup),
  accessLogFormat: aws_apigateway.AccessLogFormat.jsonWithStandardFields()
})

store and publish stage arn so we can attach waf to these arn later on inside the waf stack

this.apiArns.push(prodStage.stageArn)

Protect API using WAF#

create three waf rules to protect the api. First rule is an AWS managed rule which block IP addresses typically associated with bots from Amazon internal threat intelligence

const awsMangedRuleIPReputationList: aws_wafv2.CfnWebACL.RuleProperty = {
  name: 'AWSManagedRulesCommonRuleSet',
  priority: 10,
  statement: {
    managedRuleGroupStatement: {
      name: 'AWSManagedRulesCommonRuleSet',
      vendorName: 'AWS'
    }
  },
  overrideAction: { none: {} },
  visibilityConfig: {
    sampledRequestsEnabled: true,
    cloudWatchMetricsEnabled: true,
    metricName: 'AWSIPReputationList'
  }
}

second waf rule geo restrict block from a list of countries

const ruleGeoRestrict: aws_wafv2.CfnWebACL.RuleProperty = {
  name: 'RuleGeoRestrict',
  priority: 2,
  action: {
    block: {}
  },
  statement: {
    geoMatchStatement: {
      countryCodes: ['SG']
    }
  },
  visibilityConfig: {
    sampledRequestsEnabled: true,
    cloudWatchMetricsEnabled: true,
    metricName: 'GeoMatch'
  }
}

third rule is ip rate based which block if more than 2000 request per second from an IP address

const ruleLimiteRequestsThreshold: aws_wafv2.CfnWebACL.RuleProperty = {
  name: 'LimiteRequestsThreshold',
  priority: 1,
  action: {
    block: {}
  },
  statement: {
    // 2000 requests within 5 minutes
    rateBasedStatement: {
      limit: 2000,
      aggregateKeyType: 'IP'
    }
  },
  visibilityConfig: {
    sampledRequestsEnabled: true,
    cloudWatchMetricsEnabled: true,
    metricName: 'LimitRequestsThreshold'
  }
}

API WAF Test#

using Artillery, send more than 2000 requests from client using 10 threads

artillery quick -n 2100 --count 10 ENDPOINT

then the IP will be blocked and received 403 (fobiden error) in the following requests

Api Gateway Model and Transform#

first example is to create a GET request and using both request, and response mapping template. Request mapping template in TVL

{
  "id": "$input.params('userid')",
  "name": "$input.params('username')"
}

and response mapping template

#set($inputRoot = $input.path('$'))
{
    "id": "$inputRoot.body.id",
    "message": "$inputRoot.body.message"
}

testing GET with request query parameter, print event or echo the event in lambda to see what Lambda receives from api gateway

userid=111&username=haimtran

expected response as the response mapping template

{
  "id": "",
  "message": ""
}

Reference#