Treat Lambda Functions as Build Artifacts

Published on Sat, Jul 2, 2022 by Étienne Brouillard

The Serverless Application Model (AWS SAM) is a framework that speeds up the development of serverless applications on AWS. AWS SAM covers a lot of ground from Infrastructure, Application development to Deployment with focus on Lambda Functions. To do all this, AWS SAM uses AWS services to build and deploy Lambda functions. Because there is no magic and understanding the way it works behind the scenes is important, this article explains how to deploy Lambda Functions with CodeDeploy.

TL;DR

Because Lambda Functions have versions and aliases, they should be considered as build products and not deployment targets. One simple way provided by AWS to do so is CodeDeploy. CodeDeploy is a managed deployment service which supports Lambda Functions and ECS (among many others). It needs AppSpec files to execute its tasks. AppSpec files describe the resources which CodeDeploy should update. In the case of Lambda Function, it proceeds to safe and repeatable deployment leveraging Aliases and Versions. For even greater simplicity, you should use CodeBuild and CodeDeploy orchestrated by CodePipeline. This article focuses on explaining function versions and aliases and how to make successful deployments using AppSpec files.

An Erroneous Approach

A Lambda Function at its simplest defintion is a serverless compute platform. As such, it might then be tempting to consider a Lambda Function as a simple component which only runs built code, be it an archive zip file or a container image:

Since most of the time a Lambda Function depends on other AWS resources such as SQS queues or DynamoDB tables, these dependencies usually are provisionned at the same time. This results in this general and simplistic (although common) process:

Build and package code,
Publish build artifacts (archive and/or container image),
Provision or change the infrastructure.

Steps 1 and 2 are usually done in the CI part of the CI/CD pipeline while step 3 ends up in the CD part. This process works. In one project we deploy regularly over two dozens of Lambda Functions this way and we're still fine. However we're not leveraging two key features of Lambda Function and we're missing on all their benefits.

Lambda functions versions and aliases

Lambda Functions support versioning and aliases. Lambda Function can have multiple versions and aliases are named pointers to a specific version. Versions and aliases can be used to ensure safe and reliable deployment of Lambda Functions.

Versions

Lambda Function versionning locks the settings the code and the dependencies of an unpublished version of a function. An unpublished version has an unqualified ARN (ex: arn:aws:lambda:aws-region:acct-id:function:helloworld) and has the $LATEST tag.

Publishing a version is done through the following AWS CLI command:

aws lambda publish-version --function-name my-function

A published version includes:

The code and its dependencies (layers)
The Lambda runtime
All the function settings and environment variables
A qualified (unique) ARN (ex: arn:aws:lambda:aws-region:acct-id:function:helloworld:42)

Only a function with changes to its code, dependencies or settings can be published. New versions of a Lambda Function always stem from the unpublished ($LATEST) version and as such, the code can't be changed.

A function that has been published can however be modified to some extent. A published Lambda Function can have different triggers and destinations than the unpublished version.

In a CI/CD using the publishing mechanism would result in this kind of workflow:

Build code and dependencies,
Update unpublished function's code, dependencies and settings,
Publish new version.

Aliases

Aliases are logical pointers to Lambda Function versions. A Lambda Function with an alias has a qualified ARN (ex: arn:aws:lambda:aws-region:acct-id:function:helloworld:production). An alias can only refer to a specifc function version or $LATEST and not another alias. An alias has similar characteristics than a published version. The code and dependencies can't be changed but the triggers and destinations can.

And because aliases support routing configurations, it's possible to send the traffic to up to two versions allowing for blue/green deployment. However there are some caveats:

Both versions must have the same execution role,
Both versions must have the same dead letter queue configuration (or no DLQ),
Both versions must be published (no $LATEST).

Creating an alias is done through the create-alias command:

aws lambda create-alias --name routing-alias --function-name my-function --function-version 1

Updating an alias can be done through the update-alias action. The example below describes how to update the routing config.

aws lambda update-alias --name routing-alias --function-name my-function  --routing-config AdditionalVersionWeights={"2"=0.05}

When invoked, CloudWatch logs will always carry the invoked version in the START event log entry. Also upon synchronous invocation the response header x-amz-executed-version will carry the executed version.

So Treat Lambda Functions as Artifacts

Understanding versions and aliases, the initial pipeline design is flawed: it only updates the unpublished version thus missing out on the added benefits provided by the two features.

A Better Pipeline

A Lambda Function is a product of the build process rather than a deployment item. The first pipeline described above is all about configuration and it's impossible to use versions and aliases. To address this, here are new proposed build steps:

Build the function code package or container image.
Publish the code and layers to an S3 Bucket or the container image in an ECR repository,
Update the target Lambda Function code, dependencies and settings. This will only affect the unpublished version, leaving the published versions unchanged.
Publish a new version of the Lambda Function.
Generate AppSpec file.
Publish build artifacts.

Building artifacts for Amazon CodeDeploy

Amazon CodeDeploy is a managed service which simplifies the deployment of application onto AWS Services such as Lambda Functions and Elastic Container Service. In the buid steps above you may have noticed that there's a step called "Generate AppSpec file". Amazon CodeDeploy uses AppSpec files artifact to deploy Lambda Functions or to be more precise, change aliases version pointers. Let's say there's a "Production" alias then CodeDeploy would do the change to a newer version itself.

Let's begin with this blank JSON file:

{
    "version": 0.0,
    "Resources": [
    ],
    "Hooks": []
}

In the resources array, we need to define Lambda Functions to be deployed using CodeDeploy:

{
    "Type": "AWS::Lambda::Function",
    "Properties": {
        "Name": "funcName",
        "Alias": "funcAlias",
        "CurrentVersion": 0,
        "TargetVersion": 1
    }
}

Here it basically tells CodeDeploy to update the Lambda Function named "funcName" alias "funcAlias" to go from version 0 to version 1. An additionnal section "Hooks" allows running validation testing Lambda Functions:

"Hooks": [
    {
        "BeforeInstall": "LambdaFunctionToValidateBeforeInstall"
    },
    {
        "AfterInstall": "LambdaFunctionToValidateAfterInstall"
    },
    {
        "AfterAllowTestTraffic": "LambdaFunctionToValidateAfterTestTrafficStarts"
    },
    {
        "BeforeAllowTraffic": "LambdaFunctionToValidateBeforeAllowingProductionTraffic"
    },
    {
        "AfterAllowTraffic": "LambdaFunctionToValidateAfterAllowingProductionTraffic"
    }
]

An example

Let's look at this exerpt from one function buildspec.yaml file:

CURRENT_VERSION=$(aws lambda get-alias --function-name $FUNCTIONS_TRANSCRIPTIONMSGS_EVENTHANDLER_NAME --name prod | jq -r ".FunctionVersion")
TARGET_VERSION=$(./publish.sh $FUNCTIONS_TRANSCRIPTIONMSGS_EVENTHANDLER_NAME $AWS_ACCOUNT_ID $AWS_DEFAULT_REGION $IMAGE_REPO_NAME:$CODEBUILD_BUILD_NUMBER)
./makeappspec.js _appspec.json $FUNCTIONS_TRANSCRIPTIONMSGS_EVENTHANDLER_NAME prod $CURRENT_VERSION $TARGET_VERSION > appspec.json

The first step is to get the target alias current version (since we'll have to convey this info in the final AppSpec file.):

CURRENT_VERSION=$(aws lambda get-alias --function-name $FUNCTIONS_TRANSCRIPTIONMSGS_EVENTHANDLER_NAME --name prod | jq -r ".FunctionVersion")

The second step publishes updates to the code:

TARGET_VERSION=$(./publish.sh $FUNCTIONS_TRANSCRIPTIONMSGS_EVENTHANDLER_NAME $AWS_ACCOUNT_ID $AWS_DEFAULT_REGION $IMAGE_REPO_NAME:$CODEBUILD_BUILD_NUMBER)

And it uses this script:

#!/bin/bash
UPDATE=$(aws lambda update-function-code --function-name $1 --image-uri $2.dkr.ecr.$3.amazonaws.com/$4 | jq)
STATE=$(aws lambda get-function --function-name $1 --query 'Configuration.[LastUpdateStatus]' | jq -r '.[0]')
while [ $STATE == "InProgress" ]
do 
    STATE=$(aws lambda get-function --function-name $1 --query 'Configuration.[LastUpdateStatus]' | jq -r '.[0]') 
    sleep 3; 
done;
echo $(aws lambda publish-version --function-name $1 | jq -r ".Version")

Once the Lambda Function is updated and published, the final step is to generate an AppSpec file:

#! /usr/bin/env node
const data = require("./"+process.argv[2]); 
const funcName = process.argv[3];
const funcAlias = process.argv[4];
const currentVersion = process.argv[5];
const targetVersion = process.argv[6];

const func = {};
func[funcName] = {
    Type: "AWS::Lambda::Function",
    Properties: {
        Name: funcName,
        Alias: funcAlias,
        CurrentVersion: currentVersion,
        TargetVersion: targetVersion
    }
};
data.Resources.push(func);
console.log(JSON.stringify(data));

Although this example is for one single Lambda Function, by combining all your application's functions within one AppSpec file, it's possible to do testing and in case of failure, rollback the entire set of function's configurations thus bringing everything back to the previous "version".

Some tips

Consider using container images instead of code archives and layers.
Build one single image for your application's functions.
Generate one AppSpec for your application's functions.