AWS CodeBuild CICD - Deploy to Lambda
Posted by Kyle Pericak in cloud
This post is linked to from the AWS: Deep Dive Project
This post is a follow-up to my authenticated API Gateway post.
I've written a Lambda function and presented it using API Gateway. The API uses Amazon's AIM keys for authentication. The next step is to write the code to source control (AWS CodeBuild Git) and have it automatically deploy.
Note: This official guide on AWS is really good, and I copied big parts it. I use Python instead of JavaScript in this post, and include my own experience while following the guide.
Create an S3 Bucket
Open the S3 Console and create a bucket to store your build artifacts.
Commit your function to source control
I'm using a dedicated repository on AWS CodeCommit to store the code, and Python3 for the language. Here's a very simple Lambda function I use for testing:
def lambda_handler(event, context):
""" run a hello function """
body = 'NO HTTP METHOD'
if 'httpMethod' in event and event['httpMethod'] == 'POST':
request_body = event['body']
request_user = event['requestContext']['identity']['userArn']
body = f'{request_user} sent a POST with body: {request_body}'
elif 'httpMethod' in event and event['httpMethod'] == 'GET':
request_user = event['requestContext']['identity']['userArn']
body = f'{request_user} sent a GET'
return {
'statusCode': 200,
'body': json.dumps(body)
}
Define an AWS SAM template
Write the template file
This was my first time using AWS SAM, so consider these my observations and not necessarily fact.
SAM stands for Serverless Application Model.
The fields I've changed from Amazon's example for this post:
- Description - string describing the serverless application. This description will be attached to the CloudFormation stack.
- Resources.[resourceName] - self-defined resource, my function.
The resource is a Lambda function because it's type is
AWS::Serverless::Function
- Handler - Which method will run in the function
- Runtime - The language to be used
- CodeUri - Path to my file defining the function
- Resource.[resourceName].Events[eventName] - Name of the event which triggers the function.
- Resource.[resourceName].Events[eventName].path - path for the URL
# template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Sample Continuously deploy function
Resources:
TestCicdFunction:
Type: AWS::Serverless::Function
Properties:
Handler: cicd-test.lambda_handler
Runtime: python3.8
CodeUri: ./cicd-test.py
Events:
TestCicdAPI:
Type: Api
Properties:
Path: /Test
Method: GET
Test the template
Install the AWS SAM CLI application.
pip install --user aws-sam-cli
Build the package and upload it to your S3 bucket. From your project root, set your bucket name then run the following.
# Set the bucket name first
BUCKET=''
aws cloudformation package \
--template-file template.yml \
--s3-bucket $BUCKET \
--output-template-file outputtemplate.yml
The package command will upload your python file to S3 and output a new
template file with the S3 path replacing the CodeUri
.
Uploading to <ID> 381 / 381.0 (100.00%)
Successfully packaged artifacts and wrote output template to file outputtemplate.yml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /home/vagrant/arcus-lambda/outputtemplate.yml --stack-name <YOUR STACK NAME>
Now deploy it. Since there's no CloudFormation stack created yet, I make a new
one named test-cicd-lambda
. Note that in the above command output, the
example they give for deploy is wrong. You need to include the
--capabilities CAPABILITY_IAM
argument for this template too.
aws cloudformation deploy \
--capabilities CAPABILITY_IAM \
--template-file /home/vagrant/arcus-lambda/outputtemplate.yml \
--stack-name test-cicd-lambda
The application is now deployed.
- Stack is visible in the CloudFormation console
- API is created in the API Gateway console
- Function is created in the Lambda console. The function also has an Application page in the web console now.
Create a CodeBuild Project
I recently did a whole post on this here, so I'm going to be light on the details. Head over to that post to fill in any blanks. In that post I was building Docker images and pushing them to ECR. This is similar, except we push python files to Lambda instead.
Write the buildspec.yml
Put this in the project root, it defines what CodeBuild will do. We'll use an
environment variable for the BUCKET
definition.
If you're not sure what to pick for a runtime, check this list.
# buildspec.yml
version: 0.2
phases:
install:
runtime-versions:
python: 3.8
build:
commands:
- aws cloudformation package --template-file template.yml --s3-bucket $BUCKET --output-template-file outputtemplate.yml
artifacts:
type: zip
files:
- template.yml
- outputtemplate.yml
Commit the file to source control.
Create a role for CodeBuild
I use the role I defined for CodeBuild in my previous post. It's a bit overly permissive but it works. Other than the policies which get created automatically, mine has these:
AWSCodeCommitFullAccess
AmazonEC2ContainerRegistryFullAccess
AmazonS3FullAccess
CloudWatchLogsFullAccess
AWSCodeBuildAdminAccess
Define the build project
- Open the CodeBuild console
- Create build project
- Name the build and give it a description
- Source: Add your repository
- Environment
- Managed Image
- Operating system: Ubuntu
- Runtimes: Standard
- Image: the newest one
- Environment type: Linux
- Privileged: False
- Service Role: Existing role - choose the one you made above
- Additional Configuration - environment variables
- name: BUCKET
- value: Your bucket name
- Buildspec: use buildspec file
- Artifacts: no artifacts
- Logs: set a group name for CloudWatch
- Create build project
Test the build before moving on and make sure it works.
Create a Pipeline
Create IAM Policy for CodeDeploy
Open the IAM console and create a new policy. Define the policy using the following JSON. These wildcards will cover everything you need, though it might be a bit permissive.
{
"Statement": [
{
"Action": [
"apigateway:*",
"codedeploy:*",
"lambda:*",
"cloudformation:CreateChangeSet",
"iam:GetRole",
"iam:CreateRole",
"iam:DeleteRole",
"iam:PutRolePolicy",
"iam:AttachRolePolicy",
"iam:DeleteRolePolicy",
"iam:DetachRolePolicy",
"iam:PassRole",
"s3:GetObject",
"s3:GetObjectVersion",
"s3:GetBucketVersioning"
],
"Resource": "*",
"Effect": "Allow"
}
],
"Version": "2012-10-17"
}
Create IAM Role for CodeDeploy
In the role wizard of the IAM page:
- Create role
- Trusted Entity:
CloudFormation
- Permissions:
AWSLambdaExecute
- Your new policy created above
- Role Name: assign a name
Create the Pipeline
When code is committed, you want CodeBuild to run and re-deploy your serverless application.
- Open the CodePipeline console
- Click Create Pipeline
- Name the pipeline
- Service Role: Create a new service role for the pipeline. AWS will set sane permissions for it, no special role is needed. I like to have a consistent naming convention as you end up with a ton of IAM entities after a while.
- Role Name: Pick a name
- Allow AWS CodePipeline to create a role
- Next
- Source Provider: AWS CodeCommit
- Repository Name: Select the repository name
- Branch name: master
- Change detection: Amazon CloudWatch Events
- Next
- Build Provider: AWS CodeBuild
- Project Name: Choose the project you just made
- Next
- Deploy Provider: AWS CloudFormation
- Action Mode: Create or replace a change set
- Stack Name: choose a name. I used the one from the test above.
- Change Set Name: choose a name
- Template:
- Artifact name:
BuildArtifact
- File name:
outputtemplate.yml
- Artifact name:
- Capabilities:
CAPABILITY_IAM
- Role Name: The role you defined for CodeDeploy above
- Next
- Create Pipeline
That should do it. Now whenever you push code to your master branch, the serverless application will re-deploy.