<p>This post is a follow-up to my <a href="/aws-api-gateway-iam.html">authenticated API Gateway post</a>.</p>
<p>I've written a Lambda function and presented it using API Gateway. The API
uses Amazon's AIM keys for authentication. The next step is to write the code
to source control (AWS CodeBuild Git) and have it automatically deploy.</p>
<p><strong>Note</strong>: <a href="https://docs.aws.amazon.com/lambda/latest/dg/build-pipeline.html">This official guide on AWS</a>
is really good, and I copied big parts it. I use Python instead of
JavaScript in this post, and include my own experience while following the
guide.</p>
<hr>
<h3>Table of Contents</h3>
<ul>
<li><a href="#create-an-s3-bucket">Create an S3 Bucket</a></li>
<li><a href="#commit-your-function-to-source-control">Commit your function to source control</a></li>
<li><a href="#define-an-aws-sam-template">Define an AWS SAM template</a>
<ul>
<li><a href="#write-the-template-file">Write the template file</a></li>
<li><a href="#test-the-template">Test the template</a></li>
</ul>
</li>
<li><a href="#create-a-codebuild-project">Create a CodeBuild Project</a>
<ul>
<li><a href="#write-the-buildspecyml">Write the buildspec.yml</a></li>
<li><a href="#create-a-role-for-codebuild">Create a role for CodeBuild</a></li>
<li><a href="#define-the-build-project">Define the build project</a></li>
</ul>
</li>
<li><a href="#create-a-pipeline">Create a Pipeline</a>
<ul>
<li><a href="#create-iam-policy-for-codedeploy">Create IAM Policy for CodeDeploy</a></li>
<li><a href="#create-iam-role-for-codedeploy">Create IAM Role for CodeDeploy</a></li>
<li><a href="#create-the-pipeline">Create the Pipeline</a></li>
</ul>
</li>
</ul>
<h1>Create an S3 Bucket</h1>
<p>Open the <a href="https://s3.console.aws.amazon.com/s3/">S3 Console</a> and create a
bucket to store your build artifacts.</p>
<hr>
<hr>
<h1>Commit your function to source control</h1>
<p>I'm using a dedicated repository on AWS CodeCommit to store the code, and
Python3 for the language. Here's a very simple Lambda function I use for
testing:</p>
<pre><code class="language-python">def lambda_handler(event, context):
    """ run a hello function """
    body = 'NO HTTP METHOD'
    if 'httpMethod' in event and event['httpMethod'] == 'POST':
        request_body = event['body']
        request_user = event['requestContext']['identity']['userArn']
        body = f'{request_user} sent a POST with body: {request_body}'
    elif 'httpMethod' in event and event['httpMethod'] == 'GET':
        request_user = event['requestContext']['identity']['userArn']
        body = f'{request_user} sent a GET'
    return {
        'statusCode': 200,
        'body': json.dumps(body)
    }
</code></pre>
<hr>
<h1>Define an AWS SAM template</h1>
<h2>Write the template file</h2>
<p>This was my first time using AWS SAM, so consider these my observations and not
necessarily fact.</p>
<p>SAM stands for Serverless Application Model.</p>
<ul>
<li><a href="https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-resource-function.html">SAM Reference Page</a></li>
<li><a href="https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-property-function-api.html">API Event type</a></li>
<li><a href="https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html">What is SAM</a></li>
<li><a href="https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-specification.html">SAM Specification</a></li>
</ul>
<p>The fields I've changed from Amazon's example for this post:</p>
<ul>
<li><strong>Description</strong> - string describing the serverless application. This
description will be attached to the CloudFormation stack.</li>
<li><strong>Resources.[resourceName]</strong> - self-defined resource, my function.
The resource is a Lambda function because it's type is
<code>AWS::Serverless::Function</code></li>
<li><strong>Handler</strong> - Which method will run in the function</li>
<li><strong>Runtime</strong> - The language to be used</li>
<li><strong>CodeUri</strong> - Path to my file defining the function</li>
<li><strong>Resource.[resourceName].Events[eventName]</strong> - Name of the event which
triggers the function.</li>
<li><strong>Resource.[resourceName].Events[eventName].path</strong> - path for the URL</li>
</ul>
<pre><code class="language-yaml"># template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Sample Continuously deploy function
Resources:
  TestCicdFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: cicd-test.lambda_handler
      Runtime: python3.8
      CodeUri: ./cicd-test.py
      Events:
        TestCicdAPI:
          Type: Api
          Properties:
            Path: /Test
            Method: GET
</code></pre>
<h2>Test the template</h2>
<p>Install the AWS SAM CLI application.</p>
<pre><code class="language-bash">pip install --user aws-sam-cli
</code></pre>
<p>Build the package and upload it to your S3 bucket. From your project
root, set your bucket name then run the following.</p>
<pre><code class="language-bash"># Set the bucket name first
BUCKET=''

aws cloudformation package \
  --template-file template.yml \
  --s3-bucket $BUCKET \
  --output-template-file outputtemplate.yml
</code></pre>
<p>The package command will upload your python file to S3 and output a new
template file with the S3 path replacing the <code>CodeUri</code>.</p>
<pre><code class="language-text">Uploading to &#x3C;ID>  381 / 381.0  (100.00%)
Successfully packaged artifacts and wrote output template to file outputtemplate.yml.
Execute the following command to deploy the packaged template
aws cloudformation deploy --template-file /home/vagrant/arcus-lambda/outputtemplate.yml --stack-name &#x3C;YOUR STACK NAME>
</code></pre>
<p>Now deploy it. Since there's no CloudFormation stack created yet, I make a new
one named <code>test-cicd-lambda</code>. Note that in the above command output, the
example they give for deploy is wrong. You need to include the
<code>--capabilities CAPABILITY_IAM</code> argument for this template too.</p>
<pre><code class="language-bash">aws cloudformation deploy \
  --capabilities CAPABILITY_IAM \
  --template-file /home/vagrant/arcus-lambda/outputtemplate.yml \
  --stack-name test-cicd-lambda
</code></pre>
<p>The application is now deployed.</p>
<ul>
<li>Stack is visible in the  <a href="https://ca-central-1.console.aws.amazon.com/cloudformation">CloudFormation console</a></li>
<li>API is created in the <a href="https://ca-central-1.console.aws.amazon.com/apigateway/">API Gateway console</a></li>
<li>Function is created in the <a href="https://ca-central-1.console.aws.amazon.com/lambda/">Lambda console</a>.
The function also has an Application page in the web console now.</li>
</ul>
<hr>
<h1>Create a CodeBuild Project</h1>
<p>I recently did a whole post on this <a href="/aws-codepipeline-ecr.html">here</a>, so I'm
going to be light on the details. Head over to that post to fill in any blanks.
In that post I was building Docker images and pushing them to ECR. This is
similar, except we push python files to Lambda instead.</p>
<h2>Write the buildspec.yml</h2>
<p>Put this in the project root, it defines what CodeBuild will do. We'll use an
environment variable for the <code>BUCKET</code> definition.</p>
<p>If you're not sure what to pick for a runtime, check <a href="https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-available.html">this list</a>.</p>
<pre><code class="language-yaml"># buildspec.yml
version: 0.2
phases:
  install:
    runtime-versions:
        python: 3.8
  build:
    commands:
      - aws cloudformation package --template-file template.yml --s3-bucket $BUCKET --output-template-file outputtemplate.yml
artifacts:
  type: zip
  files:
    - template.yml
    - outputtemplate.yml
</code></pre>
<p>Commit the file to source control.</p>
<h2>Create a role for CodeBuild</h2>
<p>I use the role I defined for CodeBuild in my previous post. It's a bit overly
permissive but it works. Other than the policies which get created
automatically, mine has these:</p>
<ul>
<li><code>AWSCodeCommitFullAccess</code></li>
<li><code>AmazonEC2ContainerRegistryFullAccess</code></li>
<li><code>AmazonS3FullAccess</code></li>
<li><code>CloudWatchLogsFullAccess</code></li>
<li><code>AWSCodeBuildAdminAccess</code></li>
</ul>
<h2>Define the build project</h2>
<ol>
<li>Open the <a href="https://ca-central-1.console.aws.amazon.com/codesuite/codebuild/">CodeBuild console</a></li>
<li>Create build project</li>
<li>Name the build and give it a description</li>
<li>Source: Add your repository</li>
<li>Environment
<ol>
<li>Managed Image</li>
<li>Operating system: Ubuntu</li>
<li>Runtimes: Standard</li>
<li>Image: the newest one</li>
<li>Environment type: Linux</li>
<li>Privileged: False
<ol>
<li>Service Role: Existing role - choose the one you made above</li>
</ol>
</li>
<li>Additional Configuration - environment variables
<ul>
<li>name: BUCKET</li>
<li>value: Your bucket name</li>
</ul>
</li>
</ol>
</li>
<li>Buildspec: use buildspec file</li>
<li>Artifacts: no artifacts</li>
<li>Logs: set a group name for CloudWatch</li>
<li>Create build project</li>
</ol>
<p>Test the build before moving on and make sure it works.</p>
<hr>
<h1>Create a Pipeline</h1>
<h2>Create IAM Policy for CodeDeploy</h2>
<p>Open the IAM console and create a new policy.
Define the policy using the following JSON.
These wildcards will cover everything you need, though it might be a bit
permissive.</p>
<pre><code class="language-json">{
    "Statement": [
        {
            "Action": [
                "apigateway:*",
                "codedeploy:*",
                "lambda:*",
                "cloudformation:CreateChangeSet",
                "iam:GetRole",
                "iam:CreateRole",
                "iam:DeleteRole",
                "iam:PutRolePolicy",
                "iam:AttachRolePolicy",
                "iam:DeleteRolePolicy",
                "iam:DetachRolePolicy",
                "iam:PassRole",
                "s3:GetObject",
                "s3:GetObjectVersion",
                "s3:GetBucketVersioning"
            ],
            "Resource": "*",
            "Effect": "Allow"
        }
    ],
    "Version": "2012-10-17"
}
</code></pre>
<h2>Create IAM Role for CodeDeploy</h2>
<p>In the role wizard of the IAM page:</p>
<ol>
<li>Create role</li>
<li>Trusted Entity: <code>CloudFormation</code></li>
<li>Permissions:
<ol>
<li><code>AWSLambdaExecute</code></li>
<li>Your new policy created above</li>
</ol>
</li>
<li>Role Name: assign a name</li>
</ol>
<h2>Create the Pipeline</h2>
<p>When code is committed, you want CodeBuild to run and re-deploy your serverless
application.</p>
<ol>
<li>Open the <a href="https://ca-central-1.console.aws.amazon.com/codesuite/codepipeline/pipelines">CodePipeline console</a></li>
<li>Click Create Pipeline</li>
<li>Name the pipeline</li>
<li>Service Role: Create a new service role for the pipeline. AWS will set sane
permissions for it, no special role is needed. I like to have a consistent
naming convention as you end up with a ton of IAM entities after a while.</li>
<li>Role Name: Pick a name</li>
<li>Allow AWS CodePipeline to create a role</li>
<li>Next</li>
<li>Source Provider: AWS CodeCommit</li>
<li>Repository Name: Select the repository name</li>
<li>Branch name: master</li>
<li>Change detection: Amazon CloudWatch Events</li>
<li>Next</li>
<li>Build Provider: AWS CodeBuild</li>
<li>Project Name: Choose the project you just made</li>
<li>Next</li>
<li>Deploy Provider: AWS CloudFormation</li>
<li>Action Mode: Create or replace a change set</li>
<li>Stack Name: choose a name. I used the one from the test above.</li>
<li>Change Set Name: choose a name</li>
<li>Template:
<ul>
<li>Artifact name: <code>BuildArtifact</code></li>
<li>File name: <code>outputtemplate.yml</code></li>
</ul>
</li>
<li>Capabilities: <code>CAPABILITY_IAM</code></li>
<li>Role Name: The role you defined for CodeDeploy above</li>
<li>Next</li>
<li>Create Pipeline</li>
</ol>
<p>That should do it. Now whenever you push code to your master branch, the
serverless application will re-deploy.</p>


<p><strong>This post is linked to from the <a href="/aws.html">AWS: Deep Dive Project</a></strong></p>
<hr>
<p>This post covers how to restrict access to a Lambda function using AWS IAM
roles. It builds on the function written <a href="/aws-lambda.html">in my last post</a></p>
<hr>
<h3>Table of Contents</h3>
<ul>
<li><a href="#create-a-policy-for-your-lambda-function">Create a policy for your lambda function</a>
<ul>
<li><a href="#get-the-arn-of-your-function">Get the ARN of your function</a></li>
<li><a href="#create-the-policy">Create the policy</a></li>
</ul>
</li>
<li><a href="#invoke-lambda-from-python">Invoke Lambda from Python</a></li>
</ul>
<h1>Create a policy for your lambda function</h1>
<h2>Get the ARN of your function</h2>
<p>Navigate to your function in the AWS Lambda Console. On the top right, the ARN
is displayed. Copy it, you'll use it in the policy.</p>
<h2>Create the policy</h2>
<p>Open the IAM Console and create a new policy.
The JSON looks like this. Note the ARN copied from the Function.</p>
<pre><code class="language-JSON">{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "LambdaInvokeHelloWorld",
            "Effect": "Allow",
            "Action": [
                "lambda:InvokeFunction",
                "lambda:InvokeAsync"
            ],
            "Resource": "arn:aws:lambda:ca-central-1:850047500507:function:HelloWorld"
        }
    ]
}
</code></pre>
<p>Name the policy and assign it to your user. This user should have a Key ID and
Secret Key.</p>
<hr>
<h1>Invoke Lambda from Python</h1>
<p>Note that the payloads are in byte format.</p>
<pre><code class="language-python">import json
import boto3

key_id = 'XXXXXXXXXXXXXXXXX'
key_secret = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXX'

session = boto3.Session(
    aws_access_key_id=key_id,
    aws_secret_access_key=key_secret)

client = session.client('lambda')

body = {'name': 'Kyle'}
payload = str.encode(json.dumps(body))
response = client.invoke(FunctionName='HelloWorld', Payload=payload)
print(response['Payload'].read())
</code></pre>
<p>If the user identified by that key has the above permission, they'll be able
to execute this function. Otherwise they'll get a nice permissions error.</p>


<p>This post covers the basics of using AWS Lambda functions.</p>
<h3>Table of Contents</h3>
<ul>
<li><a href="#pricing">Pricing</a></li>
<li><a href="#hello-world-lambda-function">Hello World Lambda Function</a>
<ul>
<li><a href="#create-the-function">Create the function</a></li>
<li><a href="#define-a-test-event">Define a test event</a></li>
<li><a href="#write-the-function-code">Write the function code</a></li>
<li><a href="#define-a-trigger">Define a trigger</a>
<ul>
<li><a href="#test-the-api">Test the API</a></li>
</ul>
</li>
<li><a href="#what-are-layers">...What are layers?</a></li>
</ul>
</li>
</ul>
<h1>Pricing</h1>
<p>For the most accurate data, see the <a href="https://aws.amazon.com/lambda/pricing/">official pricing documentation</a>.</p>
<p>Lambda charges you for each request invocation, and also for how long the
request took to run. These prices are probably in USD.</p>
<ul>
<li><strong>Region</strong>: Canada (Central)</li>
<li><strong>Requests</strong>: $0.20 / 1M Requests, or $0.0000002 / Request</li>
<li><strong>Duration (General)</strong>: $0.000016667 for every GB-second</li>
<li><strong>Duration (128MB)</strong>: $0.00000208/sec, or ~$0.0075/hour.
I think if it as 3/4 of a cent per hour.</li>
</ul>
<hr>
<h1>Hello World Lambda Function</h1>
<h2>Create the function</h2>
<ol>
<li>In your browser, open the AWS console <a href="https://console.aws.amazon.com/lambda">Lambda Page</a>.</li>
<li>Click the orange Create Function button</li>
<li>Author from scratch</li>
<li>Function name: HelloWorld</li>
<li>Runtime: Python 3.7</li>
<li>Permissions: Create a new role with basic lambda permissions</li>
<li>click Create Function on the bottom right</li>
</ol>
<h2>Define a test event</h2>
<ol>
<li>In the dropdowns along the top, one is called "Select a test event".</li>
<li>Click it, and choose Configure test events</li>
<li>Event template: Hello World</li>
<li>Event name: <code>HelloWorldTest1</code></li>
<li>Enter some sample data and click Create.</li>
</ol>
<p><strong>Sample Data</strong>:</p>
<pre><code class="language-JSON">{
  "body": "{\"name\": \"Test User\"}"
}
</code></pre>
<h2>Write the function code</h2>
<p>Click on the orange lambda icon with the text "HelloWorld" to open the
"Function code" form. Expand it to full screen (button on top right) so the
Save and Test buttons appear along the top.</p>
<p>Write your Hello World function. This was the shortest one I could come up with
that wouldn't throw 500's if you called it with the wrong data.</p>
<pre><code class="language-python">import json

def lambda_handler(event, context):
    name = 'UNDEFINED'
    if 'body' in event:
        try:
            ebody = json.loads(event['body'])
            name = ebody['name']
        except:
            pass
    greeting = f'Hello {name}!'
    body = json.dumps(greeting)
    return {
        'statusCode': 200,
        'body': body
    }
</code></pre>
<p>Click Save, then Test. You should get a response with body <code>'Hello TestUser'</code>.</p>
<h2>Define a trigger</h2>
<ol>
<li>Click the "+ Add trigger" button.</li>
<li>API Gateway</li>
<li>REST API</li>
<li>Security: Open (in this case)</li>
<li>Additional Settings: Leave the defaults</li>
</ol>
<p>Now if you click on the purple API Gateway button it will open the API Gateway
form. In there, you can get the URL. Copy the link address.</p>
<h3>Test the API</h3>
<p>From bash:</p>
<pre><code class="language-bash">curl \
  -X POST \
  -H 'Content-Type: application/json' \
  -d '{"name": "MyName"}' \
  https://wg8p74asah.execute-api.us-east-1.amazonaws.com/default/HelloWorld
</code></pre>
<p>It should reply "Hello MyName".</p>
<h2>...What are layers?</h2>
<p>Prominently displayed beneath the function's name in the Designer form
is a button to manage Layers.</p>
<p><a href="https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html">Here's their official documentation</a>.</p>
<p>Layers are dependencies like archives and libraries.
Any one function can use up to 5 of them, and they have a 250MB limit.
They land in <code>/opt</code> when extracted.</p>
<p>We don't need layers for this Hello World app.</p>


Kyle Pericak

"It works in my environment"

Categories

Tags