Auto-approve CodePipeline with Lambda

CodePipeline by AWS is a great tool for setting up Continuous Delivery (or Deployment). It can, as an example, be used to automatically deploy your CloudFormation stacks to different environments on every push to your VCS. Chaining multiple stacks in a CodePipeline stage is easy and allows you to have one central view where you can see the status of your current release as it progresses through different stages.

For your most critical stacks in production you might require a manual approval step where someone or something decides if a release should be rolled out to the public. These approvals can, however, quickly become tedious and slow you down if you have a lot of them. It can be especially annoying if you have to approve a stack that was left unchanged by the current release (i.e. the resulting change set is empty). Lets add a bit of automation to these approval actions to automatically approve a stack if the change set is empty.

Use case

Lets say that our service consists of four separate CloudFormation stacks that we want to deploy in the same CodePipeline. We have the following stacks that will get deployed in the order listed:

1. Network stack
2. Database stackx
3. Buckets and SQS stack
4. Application stack

We will trigger this CodePipeline on every push to master in our VCS, pushes that most often only updates the application stack. Accidental changes to our network or database stack could have dire consequences for our service so to add an extra layer of security we decide to require a manual approval step for these stacks.

The problem that arises now is that every time the pipeline runs it will get stuck at the approval actions for both stacks, even if we are only pushing out changes to the application stack. It would be amazing to be able to skip the approval step if the resulting CloudFormation Change Set for the critical stacks are empty.

Sounds interesting? Read on!

Architecture

So how are we going to do this? Yeah, you guessed it… Lambda to the rescue. A CodePipeline manual approval action can be configured to publish a message to a SNS topic when it is triggered (1). We will subscribe a Lambda to this topic (2), a Lambda that will query the CodePipeline API for the status of the action that creates the CloudFormation Change Set (3). If the Change Set was created without any changes, the Lambda function will automatically approve the manual approval action (4).

Pipeline Structure

This guide assumes you have a pipeline with a stage that resembles the one seen in the architecture diagram above. It should be fairly simple to set up but if you need guidance, please scroll down to the Sample repository section. You will also need to create a SNS topic that will be used for approval notifications.

Set the following configuration on your approval action:

• SNS Topic ARN: YOUR_SNS_TOPIC_ARN
• Comments: ActionToCheck=NAME_OF_YOUR_CREATE_CHANGE_SET_ACTION

If you click the small information button on the approval action you should see the following configuration (but with your values):

CodePipeline API

To understand how we will decide in the Lambda function whether or not to approve the action we need to look at the CodePipeline API. This is easiest done with the AWS CLI:

1
$ aws codepipeline get-pipeline-state --name TestPipeline
2
{
3
    "pipelineName": "TestPipeline",
4
    "pipelineVersion": 1,
5
    "stageStates": [
6
        {
7
            "stageName": "Source",
8
            "inboundTransitionState": {
9
                "enabled": true
10
            },
11
            "actionStates": [
12
                {
13
                    "actionName": "Source",
14
                    "currentRevision": {
15
                        "revisionId": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
16
                    },
17
                    "latestExecution": {
18
                        "status": "Succeeded",
19
                        "summary": "Amazon S3 version id: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
20
                        "lastStatusChange": 1567104982.635,
21
                        "externalExecutionId": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
22
                    },
23
                    "entityUrl": "https://console.aws.amazon.com/s3/home?#"
24
                }
25
            ],
26
            "latestExecution": {
27
                "pipelineExecutionId": "xxxxxxxx-1111-2222-3333-xxxxxxxxxxxx",
28
                "status": "Succeeded"
29
            }
30
        },
31
        {
32
            "stageName": "Deploy",
33
            "inboundTransitionState": {
34
                "enabled": true
35
            },
36
            "actionStates": [
37
                {
38
                    "actionName": "CreateChangeSetStack1",
39
                    "latestExecution": {
40
                        "status": "Succeeded",
41
                        "summary": "Change set pipeline-test-stack-1-changeset was created.",
42
                        "lastStatusChange": 1567105025.115,
43
                        "externalExecutionId": "...",
44
                        "externalExecutionUrl": "..."
45
                    },
46
                    "entityUrl": "https://eu-west-1.console.aws.amazon.com/cloudformation/home?/"
47
                },
48
                {
49
                    "actionName": "ApprovalStack1",
50
                    "latestExecution": {
51
                        "status": "InProgress",
52
                        "token": "12345678-abcd-1234-fghi-123456789abc"
53
                    }
54
                },
55
                {
56
                    "actionName": "ExecuteChangeSetStack1",
57
                    "entityUrl": "https://eu-west-1.console.aws.amazon.com/cloudformation/home?/"
58
                }
59
            ],
60
            "latestExecution": {
61
                "pipelineExecutionId": "xxxxxxxx-1111-2222-3333-xxxxxxxxxxxx",
62
                "status": "InProgress"
63
            }
64
        }
65
    ],
66
    "created": 1567103289.407,
67
    "updated": 1567103289.407
68
}

From this we can see that the pipeline is stuck at the action ApprovalStack1 due to the "status": "InProgress". From the API we also get the approval token that can be used to approve the action through the CodePipeline PutApprovalResult API.

We can try approving the action manually through the CLI:

1
$ aws codepipeline put-approval-result \
2
  --pipeline-name TestPipeline \
3
  --stage-name Deploy \
4
  --action-name ApprovalStack1 \
5
  --result "summary=Looks good,status=Approved" \
6
  --token 12345678-abcd-1234-fghi-123456789abc
7
{
8
    "approvedAt": 1567105541.143
9
}

If you go to the AWS console you should now see that the pipeline has moved past the approval action and started executing the change set. If you check the details of the approval action you should see the following:

Lets get back to the GetPipelineState response above. If we look at the status and summary of the CreateChangeSetStack1 action we can see that we have:

1
"status": "Succeeded",
2
"summary": "Change set pipeline-test-stack-1-changeset was created.",

If the change set did not include any changes the response would instead be:

1
"status": "Succeeded",
2
"summary": "Change set pipeline-test-stack-1-changeset was created with no changes.",

Notice the added with no changes in the end? That small addition to the summary is what we will utilize in our Lambda function to be able to determine whether to automatically approve the action or not.

Lambda

We need to create a Lambda function that subscribes to our SNS Topic that the approval actions sends a notification to. Whenever this notification reaches our function we will get the following data in the Sns.Message event field (your pipeline, stage and action names might of course differ from mine):

1
{
2
  "region": "eu-west-1",
3
  "consoleLink": "https://console.aws.amazon.com/codepipeline/home?region=eu-west-1#/view/TestPipeline",
4
  "approval": {
5
    "pipelineName": "TestPipeline",
6
    "stageName": "Deploy",
7
    "actionName": "ApprovalStack1",
8
    "token": "12345678-abcd-1234-fghi-123456789abc",
9
    "expires": "2019-08-29T08:48Z",
10
    "externalEntityLink": null,
11
    "approvalReviewLink": "...",
12
    "customData": "ActionToCheck=CreateChangeSetStack1"
13
  }
14
}

We get all the information we need from the event to query the CodePipeline GetPipelineState API for information about the stage. We know that we want to look at the status and summary of the CreateChangeSetStack action in the Deploy stage in the TestPipeline pipeline. We also get the approval token as well as the name of the approval action to approve in the event.

For the Pythonistas out there this is easily achievable with boto3:

1
client = boto3.client('codepipeline')
2
response = client.get_pipeline_state(name=pipeline_name)
3

4
approve = _json_parsing_magic(response)
5

6
if approve:
7
  client.put_approval_result(
8
    pipelineName=pipeline_name,
9
    stageName=stage_name,
10
    actionName=action_name,
11
    result={
12
        'summary': 'Automatically approved by Lambda.',
13
        'status': 'Approved'
14
    },
15
    token=token
16
  )

When an approval step is reached after a change set action that had no changes it should be automatically approved by the Lambda function and you should be able to see something like the following in the CodePipeline console:

Sample repository

I’ve created a sample repository that you can refer to. It includes a CloudFormation template that sets up a CodePipeline and a few other required resources such as Buckets and a SNS Topic. The repository also contains code for a Lambda function, which can be deployed with SAM, that queries the CodePipeline API as outlined above to automatically approve manual approval actions.

Conclusion

Now we have a pipeline that can deploy all the stacks that make up our service to get a central view of the entire release process. The pipeline assures us that changes to critical stacks doesn’t get deployed unknowlingly without becoming slow and tedious, if there are no changes to be made to a stack the pipeline automatically continues to the next one. As always, the building blocks of AWS integrates seamlessly and lets you build whatever integrations you can imagine.

I hope you found this post interesting and that you maybe even learned a thing or two. Until next time!