AWS Custom Resources

We love to use AWS CloudFormation to deploy our environments. It’s like configuration management for our AWS infrastructure in the sense that we write a desired state as code and apply it to our environment. But sometimes, there are tasks that we want to complete that aren’t part of CloudFormation. For instance, what if we wanted to use CloudFormation to deploy a new account which needs to be done through the CLI, or if we need to return some information to our CloudFormation template before deploying it? Luckily for us, we can use a Custom Resource to achieve our goals. This post shows how you can use CloudFormation with a Custom Resource to execute a very basic Lambda function as part of a deployment.

Solution Overview

To demonstrate our Custom Resource, we’ll need a Lambda function that we can call. CloudFormation will deploy this function from a Zip file and after deployed, will execute this function, and return the outputs to our CloudFormation template. The diagram below demonstrates the process of retrieving this zip file form an existing S3 bucket, deploying it, executing it, and having the Lambda function return data to CloudFormation.

The CloudFormation Template

First, let’s take a look at the CloudFormation template that we’ll be using to deploy our resources.

In the parameters section you can see we’re looking for the S3 bucket with our module in it, the name of the module and a generic input for the CloudFormation template to pass to Lambda as a string.

In the resources section, we have a HelloWorld object which is our custom resource of type Custom::DESCRIPTIONHERE. We need to pass a ServiceToken along, which tells the stack which Custom Resource to be executed. We’re also adding an input which will be passed to Lambda named “Input1” and we’ll reference the parameter seen earlier.

Below this, is the Lambda function deployment. This piece of the resources section of the template shows where the Lambda module comes from, the runtime, timeout, and which role will have permissions be used for it.

Next, there is a section for setting up permissions for the Lambda function to write to CloudWatch. Depending on your environment, you may need to provide access to other resources. For example if your function reads EC2 data, then you’d need to ensure it had the appropriate permissions to read those properties.

We won’t look at the next piece until our Lambda function finishes running, but we’re going to get some return information from the function and print it as a CloudFormation output.

Lambda Function

Let’s look at the Lambda Function we’ll be using for this example. This is a python 2.7 function that takes a basic string input from CloudFormation, concatenates it with another string and returns it. Nothing too crazy here for the example, but there are some important pieces that must be in your Lambda function so that CloudFormation knows that the function is done running, and if it executed correctly.

Let’s look at the main function that will be executed. First, we’ll initialize some of our variables. Next, we want to retrieve our input parameter (named Input1 and passed from CloudFormation), and then log it. After this, there is a simple operation to concatenate the input with another string just to do something simple in our function. The next step is to provide some return data that will end up being our CloudFormation output. This is a JSON object so if you don’t need to return any custom info to CloudFormation, use an empty JSON object {}.

Below this, we’re calling a respond function (also located in our Lambda script which will retrieve info to send back to CloudFormation about the state of the Lambda script run. After this we return our data from the function.

Let’s look closer at the respond function. When we’re executing this function we need to send certain items back to CloudFormation so that the stack knows if it worked or not. Specifically, it must return a response of SUCCESS or FAILED to a pre-signed URL. There are a list of response objects, specifically:

  • Status (Required)
  • Reason (Required if FAILED)
  • PhysicalResourceId (Required)
  • StackId (Required)
  • RequestId (Required)
  • LogicalResourceId (Required)
  • NoEcho
  • Data

The first part of this function builds the JSON object so that we can send it back to CloudFormation. We are then converting it to a string and logging the data for later reference. We set our responseURL which is passed to us from CloudFormation in the event parameter. After that, we set the headers and then we try to do a PUT REST call with our return data. To do all of this, we had to import certain modules for our function which are seen in the full script, but none of these need to be provided in your zip file. It should be noted that this can also be done if you add your Lambda function in-line within your CFn template. If you use that method, there is a “cfn-response” module that can be called which eliminates the need to use the requests module.

See It In Action

Just so we can show some screenshots of the process, here is the input for my CloudFormation template as I’m deploying it through the AWS Console. See that I’ve got an input message of “Test Message” and I’m specifying information about my Lambda Function’s location.

 

You can also see my Lambda function neatly zipped up in my S3 bucket below.

Once the Lambda function has been deployed, we can see it in the Lambda Functions console.

If we look at the CloudWatch Logs for our function, we can see the data being returned to CloudFormation.

Lastly, we can see that in the CloudFormation template that we deployed, it has finished the creation and in the outputs tab, we can see the message that was returned to the stack. This output could be used for other stacks or purely informational.

Summary

Custom Resources might not be necessary very often, but they can let you do virtually anything you want within AWS. Maybe they execute a Lambda function to gather data, or maybe they trigger a Step Function that has tons of logic built into it to do something else magical. The world is your oyster now, so what will you build with your new knowledge?

(This blog post originally appeared on The IT Hollow.)


To stay up-to-date on the latest in tech news and events, follow AHEAD on Twitter and LinkedIn or subscribe to our blog newsletter above.



mm
Author: Eric Shanks
Eric is a top virtualization blogger who designs and implements a wide variety of software and hardware solutions relating to enterprise infrastructure.

Leave a Reply