Securing GitHub Tokens in a Serverless CodePipeline

EDIT: If you’re here for the solution, skip straight to the “Solving the Problem with Secrets Manager” section!

Using an Infrastruture-as-Code approach for a Serverless application is a given. The Serverless Framework gives you the tooling to define you AWS resources using CloudFormation definitions, giving you benefits like stacks, versioning and rollback support.

When it comes to implementing a continuous build, integration and deployment pipeline for Serverless, there are quite a few options. We generally go with an AWS-native approach for these things. Since we first started adopting CodeBuild and CodePipeline, the services have matured quite a bit. Critically, their integration with other AWS services means we can use the same tooling and approach for managing our DevOps as we do for our the applications we build. This means our CI/CD pipelines are also defined as code using CloudFormation or serverless.yml files.

One thing to get right from the start is the question of how to secure the secrets required to execute your build. The example used here is a GitHub OAuth token required to fulfill the Source stage of a CodePipeline. The goals are as follows.

1. Never store tokens in source control
2. Never store secrets anywhere in plaintext
3. Don’t pass secrets in plaintext over the network
4. Don’t allow secrets to appear in logs

The Systems Manager Parameters Store Attempt

I first reached for AWS Systems Manager Parameters Store. It allows you to store parameters, encrypted with a KMS key. It is also supported in AWS’ CloudFormation templates using Dynamic References.

Storing a Secure String with Systems Manager Parameter Store

This can then be referenced in a serverless.yml or CloudFormation template with this syntax: {{resolve:ssm-secure:GitHubPersonalAccessToken:1}}. I tried this with the OAuth Token for a GitHub source in CodePipeline:

Doomed Attempt to use Parameter Store Secure Strings in CodePipeline

The CloudFormation validation step run by the Serverless framework immediately threw back this error.

SSM Secure reference is not supported in: [AWS::CodePipeline::Pipeline/Properties/Stages]

It turns out that Parameter Store secure strings are only supported for specific resources such as IAM Users and RDS configurations.

Solving the Problem with Secrets Manager

AWS Secrets Manager is a new service from AWS. At first glance, it seems similar to the secret strings in Parameter Store. The major feature difference is that it supports key rotation. It is also supported in CloudFormation dynamic variables with the small exception for custom resources. It’s also worth noting that the structure is slightly different. With Secrets Manager, you define a secret as a set of key-value pairs which are stored and can be retrieved as JSON.

The secret can be stored initially using the AWS CLI, API or Console.

Storing a GitHub Token in Secrets Manager

We then use the secretsmanager format for our dynamic reference in our CloudFormation CodePipeline resource:

Serverless YML/CloudFormation Syntax for Secrets Manager Variables

This method ensures that the secret never has to be stored with our code, is encrypted at rest using a KMS key, never appears in CloudFormation logs and is used in the CodePipeline without ever being exposed.

Together with my fourTheorem colleagues, we’ll soon be publishing a complete production-ready serverless project with all of this included. UPDATE: This is now available on GitHub:

fourTheorem/slic-starter

Stay tuned, clap and follow for updates!

References

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/dynamic-references.html