Terraform, AWS & Sourcing Secrets

A _coupled_ pattern for sourcing provider secrets from AWS SecretsManager

Table Of Contents

Today I Explained

It isn’t always the case that you are solely working with the Terraform AWS provider, and in the cases in which you are working with providers for external services, you don’t necessarily have access to the same authentication pattern as IAM. Often you connect to these external services using a URL and a token credential. For working with these kind of terraform modules, the token credential isn’t going to be readily available on the workstation. Sometimes a helper may exists within /usr/secrets/bin to dynamically pull the secret from an external provider, but this isn’t always the case.

This is why for some Terraform stacks you’ll see the responsibility of retrieving the token credential be pushed onto the Terraform AWS provider. Using a secret store like AWS SecretsManager, the provider can retrieve the token credential using the combination of aws_secretsmanager_secret and aws_secretsmanager_secret_version:

provider "aws" {
  # ...
}

data "aws_secretsmanager_secret" "saas" {
  name = "saas-token-credential"
}

data "aws_secretsmanager_secret_version" "saas" {
  secret_id = data.aws_secretsmanager_secret.saas.id
}

provider "saas" {
    fqdn  = jsondecode(data.aws_secretsmanager_secret_version.saas.secret_string)["fqdn"]
    token = jsondecode(data.aws_secretsmanager_secret_version.saas.secret_string)["token"]
}

This pattern can work for cases where you have a Terraform executor with limited capabilities when it comes to parameter management.

A note on coupling

One of the biggest frustrations with this approach is that the module is now tightly coupled to existing infrastructure outside of the Terraform module. This cannot easily be deployed within a different AWS Account if a token is available through other means. There are improvements to see the design that can be made to assist with these frustrations, such as supporting weak references to the secrets (to avoid explicit reserved names), allow overriding the target secret, allow overriding the token credential. However ultimately these are just creating two workflows, one for workstation development, and another for the deployment scenario.

As a way of working around a limited capability Terraform deployer, this is viable.