Kubernetes, CloudFormation and OIDC Magic Numbers

Using output-only CloudFormation Stacks to facilitate IAM Roles for Service Accounts in CloudFormation

Today I Explained

Magic numbers in programming refer to numeric (or other value) literals that are used within code without any explanation of its meaning, or providing any guidance on where to discover the value. An example of this comes with Kubernetes & IAM OIDC Identity Providers. These value ARNs look like the following:

arn:aws:iam::012345678901:oidc-provider/oidc.eks.us-east-2.amazonaws.com/id/7185F12D2B62B8DA97B0ECA713F66C86

If you are running a service within Kubernetes that you wish to have communicate with authenticated AWS APIs, you’ll need to make use of IAM Role for Service Accounts (IRSA). This works through an OIDC Provider which is able to permit authentication with an IAM Role. Within Kubernetes, a service account can then be mapped to that IAM Role, using the OIDC Provider to authenticate.

When it is understood, the usage of this OIDC Provider is understandable. Without the context behind it, these values appear as magic numbers, which present a uncertainty about which Kubernetes cluster they might be referencing, and how one would retrieve determine this.

Due to CloudFormation lacking the ability to lookup the OIDC Provider for a Kubernetes cluster by the name of the Kubernetes cluster, it can result in the value being passed as a parameter without the appropriate metadata context.

An alternative approach that can be considered is leveraging the capabilities of output-only CloudFormation stacks to provide details to the Kubernetes cluster for CloudFormation. Useful properties that can be provided are:

OIDC Provider Arn
Additional Security Groups
Kubernetes Version
Kubernetes Metadata

This can look something like the following within the output-only stack in which the stack is named after the Kubernetes cluster,

Resources:
  OIDCArn:
    Type: AWS::SSM::Parameter
    Properties: 
      Name: 
        !Sub "${AWS::StackName}-eks-oidc-arn"
      Type: String
      Value: arn:aws:iam::012345678901:oidc-provider/oidc.eks.us-east-2.amazonaws.com/id/7185F12D2B62B8DA97B0ECA713F66C86

Outputs:
  OIDCArn:
    Value:
      !Ref OIDCArn
    Export:
      Name:
        'Fn::Sub': '${AWS::StackName}-eks-oidc-arn'

This can then be used within other CloudFormation stacks using ImportValue and resolve:ssm, and should any attempt be made to remove the output or delete the stack, it will fail with an error that Export ...-eks-oidc-arn cannot be deleted as it is in use by <stackname>.

The CloudFormation for importing the value is:

Outputs:
  ArnForIAMRole:
    Value:
      Fn::Sub:
        - "{{resolve:ssm:/${Stack}}}"
        - Stack:
            Fn::ImportValue: 'eks-myclustername-eks-oidc-arn'

Kubernetes, CloudFormation and OIDC Magic Numbers

Table Of Contents

Today I Explained