IAM Roles with Service Accounts

Use IAM Roles with Kubernetes Service Accounts to provide AWS authentication for Edge Delta.

  6 minute read  

Overview

Maintaining security is crucial when handling AWS credentials. Core principles include:

  • Use Temporary Credentials: Use IAM roles and temporary credentials whenever possible to minimize the risk of credential exposure.
  • Environment Variable Security: Ensure that environment variables containing credentials are managed securely and not hard-coded.
  • Rotate Credentials Regularly: Regularly rotate IAM roles and access keys to enhance security and minimize the impact if credentials are compromised.
  • Monitor and Audit Activities: Track and audit actions performed with your AWS credentials, ensuring ongoing oversight and compliance.

Authorizing Edge Delta

There are three methods to provide AWS authentication for Edge Delta. Leveraging IAM roles with Kubernetes service accounts (Option 3) is the best practice. It enhances the security of your AWS authentication processes within Edge Delta while simplifying credential management and reducing potential risks.

Option 1. Providing AWS Access Keys and Secrets in the UI

This involves entering your AWS access key and secret key directly in the node output form on Edge Delta.

Create an AWS destination node such as an S3 destination:

  1. Click Pipelines.
  2. Select a fleet that you want to add an AWS destination to.
  3. Click View/Edit Pipeline.
  4. Click Edit Mode.
  5. Click Add Destination.
  6. Expand Archive and select S3 Destination.
  7. Configure the output by filling in the form, including the AWS Key and AWS Secret Key.

The Edge Delta platform will securely store and use these credentials for authenticating with AWS services.

Option 2. Setting Environment Variables

This method requires setting the AWS access key and secret key as environment variables within your Kubernetes configuration for Edge Delta.

Kubectl Example:

  1. Download the Edge Delta Kubernetes Manifest.

You can also find the Edge Delta manifest location in the Kubernetes installation instructions:

  1. Open the manifest file and edit the DaemonSet to add environment variables:
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: edgedelta
  namespace: edgedelta
spec:
  containers:
    - name: edgedelta-agent
      image: gcr.io/edgedelta/agent:v0.1.105
      env:
        - name: AWS_ACCESS_KEY_ID
          value: "<YOUR_ACCESS_KEY>"   
        - name: AWS_SECRET_ACCESS_KEY
          value: "<YOUR_SECRET_KEY>"   
...

Replace <YOUR_ACCESS_KEY> and <YOUR_SECRET_KEY> with your AWS credentials.

  1. Use kubectl to apply the updated manifest:
kubectl apply -f updated-manifest.yaml
  1. Reference each secret as follows in the Edge Delta UI or YAML when configuring a Pipeline:

{{ Env "AWS_ACCESS_KEY_ID" }} {{ Env "AWS_SECRET_ACCESS_KEY" }}

For example:

- name: s3_output
  type: s3_output
  region: us-west-2
  aws_key_id: `{{ Env "AWS_ACCESS_KEY_ID" }}`
  aws_sec_key: `{{ Env "AWS_SECRET_ACCESS_KEY" }}`
  bucket: test
  use_native_compression: false
  disable_metadata_ingestion: false

Helm Example:

  1. Pass the environment variables in when installing with helm:
helm upgrade edgedelta edgedelta/edgedelta -i --version v0.1.105 --set secretApiKey.value=12345678998765431 -n edgedelta --create-namespace \
  --set agent.env[0].name=AWS_ACCESS_KEY_ID \
  --set agent.env[0].value=<YOUR_ACCESS_KEY> \
  --set agent.env[1].name=AWS_SECRET_ACCESS_KEY \
  --set agent.env[1].value=<YOUR_SECRET_KEY>

Replace <YOUR_ACCESS_KEY> and <YOUR_SECRET_KEY> with your AWS credentials.

Alternatively, create Kubernetes secrets with a secrets manager such as bitnami kubeseal. See GitOps Principles Deployment of Edge Delta

  1. Reference each secret as follows in the Edge Delta UI or YAML when configuring a Pipeline:

{{ Env "AWS_ACCESS_KEY_ID" }} {{ Env "AWS_SECRET_ACCESS_KEY" }}

For example:

- name: s3_output
  type: s3_output
  region: us-west-2
  aws_key_id: `{{ Env "AWS_ACCESS_KEY_ID" }}`
  aws_sec_key: `{{ Env "AWS_SECRET_ACCESS_KEY" }}`
  bucket: test
  use_native_compression: false
  disable_metadata_ingestion: false

Option 3. Leveraging IAM Roles with Kubernetes Service Accounts (Best Practice)

This method is considered the best practice for providing AWS credentials to applications running in Kubernetes, especially on Amazon EKS. It leverages IAM roles to grant Kubernetes pods the necessary AWS permissions without embedding long-lived credentials in the application.

Kubectl Example:

  1. Download the Edge Delta Kubernetes Manifest.

You can also find the Edge Delta manifest location in the Kubernetes installation instructions:

  1. Update the Edge Delta manifest’s service account and annotate it with the IAM role ARN. This ties the Kubernetes service account to an IAM role that has the necessary permissions.
apiVersion: v1
kind: ServiceAccount
metadata:
  name: edgedelta 
  namespace: edgedelta
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME>

Replace arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME> with the ARN of the IAM role you want the service account to assume.

  1. Ensure the Edge Delta DaemonSet and StatefulSets Use this Service Account:

The Edge Delta DaemonSet, along with the Compactor Agent StatefulSets and rollup Agent Deployment, should refer to this service account in their specifications.

For example, the DaemonSet in the manifest should look like this:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: edgedelta
  namespace: edgedelta
spec:
  serviceAccountName: edgedelta  # Use the service account specified above
  containers:
    - name: edgedelta-agent
      image: gcr.io/edgedelta/agent:v0.1.105
      # Other container specs...
  1. When configuring an AWS output or destination such as an S3 destination, leave the aws_key_id and aws_sec_key fields blank.

Helm Example 1:

  1. Create a Helm override file (for example values.yaml) to update the Edge Delta service account and annotate it with the IAM role ARN. This ties the Kubernetes service account to an IAM role that has the necessary permissions.
serviceAccount:
  create: true
  name: edgedelta
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME>  # Replace with your actual IAM Role ARN

Replace arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME> with the ARN of the IAM role you want the service account to assume.

  1. Pass in values override by adding -f values.yaml to the helm upgrade command:
helm upgrade edgedelta edgedelta/edgedelta -i --version v0.1.105 --set secretApiKey.value=1234567899876543621 -n edgedelta --create-namespace -f values.yaml
  1. When configuring an AWS output or destination such as an S3 destination, leave the aws_key_id and aws_sec_key fields blank.

Helm Example 2: Alternatively, instead of creating an override file, you can achieve the same configuration by passing in parameters directly via the Helm command line.

  1. Pass in role annotation
helm upgrade edgedelta edgedelta/edgedelta -i --version v0.1.105 --set secretApiKey.value=1234567899876543621 -n edgedelta --create-namespace \
  --set serviceAccount.create=true \
  --set serviceAccount.name=edgedelta \
  --set serviceAccount.annotations.eks\.amazonaws\.com/role-arn=arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME>

Replace arn:aws:iam::<ACCOUNT_ID>:role/<ROLE_NAME> with the ARN of the IAM role you want the service account to assume.

  1. When configuring an AWS output or destination such as an S3 destination, leave the aws_key_id and aws_sec_key fields blank.

Summary

When leveraging IAM Roles with Kubernetes Service Accounts, kubernetes injects a token into the pod, which the AWS SDK uses to obtain temporary credentials for accessing AWS services. This token is used by the AWS SDK to dynamically obtain temporary credentials that IAM roles provide.

Benefits of Using IAM Roles with Kubernetes Service Accounts

  • Security: Least Privilege: Easily enforce the principle of least privilege by assigning only the necessary permissions to the IAM role.
  • Credential Management: Reduces the operational overhead of managing and rotating long-lived credentials.
  • Simplicity and Automation: Automatic Token Rotation: AWS handles the automatic rotation and expiration of the temporary credentials.
  • Seamless Integration: AWS SDKs are built to work seamlessly with IAM roles, making the integration straightforward.
  • Scalability: Centralized Management: Centralize permissions management through IAM roles, making it easier to audit and update permissions. Replicability: Easily replicate configurations across different environments by reusing IAM roles.