Skip to content

Automating AWS Access Key Rotation with Terraform, Keybase, Bitwarden Secrets Manager

In the evolving landscape of cloud security, managing AWS IAM Access Keys securely and efficiently is paramount. This guide introduces a robust method to automate the creation, deletion, and rotation of AWS IAM Access Keys using Terraform, Keybase, and Bitwarden Secrets Manager, ensuring your credentials are always up-to-date and securely stored.

This solution leverages Terraform to manage the lifecycle of AWS IAM Access Keys, Keybase for pgp encryption and decryption of secrets from AWS, and optionally Bitwarden Secrets Manager for storing and retrieving secrets across projects.

NOTE

For seamless project switching with automatic secret retrieval as environment variables, see Bitwarden Secrets Manager: Elevating Developer Environments.

Dependencies

Terraform

Terraform or OpenTofu is the Infrastructure as Code (IaC) tool used here to manage the lifecycle of these AWS IAM Access Keys. Terraform is a popular choice for managing cloud resources due to its declarative syntax and support for multiple cloud providers. With this approach, we can simply apply and/or destroy our Terraform configuration to create, delete, or update our AWS Access Keys. Multiple keys for various environments can be handled using Terraform Workspaces.

Keybase

To avoid storing the secret in plaintext in tfstate, the secret is encrypted with a pgp_key, in this case, the pgp key from Keybase. Keybase is a secure messaging and file-sharing platform that provides end-to-end encryption for messages and files. We can use Keybase to encrypt and decrypt our secrets securely.

Bitwarden Secrets Manager

Bitwarden Secrets Manager allows me to persist my secrets across projects and environments. I can easily retrieve my secrets as environment variables using the Secrets Manager CLI (bws). This is especially useful when working on multiple projects or switching between environments.

Solution

My tdharris/terraform-aws-access-key repository contains the Terraform configuration and supporting script to easily create, delete, and rotate AWS IAM Access Keys using Terraform, Keybase, and Bitwarden Secrets Manager.

Ensure you are in the correct AWS account before each operation:

shell
aws sts get-caller-identity

For all operations, select the appropriate Terraform workspace for the account or environment:

shell
terraform workspace select <name>

If the workspace does not exist, create it with:

shell
terraform workspace new <name>

Usage

The aws-key.sh script is a wrapper around the Terraform configuration to simplify the process of creating, rotating, and deleting AWS IAM Access Keys and decrypting them using Keybase. It also provides an option to sync the Key to Bitwarden Secrets Manager for storing and retrieving secrets across projects.

shell
Commands:
    create      Create a new AWS Key.
    delete      Delete an existing AWS Key.
    rotate      Rotate an existing AWS Key.
    decrypt     Output decrypted AWS Key.
    update      Update the status of an AWS Key (active or inactive). (default is active)
    bws-sync    Sync the AWS Key to Bitwarden Secrets Manager (customized).
  1. Create Key:

    shell
    ./aws-key.sh create
    Show Terraform Apply
    hcl
    Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
    + create
    
    Terraform will perform the following actions:
    
    # aws_iam_access_key.example will be created
    + resource "aws_iam_access_key" "example" {
        + create_date          = (known after apply)
        + encrypted_secret     = (known after apply)
        + id                   = (known after apply)
        + key_fingerprint      = (known after apply)
        + pgp_key              = "keybase:some_person_that_exists"
        + secret               = (sensitive value)
        + ses_smtp_password_v4 = (sensitive value)
        + status               = "Active"
        + user                 = "mycli"
        }
    
    Plan: 1 to add, 0 to change, 0 to destroy.
    
    Changes to Outputs:
    + aws_access_key_id   = (known after apply)
    + aws_caller_identity = {
        + account_id = "<account-id>"
        + arn        = "arn:aws:iam::<account-id>:user/mycli"
        + id         = "<account-id>"
        + user_id    = "<user-id>"
        }
    + aws_user_name       = "mycli"
    + encrypted_secret    = (sensitive value)
    aws_iam_access_key.example: Creating...
    aws_iam_access_key.example: Creation complete after 1s [id=<new-access-key-id>]
    
    Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
    
    Outputs:
    
    aws_access_key_id = "<new-access-key-id>"
    aws_caller_identity = {
    "account_id" = "<account-id>"
    "arn" = "arn:aws:iam::<account-id>:user/mycli"
    "id" = "<account-id>"
    "user_id" = "<user-id>"
    }
    aws_user_name = "mycli"
    encrypted_secret = <sensitive>
    shell
    # Retrieved from tfstate, parsed and decrypted using Keybase
    Decrypted Outputs:
    
    aws_user_name = "mycli"
    aws_access_key_id = "<new-access-key-id>"
    aws_decrypted_secret = "<decrypted-secret>"
  2. Rotate Key:

    shell
    ./aws-key.sh rotate
    shell
    # Example
    $ ./aws-key.sh rotate
    Previously invoked identity (tfstate):
    {
        "account_id" = "<account-id>"
        "arn" = "arn:aws:iam::<account-id>:user/mycli"
        "id" = "<account-id>"
        "user_id" = "<user-id>"
    }
    Current AWS Identity:
    {
        "UserId": "<user-id>",
        "Account": "<account-id>",
        "Arn": "arn:aws:iam::<account-id>:user/mycli"
    }
    Validating tfstate identity is the same as current aws identity...
     Success
    
    Rotate aws access key '<old-access-key-id>' ? [y/n] y
    Rotating access key with terraform destroy, then apply...
    Show Terraform Destroy & Apply
    hcl
    # Destroy
    aws_iam_access_key.example: Refreshing state... [id=<old-access-key-id>]
    
    Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
    - destroy
    
    Terraform will perform the following actions:
    
    # aws_iam_access_key.example will be destroyed
    - resource "aws_iam_access_key" "example" {
        - create_date          = "<date>" -> null
        - encrypted_secret     = "<secret>" -> null
        - id                   = "<old-access-key-id>" -> null
        - key_fingerprint      = "<fingerprint>" -> null
        - pgp_key              = "keybase:some_person_that_exists" -> null
        - ses_smtp_password_v4 = (sensitive value)
        - status               = "Active" -> null
        - user                 = "mycli" -> null
        }
    
    Plan: 0 to add, 0 to change, 1 to destroy.
    
    Changes to Outputs:
    - aws_access_key_id   = "<old-access-key-id>" -> null
    - aws_caller_identity = {
        - account_id = "<account-id>"
        - arn        = "arn:aws:iam::<account-id>:user/mycli"
        - id         = "<account-id>"
        - user_id    = "<user-id>"
        } -> null
    - aws_user_name       = "mycli" -> null
    - encrypted_secret    = (sensitive value)
    aws_iam_access_key.example: Destroying... [id=<old-access-key-id>]
    aws_iam_access_key.example: Destruction complete after 0s
    
    Destroy complete! Resources: 1 destroyed.
    hcl
    # Apply
    Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
    + create
    
    Terraform will perform the following actions:
    
    # aws_iam_access_key.example will be created
    + resource "aws_iam_access_key" "example" {
        + create_date          = (known after apply)
        + encrypted_secret     = (known after apply)
        + id                   = (known after apply)
        + key_fingerprint      = (known after apply)
        + pgp_key              = "keybase:some_person_that_exists"
        + secret               = (sensitive value)
        + ses_smtp_password_v4 = (sensitive value)
        + status               = "Active"
        + user                 = "mycli"
        }
    
    Plan: 1 to add, 0 to change, 0 to destroy.
    
    Changes to Outputs:
    + aws_access_key_id   = (known after apply)
    + aws_caller_identity = {
        + account_id = "<account-id>"
        + arn        = "arn:aws:iam::<account-id>:user/mycli"
        + id         = "<account-id>"
        + user_id    = "<user-id>"
        }
    + aws_user_name       = "mycli"
    + encrypted_secret    = (sensitive value)
    aws_iam_access_key.example: Creating...
    aws_iam_access_key.example: Creation complete after 1s [id=<new-access-key-id>]
    
    Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
    
    Outputs:
    
    aws_access_key_id = "<new-access-key-id>"
    aws_caller_identity = {
    "account_id" = "<account-id>"
    "arn" = "arn:aws:iam::<account-id>:user/mycli"
    "id" = "<account-id>"
    "user_id" = "<user-id>"
    }
    aws_user_name = "mycli"
    encrypted_secret = <sensitive>
    shell
    # Retrieved from tfstate, parsed and decrypted using Keybase
    Decrypted Outputs:
    
    aws_user_name = "mycli"
    aws_access_key_id = "<new-access-key-id>"
    aws_decrypted_secret = "<decrypted-secret>"
  3. Decrypt Key:

    shell
    ./aws-key.sh decrypt
    shell
    # Retrieved from tfstate, parsed and decrypted using Keybase
    Decrypted Outputs:
    
    aws_user_name = "mycli"
    aws_access_key_id = "<new-access-key-id>"
    aws_decrypted_secret = "<decrypted-secret>"
  4. Import Existing Key:

    shell
    terraform import aws_iam_access_key.key <access_key_id>
  5. Deactivate/Activate Key:

    shell
    ./aws-key.sh update inactive
    ./aws-key.sh update active
  6. Delete Key:

    shell
    ./aws-key.sh delete
    Show Terraform Destroy
    hcl
    # Destroy
    aws_iam_access_key.example: Refreshing state... [id=<old-access-key-id>]
    
    Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
    - destroy
    
    Terraform will perform the following actions:
    
    # aws_iam_access_key.example will be destroyed
    - resource "aws_iam_access_key" "example" {
        - create_date          = "<date>" -> null
        - encrypted_secret     = "<secret>" -> null
        - id                   = "<old-access-key-id>" -> null
        - key_fingerprint      = "<fingerprint>" -> null
        - pgp_key              = "keybase:some_person_that_exists" -> null
        - ses_smtp_password_v4 = (sensitive value)
        - status               = "Active" -> null
        - user                 = "mycli" -> null
        }
    
    Plan: 0 to add, 0 to change, 1 to destroy.
    
    Changes to Outputs:
    - aws_access_key_id   = "<old-access-key-id>" -> null
    - aws_caller_identity = {
        - account_id = "<account-id>"
        - arn        = "arn:aws:iam::<account-id>:user/mycli"
        - id         = "<account-id>"
        - user_id    = "<user-id>"
        } -> null
    - aws_user_name       = "mycli" -> null
    - encrypted_secret    = (sensitive value)
    aws_iam_access_key.example: Destroying... [id=<old-access-key-id>]
    aws_iam_access_key.example: Destruction complete after 0s
    
    Destroy complete! Resources: 1 destroyed.
  7. Sync Key to Bitwarden Secrets Manager:

    shell
    ./aws-key.sh bws-sync
    shell
     ./aws-key.sh bws-sync
    INFO: Exec Decrypt AWS Key...
    
    Decrypted Outputs:
    
    aws_user_name = "<redacted>"
    aws_access_key_id = "<redacted>"
    aws_decrypted_secret = "<redacted>"
    
    INFO: Exec Bitwarden Secrets Manager Sync...
    INFO: Retrieving secret name based on current terraform workspace...
    INFO: Finding secret '<terraform_workspace_name>'...
    INFO: Editing secret '<redacted>'...
    INFO: New Secret Value:
    {
        "BW_ITEM_NAME": "<redacted>",
        "AWS_ACCESS_KEY_ID": "<redacted>",
        "AWS_SECRET_ACCESS_KEY": "<redacted>",
        "AWS_MFA_ARN": "arn:aws:iam::<redacted>:mfa/<redacted>",
        "AWS_DEFAULT_REGION": "<redacted>"
    }
    Update secret '<redacted>' with above value? [y/n] y
    {
        "id": "<redacted>",
        "organizationId": "<redacted>",
        "projectId": "<redacted>",
        "key": "<redacted>",
        "value": "{\n  \"BW_ITEM_NAME\": \"<redacted>\",\n  \"AWS_ACCESS_KEY_ID\": \"<redacted>\",\n  \"AWS_SECRET_ACCESS_KEY\": \"<redacted>\"\n}",
        "note": "",
        "creationDate": "<date>",
        "revisionDate": "<date>"
    }
    
     Success

    NOTE

    • The name of the secret is the same as the Terraform workspace name.
    • Since I use Bitwarden Secrets Manager to load my secrets as environment variables across projects, I am automatically all set anywhere I need to use these secrets. Checkout Bitwarden Secrets Manager: Elevating Developer Environments for more details.

Function for Rotating AWS Access Keys

Now we can tie all this together in a convenient helper script that does all the heavy lifting for us, making it easy to rotate our AWS Access Keys with a single command. It performs the following steps:

  1. Navigates to the specified Terraform AWS Access Key directory.
  2. Verifies the current AWS identity.
  3. Prompts the user to select the appropriate Terraform workspace.
  4. Ensures Keybase is running.
  5. Rotates the AWS Access Key using a script.
  6. Optionally syncs the new key with Bitwarden if enabled.
shell
TERRAFORM_AWS_ACCESS_KEY_DIR="<path-to>/terraform-aws-access-key"

function secret_rotate_aws_keys {
    local -r enable_bws_sync="${1:-true}"
    local -r log_prefix="[secret_rotate_aws_keys]"

    cd "$TERRAFORM_AWS_ACCESS_KEY_DIR" || {
        log error "Failed to cd into $TERRAFORM_AWS_ACCESS_KEY_DIR"
        return 1
    }

    log info "$log_prefix Starting AWS Access Key Rotation"
    aws sts get-caller-identity --no-cli-pager
    if ! ask "Do you want to continue with the current aws identity?"; then
        return 1
    fi

    log info "$log_prefix Select the TF Workspace aligned with the AWS Access Key to rotate"
    local -r workspace="$(terraform workspace list | fzf --prompt " Select the TF Workspace > " | sed 's|* ||g;s|^[[:space:]]*||')"

    terraform workspace select "$workspace" || {
        log error "Failed to select the TF Workspace: $workspace"
        return 1
    }
    log info "$log_prefix tf workspace: $workspace"

    log info "Ensuring keybase is running"
    if ! keybase whoami 2>/dev/null; then
        log info "$log_prefix Keybase is not running, attempting to start it with: run_keybase -g"
        run_keybase -g && keybase whoami
    fi

    log info "$log_prefix Rotating AWS Access Key"
    ./aws-key.sh rotate || {
        log error "Failed to rotate AWS Access Key"
        return 1
    }

    if [[ "$enable_bws_sync" == "true" ]]; then
        log info "$log_prefix Syncing with Bitwarden: ./aws-key.sh bws-sync"
        ./aws-key.sh bws-sync
    fi
}

Final Thoughts

Automating the creation, deletion, and rotation of AWS IAM Access Keys is essential for maintaining a secure and efficient cloud environment. By using Terraform, Keybase, and Bitwarden Secrets Manager, we can ensure our credentials are always up-to-date and securely stored. This solution provides a robust method for managing AWS IAM Access Keys across multiple environments and projects, making it easier to maintain a secure cloud infrastructure.

Deployed on Deno 🦕