Terraform surgery: safely rename resources without destroying them

Have you ever run into this scenario while renaming a resource in your terraform code?

Plan: 1 to add, 0 to change, 1 to destroy.

But you didn't add a new resource. And there's still data in your S3 bucket. Destroying the bucket would far from ideal. How do we tell Terraform that we don't want to replace the resource but simply rename it?

Let's walk through a straightforward example: a super generic S3 bucket declaration.

resource aws_s3_bucket s3_bucket {
  bucket = "assets"
}

At some point, you apply your terraform changes. But you forgot something. You realize the S3 bucket name is too generic. So you rename your S3 bucket to something specific so you can better remember its purpose:

resource aws_s3_bucket very_specific_s3_bucket {
  bucket = "assets"
}

Now you run terraform plan to see the expected changes when you see the dreaded message: Plan: 1 to add, 0 to change, 1 to destroy.

$ terraform plan
Terraform will perform the following actions:

  # aws_s3_bucket.very_specific_s3_bucket will be created
  + resource "aws_s3_bucket" "very_specific_s3_bucket" {
      + acceleration_status         = (known after apply)
      + acl                         = (known after apply)
      + arn                         = (known after apply)
      + bucket                      = "assets"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + object_lock_enabled         = (known after apply)
      + policy                      = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = {
          + "CreatedBy"   = "terraform"
        }
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)
    }

  # aws_s3_bucket.s3_bucket will be destroyed
  # (because aws_s3_bucket.s3_bucket is not in configuration)
  - resource "aws_s3_bucket" "s3_bucket" {
      - arn                         = "arn:aws:s3:::assets" -> null
      - bucket                      = "assets" -> null
      - bucket_domain_name          = "assets.s3.amazonaws.com" -> null
      - bucket_regional_domain_name = "assets.s3.amazonaws.com" -> null
      - force_destroy               = false -> null
      - hosted_zone_id              = "..." -> null
      - id                          = "assets" -> null
      - object_lock_enabled         = false -> null
      - region                      = "us-east-1" -> null
      - request_payer               = "BucketOwner" -> null
      - tags                        = {} -> null
      - tags_all                    = {
          - "CreatedBy"   = "terraform"
        } -> null
      - grant {
          - id          = "..." -> null
          - permissions = [
              - "FULL_CONTROL",
            ] -> null
          - type        = "CanonicalUser" -> null
        }
      - server_side_encryption_configuration {
          - rule {
              - bucket_key_enabled = false -> null
              - apply_server_side_encryption_by_default {
                  - sse_algorithm = "AES256" -> null
                }
            }
        }
      - versioning {
          - enabled    = false -> null
          - mfa_delete = false -> null
        }
    }
Plan: 1 to add, 0 to change, 1 to destroy.

How do you fix this? Start by adding this line:

moved {
  from = aws_s3_bucket.s3_bucket
  to = aws_s3_bucket.very_specific_s3_bucket
}

Then run terraform plan to see if your terraform output changes. You should see this output upon running the plan command:

  # aws_s3_bucket.s3_bucket has moved to aws_s3_bucket.very_specific_s3_bucket
    resource "aws_s3_bucket" "very_specific_s3_bucket" {
        id                          = "assets"
        tags                        = {}
        # (10 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

Now that you've validated your Terraform state will be updated to reflect this resource reference change, you can confidently run terraform apply without fear that your S3 bucket will be accidentally destroyed. Once the changes are applied successfully, you can remove the moved {} block. No subsequent terraform apply is necessary.

But what about terraform state mv?

An alternative to using the moved { ... } block is the terraform state mv command. But using it runs counter to the intent of terraform. By running this command, you lose the chance to see your changes planned before they're applied. Without that, it's very easy to make mistakes. Whereas using the moved {} block allows you to perform a dry run of your changes and confirm that you will not accidentally make an disruptive change.

If you're still not convinced that the moved {} block is the better solution, consider why you're even using Terraform in the first place: "infrastructure as code". When submitting a pull request for your code changes, it should be clear to the person reviewing the changes before you actually make the change. You can always submit a subsequent pull request to delete the moved blocks after you've confirmed that the resource name changes have been successfully applied. If neither of my reasons convince you, Terraform is probably not the right tool for you.

Despite having said that, I'll explain one caveat...

Okay but what about for_each loops?

There's one exception to that rule that I've seen so far. And it's not really an exception if you're using the latest stable release of Terraform (which is v1.6.6 as of this writing).

For versions as recent as v1.5.7, the moved { ... } block does not work when applied to resources created using a for_each loop. This isn't ideal if your infrastructure is sufficiently large enough because it's annoying to repeat yourself.

Here's a basic example:

resource aws_s3_bucket s3_bucket {
  bucket = "assets"
}
moved {
  from = aws_s3_bucket.s3_bucket
  to = aws_s3_bucket.all_my_s3_buckets["very_specific_s3_bucket"]
}

resource aws_s3_bucket second_s3_bucket {
  bucket = "other-assets"
}
moved {
  from = aws_s3_bucket.second_s3_bucket
  to = aws_s3_bucket.all_my_s3_buckets["another_s3_bucket"]
}

variable lots_of_buckets {
  type = map(string)
  default = {
    "very_specific_s3_bucket": "assets",
    "another_s3_bucket": "other-assets"
  }
}
resource aws_s3_bucket all_my_s3_buckets {
  for_each = var.lots_of_buckets
  bucket   = each.value
}

Here, I've defined two S3 buckets and I've consolidated them because they're configured exactly the same way with the exception of their bucket names.

How will Terraform behave when you do this? You'll get this message:

Plan: 2 to add, 0 to change, 2 to destroy.

But why? I don't know. But suffice it to say there's probably an article about it somewhere else. I'll limit my discussion to when it's ACTUALLY worth using the terraform state mv command.

This is what the output of terraform plan will look like when trying to move distinct resource declarations into a for_each loop:

$ terraform plan
Terraform will perform the following actions:

  # aws_s3_bucket.all_my_s3_buckets["very_specific_s3_bucket"] will be created
  + resource "aws_s3_bucket" "all_my_s3_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = (known after apply)
      + arn                         = (known after apply)
      + bucket                      = "assets"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + object_lock_enabled         = (known after apply)
      + policy                      = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = {
          + "CreatedBy"   = "terraform"
        }
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)
    }

  # aws_s3_bucket.very_specific_s3_bucket will be destroyed
  # (because aws_s3_bucket.very_specific_s3_bucket was moved to aws_s3_bucket.all_my_s3_buckets["very_specific_s3_bucket"], which is not in configuration)
  # (moved from aws_s3_bucket.very_specific_s3_bucket)
  - resource "aws_s3_bucket" "very_specific_s3_bucket" {
      - arn                         = "arn:aws:s3:::assets" -> null
      - bucket                      = "assets" -> null
      - bucket_domain_name          = "assets.s3.amazonaws.com" -> null
      - bucket_regional_domain_name = "assets.s3.amazonaws.com" -> null
      - force_destroy               = false -> null
      - hosted_zone_id              = "..." -> null
      - id                          = "assets" -> null
      - object_lock_enabled         = false -> null
      - region                      = "us-east-1" -> null
      - request_payer               = "BucketOwner" -> null
      - tags                        = {} -> null
      - tags_all                    = {
          - "CreatedBy"   = "terraform"
        } -> null
      - grant {
          - id          = "..." -> null
          - permissions = [
              - "FULL_CONTROL",
            ] -> null
          - type        = "CanonicalUser" -> null
        }
      - server_side_encryption_configuration {
          - rule {
              - bucket_key_enabled = false -> null
              - apply_server_side_encryption_by_default {
                  - sse_algorithm = "AES256" -> null
                }
            }
        }
      - versioning {
          - enabled    = false -> null
          - mfa_delete = false -> null
        }
    }

  # aws_s3_bucket.all_my_s3_buckets["another_s3_bucket"] will be created
  + resource "aws_s3_bucket" "all_my_s3_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = (known after apply)
      + arn                         = (known after apply)
      + bucket                      = "other-assets"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = false
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + object_lock_enabled         = (known after apply)
      + policy                      = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = {
          + "CreatedBy"   = "terraform"
        }
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)
    }

  # aws_s3_bucket.another_s3_bucket will be destroyed
  # (because aws_s3_bucket.another_s3_bucket was moved to aws_s3_bucket.all_my_s3_buckets["another_s3_bucket"], which is not in configuration)
  # (moved from aws_s3_bucket.another_s3_bucket)
  - resource "aws_s3_bucket" "another_s3_bucket" {
      - arn                         = "arn:aws:s3:::assets" -> null
      - bucket                      = "other-assets" -> null
      - bucket_domain_name          = "other-assets.s3.amazonaws.com" -> null
      - bucket_regional_domain_name = "other-assets.s3.amazonaws.com" -> null
      - force_destroy               = false -> null
      - hosted_zone_id              = "..." -> null
      - id                          = "other-assets" -> null
      - object_lock_enabled         = false -> null
      - region                      = "us-east-1" -> null
      - request_payer               = "BucketOwner" -> null
      - tags                        = {} -> null
      - tags_all                    = {
          - "CreatedBy"   = "terraform"
        } -> null
      - grant {
          - id          = "..." -> null
          - permissions = [
              - "FULL_CONTROL",
            ] -> null
          - type        = "CanonicalUser" -> null
        }
      - server_side_encryption_configuration {
          - rule {
              - bucket_key_enabled = false -> null
              - apply_server_side_encryption_by_default {
                  - sse_algorithm = "AES256" -> null
                }
            }
        }
      - versioning {
          - enabled    = false -> null
          - mfa_delete = false -> null
        }
    }
Plan: 2 to add, 0 to change, 2 to destroy.

How do you solve this?

$ terraform state mv 'aws_s3_bucket.very_specific_s3_bucket' 'aws_s3_bucket.all_my_s3_buckets["very_specific_s3_bucket"]')
Move "aws_s3_bucket.very_specific_s3_bucket" to "aws_s3_bucket.all_my_s3_buckets[\"very_specific_s3_bucket\"]"
Successfully moved 1 object(s).
Releasing state lock. This may take a few moments...

$ terraform state mv 'aws_s3_bucket.another_s3_bucket' 'aws_s3_bucket.all_my_s3_buckets["another_s3_bucket"]')
Move "aws_s3_bucket.another_s3_bucket" to "aws_s3_bucket.all_my_s3_buckets[\"another_s3_bucket\"]"
Successfully moved 1 object(s).
Releasing state lock. This may take a few moments...

That's about it. Drop me a message on Mastodon if you have feedback.