Error: Invalid reference from destroy provisioner-CodePudding

While upgrading terraform version, Facing such an error

    on athena.tf line 29, in resource "null_resource" "athena_views_base_views_1":
  29:     command = <<EOF
  30:       AWS_DEFAULT_REGION=${local.region} AWS_ACCESS_KEY_ID=${local.aws_access_key_id} AWS_SECRET_ACCESS_KEY=${local.aws_secret_key} \
  31:       aws athena start-query-execution \
  32:         --output json \
  33:         --query-string 'DROP VIEW IF EXISTS ${each.key}' \
  34:         --query-execution-context "Database=${self.triggers.database_name}" \
  35:         --result-configuration "OutputLocation=s3://${self.triggers.s3_bucket_query_output}"
  36: EOF

Destroy-time provisioners and their connection configurations may only reference attributes of the related resource, via 'self', 'count.index', or 'each.key'.

References to other resources during the destroy phase can cause dependency cycles and interact poorly with create_before_destroy.

Provider block is

resource "null_resource" "athena_views_base_views_1" {
  for_each = {
  for filename in fileset("${path.module}/sql-views/base/1_views/", "**") :
  replace(filename, "/", "_") => replace(file("${path.module}/sql-views/base/1_views/${filename}"),"<environment>", "${local.environment}")
  }


  triggers = {
    md5 = md5(each.value)

    database_name = data.terraform_remote_state.datalake.outputs.data_glue_catalog_base_name
    s3_bucket_query_output = data.terraform_remote_state.datalake.outputs.data_athena_results_bucket
  }

  provisioner "local-exec" {
    command = <<EOF
      AWS_DEFAULT_REGION=${local.region} AWS_ACCESS_KEY_ID=${local.aws_access_key_id} AWS_SECRET_ACCESS_KEY=${local.aws_secret_key} \
      aws athena start-query-execution \
        --output json \
        --query-string '${each.value}' \
        --query-execution-context "Database=${self.triggers.database_name}" \
        --result-configuration "OutputLocation=s3://${self.triggers.s3_bucket_query_output}"
EOF
  }

  provisioner "local-exec" {
    when = destroy
    command = <<EOF
      AWS_DEFAULT_REGION=${local.region} AWS_ACCESS_KEY_ID=${local.aws_access_key_id} AWS_SECRET_ACCESS_KEY=${local.aws_secret_key} \
      aws athena start-query-execution \
        --output json \
        --query-string 'DROP VIEW IF EXISTS ${each.key}' \
        --query-execution-context "Database=${self.triggers.database_name}" \
        --result-configuration "OutputLocation=s3://${self.triggers.s3_bucket_query_output}"
EOF
  }
}

Can someone help???

CodePudding user response：

Handling AWS credentials directly in Terraform expressions is a bit unconventional, and will cause your credentials to be saved as a part of your Terraform state. If you do this, please take care to ensure that you are storing your state snapshots appropriately to avoid compromising those credentials.

The meaning of this error is that the configuration for a destroy-time provisioner cannot refer to anything except the resource instance that's being destroyed, because by the time a destroy-time provisioner is running anything that resource depends on must already have been destroyed; the ordering for destroy operations is the inverse of the ordering for create/update operations.

If you need to keep track of some data used during the last update so that you can make use of it during destroy, you'll need to save that data as part of the resource instance's own attributes. Since you are using null_resource, the easiest way to achieve that is to place those values in the triggers argument, which is an map of arbitrary strings and so can contain any string values you wish:

  triggers = {
    md5 = md5(each.value)

    database_name = data.terraform_remote_state.datalake.outputs.data_glue_catalog_base_name
    s3_bucket_query_output = data.terraform_remote_state.datalake.outputs.data_athena_results_bucket

    aws_region        = local.region
    aws_access_key_id = local.aws_access_key_id
    aws_secret_key    = sensitive(local.aws_secret_key)
  }

  # ...

  provisioner "local-exec" {
    when = destroy
    command = <<EOF
      AWS_DEFAULT_REGION=${self.triggers.aws_region} AWS_ACCESS_KEY_ID=${self.triggers.aws_access_key_id} AWS_SECRET_ACCESS_KEY=${self.triggers.aws_secret_key} \
      aws athena start-query-execution \
        --output json \
        --query-string 'DROP VIEW IF EXISTS ${each.key}' \
        --query-execution-context "Database=${self.triggers.database_name}" \
        --result-configuration "OutputLocation=s3://${self.triggers.s3_bucket_query_output}"
EOF
  }

However, this approach can only work if the credentials most recently used to create or update this object remain valid at the time when you destroy it, and so I would not recommend this design.

Instead, a more typical answer for this problem of mixing use of both the Terraform AWS provider and the AWS CLI would be to place the credentials for both in one of the standard locations where AWS tools look for credentials -- in environment variables set from outside Terraform, in the ~/.aws/credentials file, etc -- and then both the provider and the CLI will both look in that location to find the credentials to use, without them appearing as any part of your Terraform configuration.

A Terraform configuration should only be a description of the desired state, not a description of which credentials to use to reach that desired state. Credentials should be placed in whatever locations are standard for the target system in question, and then the Terraform provider for that system should look in those standard locations directly, without the credentials ever passing through the Terraform configuration.