Terraform always shows archive_file in plan

[maroux]

Hi there,

Terraform Version

Terraform v0.11.3

• provider.archive v1.0.0
• provider.null v1.0.0

Affected Resource(s)

Please list the resources as a list, for example:

• data.archive_file

Terraform Configuration Files

  triggers {
    index = "${base64sha256(file("${path.module}/lambda-files/index.js"))}"
  }
}

data "archive_file" "lambda_exporter" {
  output_path = "${path.module}/lambda-files.zip"
  source_dir  = "${path.module}/lambda-files/"
  type        = "zip"

  depends_on = ["null_resource.lambda_exporter"]
}

Debug Output

Apply once, and then re-plan:

------------------------------------------------------------------------

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
 <= read (data resources)

Terraform will perform the following actions:

 <= data.archive_file.lambda_exporter
      id:                  <computed>
      output_base64sha256: <computed>
      output_md5:          <computed>
      output_path:         "/Users/amaru/work/tfbug/lambda-files.zip"
      output_sha:          <computed>
      output_size:         <computed>
      source.#:            <computed>
      source_dir:          "/Users/amaru/work/tfbug/lambda-files/"
      type:                "zip"


Plan: 0 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Expected Behavior

Terraform shouldn't show any changes on terraform plan since the hash of file has not changed.

Actual Behavior

Terraform shows a change with read (data resources) for data.archive_file.lambda_exporter

Steps to Reproduce

Please list the steps required to reproduce the issue, for example:

1. terraform apply
2. terraform plan

[apparentlymart]

Hi @maroux! Sorry I didn't notice this issue before.

What you are seeing here is actually a Terraform Core limitation, where depends_on is not properly supported for data resources. You can see more details about this in hashicorp/terraform#17034, along with our current plan for how to address it in a future release.

The usual workaround for this is to use implicit dependencies rather than explicit dependencies. This works because the explicit dependency gives Terraform's planning engine more information: it can tell what aspect of the other resource is significant and thus determine when it is safe to produce the archive early, during the plan phase.

Unfortunately this is hard to do directly with the archive_file data source because it doesn't have any convenient attributes that can be interpolated from the null_resource to create those implicit dependencies. This can in turn be worked around by making use of the null_data_source data source as an extra indirection:

resource "null_resource" "lambda_exporter" {
  # (some local-exec provisioner blocks, presumably...)

  triggers {
    index = "${base64sha256(file("${path.module}/lambda-files/index.js"))}"
  }
}

data "null_data_source" "wait_for_lambda_exporter" {
  inputs = {
    # This ensures that this data resource will not be evaluated until
    # after the null_resource has been created.
    lambda_exporter_id = "${null_resource.lambda_exporter.id}"

    # This value gives us something to implicitly depend on
    # in the archive_file below.
    source_dir = "${path.module}/lambda-files/"
  }
}

data "archive_file" "lambda_exporter" {
  output_path = "${path.module}/lambda-files.zip"
  source_dir  = "${data.null_data_source.wait_for_lambda_exporter.outputs["source_dir"]}"
  type        = "zip"
}

Introducing this extra data source allows us to use an implicit dependency rather than an explicit dependency in both cases: data.null_data_source.wait_for_lambda_exporter implicitly depends on null_resource.lambda_exporter, and archive_file.lambda_exporter implicitly depends on datanull_data_source.wait_for_lambda_exporter.

This extra complexity should be unnecessary after we complete the work described in hashicorp/terraform#17034.

Please note that we do not generally recommend using Terraform to construct deployment artifacts, because it is not an application build tool; the archive_file feature is offered to enable some unusual cases, but in most situations it's better to construct a lambda function package via a separate build process -- for example, in the CI system for the application being deployed -- and use Terraform only for the final step of creating/updating the lambda function using that zip file. There are more details on this in our guide on using AWS Lambda and API Gateway with Terraform.

[apparentlymart]

Since this is a Terraform Core issue and we already have hashicorp/terraform#17034 open for it, I'm going to close this issue just to consolidate the discussion over there. Thanks for reporting this!

[ChineduUzoka]

Nice works for me

[joestump]

@ChineduUzoka really? Just tried the pattern @apparentlymart suggested and it failed. I've reverted back to a uuid trigger on my Lambda zip files. 😢

[ChineduUzoka]

@joestump Take a look at this file here - https://github.com/ChineduUzoka/tf-module-lambda-deployment/blob/master/aws/modules/lambda_deployment/main.tf

it works flawlessly

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.