Updating Cloud Functions' source code requires changing zip path

[geekflyer]

Hi there,

I'm trying to create a cloud function via terraform (which in this particular example forwards error logs to slack, but that's irrelevant for the issue).

The problem is it seems impossible to update a cloud functions source code after its initial deployment via terraform.

As an example below is my hcl config code. You can see that as part of that code I'm packaging a node.js app located under ./app into a zip file, upload it to GCS and then use this as source for the cloud function. Whenever I change something in the source code under ./app terraform will rezip and upload the new archive to GCS. However the corresponding cloud function does not reload the source code from GCS. This is because none of the input params of the cloud function resource has been changed. In the AWS lambda resource they use an attribute source_code_hash to trigger updates to the function resource when the source code has changed.

The google_cloud_function resource doesn't have any attribute like that so I cannot trigger an update to the resource. I tried embedding the hash into the description or labels of the resource to trigger an update, and while this creates a new version, that new version still doesn't reload the new source code.
IMHO that makes the current terraform cloud function resource useless in practice. It can only be used to create an initial cloud function but not for updates.

Expectation:

Please add an attribute source_code_hash or similar to the cloud function resource to allow updates of the source code via terraform.

Terraform Version

Terraform v0.11.7
+ provider.archive v1.1.0
+ provider.google v1.16.2

Affected Resource(s)

Please list the resources as a list, for example:

• google_cloudfunctions_function

Terraform Configuration Files

main.tf

locals {
  error_log_filter = <<EOF
    resource.type="k8s_container"
    resource.labels.cluster_name="api-cluster-prod-b"
    severity>=ERROR
    EOF

  function_name    = "post-error-logs-to-slack"
  functions_region = "us-central1"
}

terraform {
  backend "gcs" {}
}

provider "google" {
  project = "${var.gcp_project}"
  region  = "${var.gcp_region}"
  version = "~> 1.16.2"
}

provider "archive" {
  version = "~> 1.1.0"
}

resource "google_pubsub_topic" "error_logs" {
  name = "error-logs"
}

resource "google_logging_project_sink" "error_logs_sink" {
  name        = "error-logs-sink"
  destination = "pubsub.googleapis.com/projects/${var.gcp_project}/topics/${google_pubsub_topic.error_logs.name}"
  filter      = "${local.error_log_filter}"
}

resource "google_storage_bucket" "functions_store" {
  name     = "solvvy-prod-functions"
  location = "${local.functions_region}"
}

data "archive_file" "function_dist" {
  type        = "zip"
  source_dir  = "./app"
  output_path = "dist/${local.function_name}.zip"
}

resource "google_storage_bucket_object" "error_logs_to_slack_function_code" {
  name   = "${local.function_name}.zip"
  bucket = "${google_storage_bucket.functions_store.name}"
  source = "${data.archive_file.function_dist.output_path}"
}

resource "google_cloudfunctions_function" "post-error-logs-to-slack" {
  name                  = "post-error-logs-to-slack"
  description           = "[Managed by Terraform] This function gets triggered by new messages in the ${google_pubsub_topic.error_logs.name} pubsub topic"
  available_memory_mb   = 128
  source_archive_bucket = "${google_storage_bucket_object.error_logs_to_slack_function_code.bucket}"
  source_archive_object = "${google_storage_bucket_object.error_logs_to_slack_function_code.name}"
  entry_point           = "sendErrorToSlack"
  trigger_topic         = "${google_pubsub_topic.error_logs.name}"
  region                = "${local.functions_region}"
}

b/249753001

[Chupaka]

1.17.0 (August 22, 2018)
IMPROVEMENTS:
cloudfunctions: Add support for updating function code in place (#1781)

So looks like you just need to update google provider :)

[geekflyer]

Nope, #1781 doesn't really solve my issue. With #1781 we only gain the ability to update the functions source if I change at the same time the path of the zip archive in GCS. In my case I only want to change the content of the .zip blob in-place in GCS (which won't trigger an update of the function currently) and not update its location / path all the time.

I can change the blob path dynamically by appending the content hash to it's path, but that's imho just an ugly workaround :).

locals {
  // we append the app hash to the filename as a temporary workaround for https://github.com/terraform-providers/terraform-provider-google/issues/1938
  filename_on_gcs = "${local.function_name}-${lower(replace(base64encode(data.archive_file.function_dist.output_md5), "=", ""))}.zip"
}

[paddycarver]

Sadly, I think that may be the only option we have, at the moment. :/ I don't see anything in the Cloud Functions API that suggests you can trigger a new deployment without changing the path of the deployed code. In theory, we could probably add something on the Terraform side to make this nicer, like a source_code_hash field, but in reality all that would do is include the source_code_hash in the file path behind the scenes, instead of making you do it. :/

I've opened an issue upstream about this.

[pmoriarty]

The REST API (https://cloud.google.com/functions/docs/reference/rest/v1/projects.locations.functions) seems to support versioning. Repeated attempts to deploy increment the returned versionId.

Would it be possible to detect that the bucket object is being updated and re-deploy cloud functions which depend on it?

[paddycarver]

Not with Terraform, unfortunately--the storage object resource's changes aren't visible to the function resource unless the field that changes is interpolated.

A simple workaround that should work is changing name = "${local.function_name}.zip" in the source archive object to name = "${local.function_name}.${data.archive_file.function_dist.output_base64sha256}.zip" or name = "${local.function_name}.${data.archive_file.function_dist.output_md5}.zip", which would include the SHA 256 sum of the contents of the zip file (or the MD5 sum) in the filename of the object, which would then make cloud functions notice that the source has changed and deploy a new version.

[edit] Which was already pointed out. Oops. Sorry about that.

[Kentzo]

This workaround is problematic because function's name cannot exceed 48 characters.

[quinn]

FWIW, I would like to thumbs down this enhancement, I don't think this is a reasonable feature request, given its not clear what heuristic would be used to detect a change. Leaving it up to the user is probably more reliable.

[timwsuqld]

I believe serverless puts each zip file in a folder with the upload timestamp. Then it updates the functions source URL to trigger the redeploy.
@quinn I don't see why this is an unreasonable request, as long as it's well documented how it works. Currently, a user spends a little while trying to get a function to update after they initially deploying it, before finding this issue and working out they need to change the file URL for each deploy. Making this automatic (src hash) makes a lot of sense.

[quinn]

@timwsuqld I see what you mean, but I think that is just part of the learning curve of cloud functions. You have to do the exact same thing with lambda + cloudformation (except cloudformation does not provide a way to zip and fingerprint automatically the way that terraform does).

Here are the issues with this that I see:

1. There's no easy way to detect changes. What if an imported file changes? or the version of a dependency? Not just the entrypoint file could change. Timestamps could work too, but that has the downside of always releasing a new version. Ideally, the config would be declarative and idempotent.
2. The zip data resource doesn't know its targeting a cloud function, so there doesn't seem to be a good place to implement this functionality. All the pieces involved need to be as decoupled as possible, and changing the path that gets used in the bucket based on the resource that is using it implies tight coupling.

[ASRagab]

Isn't a source_code_hash attribute on the resource, exactly how the aws_lambda terraform resource works? While not perfect, it feels like an 85% solution to a pretty common issue (code changes), why should the name of the deploy artifact have to change?

[bradenwright]

Any update on this, issue has been open over a year. Workaround gets it going but it's hacky