unable to upload google_storage_bucket_objects larger than ~16MiB (16777215 bytes)

[flokli]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

v0.12.23
terraform-provider-google: 3.14.0

Affected Resource(s)

• google_storage_bucket_object
• google_storage_bucket_object

Terraform Configuration Files

resource "google_storage_bucket" "image_bucket" {
  name     = "foobar-images-test"
  location = "us-west2"
}

resource "google_storage_bucket_object" "exampledata" {
  bucket       = google_storage_bucket.image_bucket.name
  content_type = "application/tar+gzip"
  name         = "example.data"
  source       = "${path.module}/example.data"
}

Debug Output

$ terraform apply -auto-approve
google_storage_bucket.image_bucket: Refreshing state... [id=foobar-images-test]
google_storage_bucket_object.nixos: Creating...

Error: Error uploading object example.data: Post "https://www.googleapis.com/upload/storage/v1/b/foobar-images-test/o?alt=json&name=example.data&prettyPrint=false&uploadType=resumable": unable to copy invalid http.Request for retry: invalid HTTP request for transport, expected request.GetBody for non-empty Body

  on buildkite_agent.tf line 6, in resource "google_storage_bucket_object" "exampledata":
   6: resource "google_storage_bucket_object" "exampledata" {

Expected Behavior

terraform uploads the file from the passed source into the bucket

Actual Behavior

It fails to upload if the file is larger than ~16MiB

Steps to Reproduce

1. dd if=/dev/zero of=example.data bs=1M count=20
2. terraform apply

[edwardmedia]

@slevenick I can repro this issue. Small objects are fine for upload, but larger ones are not. Bulow is what I got

[DEBUG] google_storage_bucket_object.exampledata: applying the planned Create change
[DEBUG] Retry Transport: starting RoundTrip re
try loop
[DEBUG] Retry Transport: request attempt 0
[DEBUG] plugin.terraform-provider-google_v3.14.0_x5: 2020/03/25 15:50:48 [DEBUG] Retry Transport: Returning after 0 attempts
[DEBUG] google_storage_bucket_object.exampledata: apply errored, but we're indicating that via the Error pointer rather than returning it: Error uploading object example.data: Post https://www.googleapis.com/upload/storage/v1/b/foobar-images-test/o?alt=json&name=examp
le.data&prettyPrint=false&uploadType=resumable: unable to copy invalid http.Request for retry: invalid HTTP request for transport, expected request.GetBody for non-empty Body
ERROR] <root>: eval: *terraform.EvalApplyPost, err: Error uploading object example.data: Post https://www.googleapis.com/upload/storage/v1/b/foobar-images-test/o?alt=json&name=example.data&prettyPrint=false&uploadType=resumable: unable to copy invalid http.Request for retry: invalid HTTP request for transport, expected request.GetBody for non-empty Body
[ERROR] <root>: eval: *terraform.EvalSequence, err: Error uploading object example.data: Post https://www.googleapis.com/upload/storage/v1/b/foobar-images-test/o?alt=json&name=example.data&prettyPrint=false&uploadType=resumable: unable to copy invalid http.Request for retry: invalid HTTP request for transport, expected request.GetBody for non-empty Body

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 hashibot-feedback@hashicorp.com. Thanks!