Terraform 0.12.19+ redirects when using source = "terraform-google-modules/bigquery/google"

[ed-hammond]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

v0.12.19+

Affected Resource(s)

• terraform-google-modules/terraform-google-bigquery

Background

Terraform modules can be referenced in the config files using following syntax:

   module "<module-name>" { 
      source  = "<source>" 
      version = "<version-constraint>" 
      // module attributes 
   }

Reference: https://www.terraform.io/docs/configuration/modules.html

---

Source Specification

Both source and version attributes have flexibility in how they are expressed

• Method Adding the base repo structure for the module, modified testing, adde… #1: source can be expressed simply as a generic path with no qualifying url as in:

source  = "terraform-google-modules/bigquery/google"
version = "~> 4.0"

This means it is a "terraform registry" module which supports a specific discovery/lookup method to find the location of the module to download; the actual download can be from anywhere though typically it’s a github repo. The version attribute represents a constraint telling terraform to restrict versions of the module to at least version 4.0. (The version attribute only works with "terraform registry" modules that use this syntax.) The download happens via HTTP(s) GET request.

• Method Digangi branch #2: source can specified with a specific url (telling terraform where specifically to download the module from):

source = "github.com/terraform-google-modules/terraform-google-bigquery?ref=v4.2.1" 
source = "git::https://github.com/terraform-google-modules/terraform-google-bigquery?ref=v4.2.1" 
source = "github.com/terraform-google-modules/terraform-google-bigquery" 

Using method #2 the version attribute in the module stanza is not supported. If a specific version is needed it must be specified in the ref= query param in the url. (Leaving it off means you will get the latest version - currently 4.2.1).

All the above examples are valid and all result in the same version of the google module that gets downloaded; but in all of these cases the download is via "git" protocol (either https or ssh clone method).

Issue

There a problem using method #1 of referencing modules versus method #2 on certain versions of Terraform.

• With Terraform v0.12.19 thru v0.12.26 (latest version)

  • When we use method Adding the base repo structure for the module, modified testing, adde… #1 with source = "terraform-google-modules/bigquery/google", the terraform discovery process results in a download location of:
    https://github.com/terraform-google-modules/terraform-google-bigquery/archive/v4.2.1.tar.gz?archive=tar.gz
    which results in a redirect to:
    https://codeload.github.com/terraform-google-modules/terraform-google-bigquery/tar.gz/v4.2.1
    which appears to have an invalid SSL certificate which is blocked by certificate validation policies.
  • When we use any choice shown in method Digangi branch #2, there is no redirect and no certificate issue.

• With Terraform versions up to and including v0.12.18 there is no issue with either method Adding the base repo structure for the module, modified testing, adde… #1 or Digangi branch #2; In these versions, the terraform discovery process results in the same "git" download location as when using the method Digangi branch #2.

Conclusion:

Since the end result of the terraform registry lookup method (in recent terraform versions) results in a significantly different result for the location (note the paths to the repo are also different; even though the resulting files are same one is an archive reference) and because the redirect points to a site with invalid cert, we believe this to be a bug with how these modules are configured in the terraform registry and/or a bug in the newer terraform versions.

Epilog

We will use method #2 to reference the modules for now since that seems to work.

[morgante]

I'm unable to replicate this (using v0.12.26) and suspect it's probably caused by an issue in your proxy or network setup.

Anyways, there's nothing for us to do here. The module registry system is maintained by HashiCorp.

[cah-ken-flannery]

@morgante correct, the issue is only reproducible if you are behind a firewall that either blocks codeload variations of github urls and/or not enforcing cert validation. But, just to clarify, are you saying terraform is directing itself to codeload? I was under the impression this was handled by the submitter registry entries? I'm not familiar w/how that process works though.

[morgante]

Correct, we're not referencing codeload. That's a redirect happening either from the registry or GitHub itself, not anything we did. You would see the same behavior for any module from the registry.