Init Command Cannot Parse/Download Valid Git Tags

[duffenterprises]

Terraform Version

v0.13.3 or greater

Terraform Configuration Files

module "my-module" {
  source          = "git::ssh://git@github.com/my-org/my-module.git?ref=v0.3.0+tf.0.13.3"
  project_id      = var.project_id
  region          = var.region
}

Debug Output

Error: Failed to download module

Could not download module "firestore" (main.tf:1) source code from
"git::ssh://git@github.com/my-org/my-module.git?ref=v0.3.0+tf.0.13.3":
error downloading
'ssh://git@github.com/tmy-org/my-module.git?ref=v0.3.0+tf.0.13.3':
/usr/bin/git exited with 1: error: pathspec 'v0.3.0 tf.0.13.3' did not match
any file(s) known to git.

Error: Failed to download module

Crash Output

NA

Expected Behavior

Module should be downloaded

Actual Behavior

See Debug Output

Steps to Reproduce

1. Create a terraform reference to a module with a reference including a 'reserved' character such as '+'
2. terraform init

Additional Context

We are attempted to use semver and want to use the metadata section following + to describe when we created this module (e.g.. what TF version it was first introduced against). The file getter.go in terraform/internal/initwd/getter.go appears to be responsible for realAddr before the download is attempted:

else {
  log.Printf("[TRACE] fetching %q to %q", realAddr, instPath)
  client := getter.Client{
    Src: realAddr,
    Dst: instPath,
    Pwd: instPath,

    Mode: getter.ClientModeDir,
    Detectors:     goGetterNoDetectors, // we already did detection above
    Decompressors: goGetterDecompressors,
    Getters:       goGetterGetters,
  }

A simple realAddr = strings.Replace(realAddr, "+", "%2B", -1) allows us to get by the issue, but doesn't seem the correct way to address this.

We could ask users to to encode the path our our module like source = "git::ssh://git@github.com/my-org/my-module.git?ref=v0.3.0%2Btf.0.13.3 but anticipate this will lead to confusion and delay.

References

NA

[apparentlymart]

Hi @duffenterprises,

I understand that this was confusing, but I'm not sure that there's a great answer here given that the version number is being represented using the typical URL query string syntax and is therefore subject to the encoding conventions of that syntax. Technically Terraform (delegating to go-getter) is working as designed here: it's decoding the query string of the URL in the standard way to get the values from it. To diverge from the typical encoding of query strings here would then make it surprising in a different way.

Because Terraform is behaving as designed but the current design has a limitation as described here, I'm going to relabel this as an enhancement and leave it open so we can think about potential changes to Terraform's design that might improve this feature.

---

I expect that this will not be a very satisfying answer due to it involving the introduction of an entirely new service, but one advantage of publishing modules using Terraform's native module registry protocol is that it's able to abstract away some of these implementation details. If the module shown in the example were indexed in a registry then the call to it in the configuration would be more straightforward, due to moving the complexity out of the configuration and into the registry server:

module "my-module" {
  source  = "tf.example.com/my-org/my-module"
  version = "0.3.0+tf.0.13.3"

  project_id = var.project_id
  region     = var.region
}

The registry protocol is, in effect, just an indirection for externalizing the construction of source strings, so the module content itself can in principle remain in the same Git repository, if the registry is set up to return a git:: source string like the hard-coded one in the original example. (The public Terraform Registry works by constructing URLs pointing to tags in repositories on GitHub, for example.)

[duffenterprises]

@apparentlymart Thanks for the detailed reply. I had really been hoping to provide a PR along with my issue anticipating that it was just going to need URLEncode applied at the appropriate location. That was proven incorrect in my debugging which is why I opened the issue. (It encodes the ? and = )

The only argument that I can make for calling this a 'bug' is that the documentation says:

The value of the ref argument can be any reference that would be accepted by the git checkout command, including branch and tag names.

And a ref like v0.3.0+tf0.13.3 is a valid ref.

In short term, I'll likely use a _ as my meta seperator, and see if I can understand the getter more.

[apparentlymart]

Thanks for the extra info, @duffenterprises. I think that documentation was written in the hope it would be clear from the context that the ref argument containing "any reference that would be accepted" would then need to be URL-encoded to appear in a query string argument, but we could certainly make that more explicit.

[duffenterprises]

Perhaps I'll prepare a PR for the documentation. The portion that I'd still have trouble explaining if I wrote that document would be why only the ref component needs to be encoded.

Encoding:

• "git::ssh://git@github.com/my-org/my-module.git?ref=v0.3.0+tf.0.13.3" to git%3A%3Assh%3A%2F%2Fgit%40github.com%2Fmy-org%2Fmy-module.git%3Fref%3Dv0.3.0%2Btf.0.13.3 Fails.
• "git::ssh://git@github.com/my-org/my-module.git?ref=v0.3.0+tf.0.13.3 togit::ssh://git@github.com/my-org/my-module.git%3Fref%3Dv0.3.0%2Btf.0.13.3 Fails.