Feature Request: Module versioning for git sources

[jgiles]

This request is a follow-up to #15613, which was closed after versioning was added for module registry modules.

It would be extremely useful to use semver constraints on modules across our terraform codebase:

module "foo" {
  source  = "github.com/hashicorp/example"
  version = "^0.1.1"
}

This would save users from widespread source code updates just to pick up patch fixes.

[rayterrill]

Would love this feature.

[rarkins]

@jgiles does a single version work today, e.g.

module "foo" {
  source  = "github.com/hashicorp/example"
  version = "0.1.1"
}

?

Or would that need to be done via ref somehow? e.g.

module "foo" {
  source  = "github.com/hashicorp/example?ref=0.1.1"
}

[lorengordon]

Also looking forward to something like this, though I'd personally prefer interpolation for the entire source parameter rather than just separating the version/ref.

[rayterrill]

We're successfully using the ref + release functionality with Github sources today, it just seems a little kludgy that it works one way for Terraform registry and another way for Github, when they're both effectively providing similar functionality.

[jgiles]

@rarkins I believe you need to do the ref= thing currently, it's what we do all over the place.

We actually have a hacky mechanism where all our TF repo release tags are like v1.2.3-repo where in place of repo we put a short version of the repo name. This is purely to facilitate simpler find/replace of the tagged versions.

[rarkins]

@jgiles thanks for the follow-up details. I added Terraform version upgrading to Renovate Bot just a couple of days ago actually, using the logic to look for semver-like ref tags like you mention. It would also work with your "hack" tags with one small config change, however an automated approach would save you from manual search/replace so you could do away with the hack instead if you preferred.

[oswalya]

any chance to see this in future releases?

[dsvua]

I guess there is conflict of interest with Terraform Enterprise's private module registry, which is paid product.

[mrsaraiva]

Looks like Terraform 0.12 broke the git ref trick for module versioning, so it would be really nice to either have it back or it actually implementing what's proposed here.

[apparentlymart]

The ref argument is still functional in 0.12, though we saw in another issue that the behavior of the upstream library we use for non-registry sources seems to have subtly changed in how it understands the //subdir syntax.

If you were using something like git::ssh://example.com/bar.git?ref=foo//subdir with the intent of selecting the subdir directory within the repository, the correct way to write that is with the subdirectory part on the path portion of the URL, like this:

git::ssh://example.com/bar.git//subdir?ref=foo

If you place it on the end, Terraform understands it instead as an attempt to use a branch called foo//subdir.

If the above doesn't explain a change in behavior you've seen, please open a new issue for it... there were no other changes intended here, so if there have been other regressions then we'll get them fixed up, but best to keep bug reports separate from feature requests so that we can track and prioritize them better.

[mrsaraiva]

@apparentlymart that was exactly the issue I was facing and I thought it was something related to specifying a ref, but it was indeed the path syntax. Thanks!

[johnnyhuy]

Any chance we can get this feature? So that we can use the expressions (>=, ~> etc.) that come out of the box to manage our SemVer tags.

[nokernel]

We have been reqesting this since 2015

#1439
#15613

[stephanebruckert]

I wrote about how we handled this situation at DAZN https://medium.com/dazn-tech/terraform-module-versioning-for-git-sources-5a792ceb74d7

Basically we use our CI (Github Action) to override major and minor tags, so that we can specify those directly, for example:

module "ecs-service" {
    source = "github.com/getndazn/terraform-module-ecs?ref=v1"

Bit hacky but solves the problem!

[flickerfly]

@stephanebruckert That doesn't provide the semver constraint, just pins to a specific ref.

I understand that to mean the inclusion of a comparator.

A comparator is composed of an operator and a version. The set of primitive operators is:

-< Less than
-<= Less than or equal to
-> Greater than
->= Greater than or equal to
-= Equal. If no operator is specified, then equality is assumed, so this operator is optional, but MAY be included.

https://github.com/npm/node-semver#ranges

[stephanebruckert]

You are right that solution doesn't handle operations, it's more of a quick win to profit from majors and minors.