Display module versions in a similar fashion that provider versions are displayed with `version`

[grimm26]

Community note

Tip

👋 Hi there, OpenTofu community! The OpenTofu team prioritizes issues based on upvotes. Please make sure to upvote this issue and describe how it affects you in detail in the comments to show your support.

OpenTofu Version

OpenTofu v1.10.6
on linux_amd64

The problem in your OpenTofu project

I'd like to be able to see what versions of modules in a unit have been downloaded. Not all modules will have an associated version since a module source can be one of a few things that do not have a version (at least that tofu knows about).
This is probably only appropriate for modules sourced from a module registry and I see that the version is indeed recorded for these in .terraform/modules/modules.json after an init. This is important information in any instance where anything but an exact version for a module is specified.

Attempted Solutions

I could not find any tofu command that would show me the version of a module that has been downloaded to be used in a plan/apply. My only solution at this point is to consume .terraform/modules/modules.json with a custom tool.

Proposal

Today, when I run tofu version in a project, I see something like:

❯ tofu version
OpenTofu v1.10.6
on linux_amd64
+ provider registry.opentofu.org/hashicorp/aws v6.13.0

I'd like to see module version info as well:

❯ tofu version
OpenTofu v1.10.6
on linux_amd64
+ provider registry.opentofu.org/hashicorp/aws v6.13.0
+ module myregistry.foo.com/namespace/iam/aws v1.4.2

And similarly in the json output in a module_selections hash.

{
  "terraform_version": "1.10.6",
  "platform": "linux_amd64",
  "provider_selections": {
    "registry.opentofu.org/hashicorp/aws": "5.100.0"
  }
  "module_selections": {
    "myregistry.foo.com/namespace/iam/aws": "1.4.2"
  }
}

Workarounds and Alternatives

the workaround is to write a custom tool to ingest the .terraform/modules/modules.json file and output something to record along with the plan/apply or otherwise use the info.

References

No response

[apparentlymart]

Thanks for this feature request! The core maintainers will discuss it soon.

The list of provider versions in tofu versions is essentially just a subset of the information from the .terraform.lock.hcl file, included there mainly so that it'll appear when someone shares the version information as part of a bug report, or similar. I expect we could do similarly with the information in the modules JSON file since mechanically that is more or less another "lock file" that just isn't kept under version control.

This does mean that the information would be included only when the working directory has already been initialized with tofu init, though. That's a little different than the provider listing which works as long as a lock file is present, regardless of whether the lock file was updated in the current working directory.

[grimm26]

@apparentlymart my main goal is to have the module versions recorded when going through init -> plan -> apply cycle in our pipeline. Currently, we record the output of tofu version right before a plan and I'd like to add modules to that as well. If one has module source set to anything but an exact version, it's hard to say what module version was used.

[apparentlymart]

One additional detail I forgot to mention in my first comment:

While OpenTofu requires that each provider have a single selected version across the whole configuration, for modules we allow each call to select a different module package and so following exactly the presentation we use for providers probably wouldn't be adequate.

We could potentially show a separate entry for each module call in the configuration though, such as:

$ tofu version
OpenTofu v1.10.6
on linux_amd64
+ provider registry.opentofu.org/hashicorp/aws v6.13.0
+ module.iam myregistry.foo.com/namespace/iam/aws v1.4.2
+ module.foo git::git.example.com/foo.git?ref=v1.0.0
+ module.foo.module.bar git::git.example.com/foo.git//modules/bar?ref=v1.0.0

(I showed a git source address with a version-shaped tag name as its ref argument here because that's the main case where I expect including those would be useful, but I'm imagining that this would also include source addresses that don't select something that's typically immutable and then leave it to the consumer of this information to infer whether a particular package address seems like it should be immutable or not.)

Another option would be to just show all of the distinct module package address and version pairs, so that e.g. if two calls both select the same address and version they would be coalesced into a single item in the tofu version output. That means it wouldn't be possible to see which call each of the selections relates to, but perhaps that's okay if the goal is only to know what dependencies were included and not exactly how they were being used.

$ tofu version
OpenTofu v1.10.6
on linux_amd64
+ provider registry.opentofu.org/hashicorp/aws v6.13.0
+ module package myregistry.foo.com/namespace/iam/aws v1.3.0
+ module package myregistry.foo.com/namespace/iam/aws v1.4.2
+ module package git::git.example.com/foo.git?ref=v1.0.0

[grimm26]

@apparentlymart I like the first version so I can see what lines up with what. For my purposes, 90% of the time, only on version of a module is used per configuration unit, but seeing that explicitly is best.

[lbordowitz]

Technically, each instantiation of a module (the module block in the configuration) could use a different version. It makes the most sense to list the address of each module and the corresponding version for it.

Module-heavy workspaces might flood the version output; to that end, outputting the module versions should be behind a flag -show-modules.

[apparentlymart]

While I understand the concern about the output getting more heavy, I'm also nervous about making tofu version more complicated by adding more different modes to it: we already have both the default "human-readable" mode and the JSON mode, and presumably -show-modules would be orthogonal to the output format and so we'd then have a total of four modes with all of the combinations of those two flags.

I'd suggest that we include module-related information by default and wait to see if it causes anyone enough of a problem to justify the complexity of offering an opt-out of that behavior. tofu version is not really part of the "primary workflow" and so it printing too much information sometimes seems like mainly a cosmetic issue. 🤔

[apparentlymart]

The core maintainers discussed this today and we've accepted the overall idea of it but still need to finalize some design details before anyone should begin working on implementing this:

1. Should we show information about each module call individually, or aggregate the information by distinct package address and version pairs?

   (This is the same question I asked in Display module versions in a similar fashion that provider versions are displayed with version #3270 (comment), and which we've already been discussing above; I've just repeated here to have everything together in one list.)

2. If we do show separate information for each call, should the human-readable output for it appear as a flat list or as a tree structure?

   We already have some precedent in the tofu providers command for showing modules in an tree format and so we could potentially copy that here although the specific symbols that command uses differ from the + "bullets" that tofu version currently uses.

3. How should we present this data in the JSON format?

   There's a similar question about whether to represent it as a tree structure of nested JSON objects, but also generally a question of how to represent each individual selection since the existing provider_selections only has to capture provider address and version whereas this new section potentially needs to represent the module call path, the module source address, and the selected version.

4. Should this list be limited only to registry-based module sources, or should it include all source address types?

   Currently only registry-based sources actually have a real concept of "version selections", but we could still show the source address alone (without a separate version number) for other types, as I was sketching in Display module versions in a similar fashion that provider versions are displayed with version #3270 (comment).

   There's also a question of what to do about local source addresses: the .terraform/modules/modules.json file currently captures those relative paths essentially just as they were written in the configuration, meaning that they're only useful if you know the source address of the nearest ancestor that used a non-local path. Coincidentally as part of prototyping something else recently I wrote a function for resolving one source address relative to another, and so we could potentially use something like that to show the effective absolute source address for each module even when some are chosen using relative paths.

5. Should we show module information by default? Should it be possible to opt in or opt out of it? Are we willing to start with it always showing this information and then add an opt-out only if later feedback suggests it would be useful?

   (This was originally raised in Display module versions in a similar fashion that provider versions are displayed with version #3270 (comment) and again I'm including it here just to have all of the questions in a single list.)

I don't really have super strong opinions about any of the above. It seems like so far consensus on the first question has been leaning towards having a separate entry for each module call instead of trying to reduce it to just a set of distinct addresses, which is fine by me since it's the most direct interpretation of the source data we'd be rendering from anyway.

I'd say that if we want to show individual module calls then showing them in a tree format is probably the best presentation, rather than using increasingly long module.this.module.that prefixes when module calls are nested, but that is a weakly-held opinion. I think the JSON probably should be arranged as a tree of JSON objects so that consumers that want a tree aren't forced to "un-flatten" the tree themselves.

I also like the idea of resolving local source addresses into their absolute (or at least root-module-relative) paths before we display them so that the consumer of this data doesn't need to implement their own version of that relative source address resolution logic in order to find the full set of remote or registry source addresses.

I'm curious to hear if anyone else has stronger opinions on any of these questions, particularly if they disagree with my loosely-held ones. 😀