indexer: Ensure declared module calls get decoded

[radeksimko]

Related: hashicorp/terraform-schema#254

The PR is not strictly hard dependency for this one but it is helpful in testing, to verify that the race condition is no longer present.

---

Context

Some time ago, we discovered a race condition, which may not have been as well understood at the time. We needed to make sure that go-to-definition and go-to-references works on first load for module inputs.

Module inputs are represented as reference origins and as with everything else, the attribute schema is what instructs the job collecting the reference origins how and whether to collect those origins. Obtaining the schema for modules is basically the job of LoadModuleMetadata - in particular we generate the schema based on parsed variables & outputs.

Prior to this PR, we never processed "module calls" directly from didOpen or didChange and only relied on the walker to get to them by the time we needed them. This would usually work fine if no documents were open at the time of initial walking. Often times though the walking may take a lot of time for larger workspaces and users may open files very early. This meant submodule data was unavailable when the user opened the module which calls those submodules, making the reference origin collection job "clue-less" about those module inputs.

We worked around that race condition by inferring the inputs - basically assuming all declared inputs have their corresponding variable declarations. As with most workarounds, this eventually caught up with us 😅 . In particular, validation (as implemented in #1368) was also acting based on this inferred (potentially incorrect) schema and worse - the inferred schema was assumed to be relevant for all modules with the same source.

Therefore, hashicorp/terraform-schema#254 removes the workaround and this PR addresses the race condition.

Technically this may result in more jobs getting scheduled but the overall performance impact should be relatively minimal because:

• it only impacts open files/modules and the assumption is that users generally don't open all modules that are in their workspace at the same time
• the jobs exit early if the module data is already available

Testing

I am frankly not sure how to best test this (in code) as the problem is usually only visible with high enough number of jobs / large workspaces.

I did test it manually in one of our large repositories and can confirm the "go-to-definition" works cca 10 out of 10 times on continuous reload, whereas previously it would end up broken on every ~5th reload.

[dbanck]

👍 Nice work. I see only a very, very small performance impact, which is definitely negligible.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.