Separate checkout directories for each instance in module "for_each"

[ocervell]

Current Terraform Version

Terraform 0.13.2

Use-cases

Currently, the folder path where the module code is checked out is the same for all iterations of the for_each loop: .terraform/modules/<MODULE_NAME>.

This causes problems with configs where a local directory is being used to i.e create archive files, because each instance of the for_each loop will share that local directory. If we're dynamically adding files to this directory they might conflict with each other (and overwrite the existing file).

This will apply to any use case where we add / change files from a path in the module checkout folder.

Attempted Solutions

For the use case discussed above, I tried creating a new temp folder for each instance created but this is incompatible with archive_file resource that will fail on the next read if the temp folder is missing.

Proposal

This issue would be alleviated if we had an option to specify separate checkout directories for each instance of the module.

[apparentlymart]

Hi @ocervell! Thanks for suggesting this.

I can see that the current behavior is inconvenient for modules are built to work with path.module. While we generally advise against having modules write files to disk as part of their work, it's sometimes unavoidable and module count/for_each have indeed created a situation where path.module is not as unique as it used to be.

With that said, I don't think a separate checkout directory for each instance is a viable solution to the problem, because module installation happens during terraform init and not during terraform plan or terraform apply, and so the module installer can't predict how many instances of the module there will be at runtime... the number might even depend on the result of reading a data source, and so evaluating the configuration alone with not be sufficient to decide.

This problem statement reminded me of an old proposal I made, in #21308. I had a different problem statement in mind at the time I created that one, but it's aiming at approximately the same place of giving each module instance a dynamically-assigned directory to use for temporary files in situations where temporary files are unavoidable. If we were to implement something like that today, I expect we'd arrange for path.temp to return a different result in each instance of a module, and thus give each of the instances its own independent work area for temporary files even though all of them would still share the same path.module.

What do you think of that as an alternative way to address the problem you raised here?

[ocervell]

Sounds good to me, support for temp directories should solve this use case if implemented right indeed.

What I am doing with archive_file is:

• Take files from a template folder
• Render them (replace some vars)
• Drop them into a folder (that temp folder)
• Creating the archive if the md5 hashes of one file is different
• Sending the archive to GCS in order to deploy the Cloud Function

Now one thing that worries me is when data archive_file re-runs and does not find that temp directory: I think we should make sure temp folders are created before any resource runs, including data sources.

[apparentlymart]

Hi @ocervell,

I think the requirement for your situation would be to make sure that the archive_file data source is always handled at apply time rather than at plan time, so that the temporary files will still be available on disk by the time the Cloud Function resource is created/updated.

With that said, as I mentioned in the writeup in #21308 the archive_file data source is a badly behaved data source which Terraform therefore cannot handle reliably. If we were to implement path.temp as I described in the other issue then it would likely work okay for your use-case, but in particular every time you run terraform apply it will plan to recreate and re-deploy the package, and therefore the configuration in question will never be able to converge on a stable state.

The archive_file data source was a design error which we continue mainly for backward compatibility, but I would suggest avoiding using it because it is likely to behave poorly in automated execution environments that aren't able to preserve files from the plan phase to the apply phase. Terraform is not intended as a build tool, so it's typically better to build and upload your application artifacts using some other tool and then use Terraform only to "deploy" them. (Waypoint will hopefully grow to be a better answer to this, but it's still early in its life.)