Terraform behaves differently depending on module source

[bitte-ein-bit]

Hi there,

I'm trying to have my modules gzip+tar-bundled to distribute them properly. I do not want to give customers access to git. There are some python files in the module and these are in a subdir which is ommited via archive download.

Terraform Version

terraform 0.9.11 on Mac OS, code seems to be the same in master.

Terraform Configuration Files

module "lambda-route53-backup" {
  source 		= "https://s3-eu-west-1.amazonaws.com/my-bucket/modules/tf-module-lambda-route53-backup-v0.8-1-gf356c74.tar.gz"
}

Debug Output

https://gist.github.com/bitte-ein-bit/c119a5b00aac97782fba599a42fdaf3d

Expected Behaviour

Module should work the same, irrelevant if git cloned or archive download.

Actual Behaviour

archive download is missing subdirectory files.

Steps to Reproduce

1. rm -rf .terraform/modules/ && terraform get

Important Factoids

Module layout:

$ gtar -tvf tf-module-lambda-route53-backup-v0.8.tar.gz
-rw-r--r-- jenkins/jenkins 3317 2017-08-04 12:02 README.md
-rw-r--r-- jenkins/jenkins 2554 2017-07-31 17:50 backup_local.tf
-rw-r--r-- jenkins/jenkins  812 2017-07-31 17:50 backup_remote.tf
-rw-r--r-- jenkins/jenkins 4528 2017-07-31 17:50 files/backup_route53.py
-rw-r--r-- jenkins/jenkins 2887 2017-08-04 10:56 main.tf
-rw-r--r-- jenkins/jenkins   86 2017-07-31 17:50 outputs.tf
-rw-r--r-- jenkins/jenkins 1694 2017-07-31 17:50 variables.tf
-rw-r--r-- jenkins/jenkins   22 2017-08-04 12:02 VERSION.json
-rw-r--r-- jenkins/jenkins    5 2017-08-04 12:02 VERSION

[jbardin]

Hi @bitte-ein-bit,

Thanks for filing the issue. The tar file you have is a bit odd, because it is missing the files/ directory entry before files/backup_route53.py.

Some tar implementations (gnu and bsd at least) accept this and will create the directory implicitly. The package used by terraform to fetch and unpack the tarball however doesn't.

The immediate workaround is to create the tar file so that there is a directory entry for files/.

[bitte-ein-bit]

Hi @jbardin,

thanks for the problem analysation. Your workaround does indeed work but the directory has to come before the file.

Works:
tar --no-recursion -czf tf-module-lambda-route53-backup-v0.8-2-g098192f.tar.gz files README.md backup_local.tf backup_remote.tf files/backup_route53.py main.tf outputs.tf variables.tf VERSION.json VERSION

Fails:
tar --no-recursion -czf tf-module-lambda-route53-backup-v0.8-2-g098192f.tar.gz README.md backup_local.tf backup_remote.tf files/backup_route53.py main.tf outputs.tf files variables.tf VERSION.json VERSION

[jbardin]

Hi @bitte-ein-bit,

Yes, that's because a tar file (Tape ARchive) is a linear set of file records, so the directory needs to be created before the file can be placed in there.

I'll file an issue upstream for this.

[jbardin]

Closed by 16023

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.