Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

source_dir property of archive_file data source doesn't work with ${path.module} and "../" #5

Open
hashibot opened this issue Jun 13, 2017 · 11 comments

Comments

@hashibot
Copy link

This issue was originally opened by @philipl as hashicorp/terraform#12929. It was migrated here as part of the provider split. The original body of the issue is below.


Terraform Version

terraform 0.9.1

Affected Resource(s)

  • archive_file

Terraform Configuration Files

data "archive_file" "function" {
    type = "zip"
    source_dir = "${path.module}/../src"
    output_path = "${path.root}/somefile.zip"
}

Debug Output

2017/03/21 09:25:52 [DEBUG] root.cloudfront: eval: *terraform.EvalReadDataApply
o:data.archive_file.function: Refreshing state...
2017/03/21 09:25:52 [ERROR] root.cloudfront: eval: *terraform.EvalReadDataApply, err: data.archive_file.function: unexpected EOF
2017/03/21 09:25:52 [ERROR] root.cloudfront: eval: *terraform.EvalSequence, err: data.archive_file.function: unexpected EOF
2017/03/21 09:25:52 [TRACE] [walkRefresh] Exiting eval tree: module.cloudfront.data.archive_file.function
2017/03/21 09:25:52 [DEBUG] plugin: terraform: panic: runtime error: invalid memory address or nil pointer dereference
2017/03/21 09:25:52 [DEBUG] plugin: terraform: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0xb02682]
2017/03/21 09:25:52 [DEBUG] plugin: terraform: 
2017/03/21 09:25:52 [DEBUG] plugin: terraform: goroutine 43 [running]:
2017/03/21 09:25:52 [DEBUG] plugin: terraform: github.com/hashicorp/terraform/builtin/providers/archive.(*ZipArchiver).ArchiveDir.func1(0xc4204f7580, 0x7c, 0x0, 0x0, 0x73e88a0, 0xc420509ad0, 0x0, 0x0)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/gopath/src/github.com/hashicorp/terraform/builtin/providers/archive/zip_archiver.go:65 +0x52
2017/03/21 09:25:52 [DEBUG] plugin: terraform: path/filepath.walk(0xc420582090, 0x8a, 0x74067a0, 0xc4203188f0, 0xc42015a400, 0x0, 0x20)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/go/src/path/filepath/path.go:372 +0x2fe
2017/03/21 09:25:52 [DEBUG] plugin: terraform: path/filepath.Walk(0xc420582090, 0x8a, 0xc42015a400, 0xc420318820, 0x0)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/go/src/path/filepath/path.go:398 +0x14c
2017/03/21 09:25:52 [DEBUG] plugin: terraform: github.com/hashicorp/terraform/builtin/providers/archive.(*ZipArchiver).ArchiveDir(0xc42015a360, 0xc420582090, 0x8a, 0x0, 0x0)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/gopath/src/github.com/hashicorp/terraform/builtin/providers/archive/zip_archiver.go:85 +0x117
2017/03/21 09:25:52 [DEBUG] plugin: terraform: github.com/hashicorp/terraform/builtin/providers/archive.archive(0xc4202da310, 0x5f, 0x74067a0)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/gopath/src/github.com/hashicorp/terraform/builtin/providers/archive/data_source_archive_file.go:158 +0x18d
2017/03/21 09:25:52 [DEBUG] plugin: terraform: github.com/hashicorp/terraform/builtin/providers/archive.dataSourceFileRead(0xc4202da310, 0x0, 0x0, 0xc420238801, 0x0)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/gopath/src/github.com/hashicorp/terraform/builtin/providers/archive/data_source_archive_file.go:123 +0xbf
2017/03/21 09:25:52 [DEBUG] plugin: terraform: github.com/hashicorp/terraform/helper/schema.(*Resource).ReadDataApply(0xc4204fac00, 0xc42015a180, 0x0, 0x0, 0xc42036e978, 0xc420509901, 0x0)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/gopath/src/github.com/hashicorp/terraform/helper/schema/resource.go:252 +0xbb
2017/03/21 09:25:52 [DEBUG] plugin: terraform: github.com/hashicorp/terraform/helper/schema.(*Provider).ReadDataApply(0xc4202d2930, 0xc4202494f0, 0xc42015a180, 0x7fcbfda7d960, 0x0, 0x0)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/gopath/src/github.com/hashicorp/terraform/helper/schema/provider.go:381 +0x91
2017/03/21 09:25:52 [DEBUG] plugin: terraform: github.com/hashicorp/terraform/plugin.(*ResourceProviderServer).ReadDataApply(0xc4202b59e0, 0xc42035a440, 0xc42035a820, 0x0, 0x0)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/gopath/src/github.com/hashicorp/terraform/plugin/resource_provider.go:565 +0x4e
2017/03/21 09:25:52 [DEBUG] plugin: terraform: reflect.Value.call(0xc4202fa2a0, 0xc420300180, 0x13, 0x4c5783b, 0x4, 0xc42062df20, 0x3, 0x3, 0x4de3300, 0xc42002aee8, ...)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/go/src/reflect/value.go:434 +0x91f
2017/03/21 09:25:52 [DEBUG] plugin: terraform: reflect.Value.Call(0xc4202fa2a0, 0xc420300180, 0x13, 0xc42002af20, 0x3, 0x3, 0xc42002af0c, 0x180001, 0x300000000)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/go/src/reflect/value.go:302 +0xa4
2017/03/21 09:25:52 [DEBUG] plugin: terraform: net/rpc.(*service).call(0xc420429a80, 0xc420429a00, 0xc42037fab8, 0xc4204f6300, 0xc420411360, 0x3ee7d00, 0xc42035a440, 0x16, 0x3ee7d40, 0xc42035a820, ...)
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/go/src/net/rpc/server.go:387 +0x144
2017/03/21 09:25:52 [DEBUG] plugin: terraform: created by net/rpc.(*Server).ServeCodec
2017/03/21 09:25:52 [DEBUG] plugin: terraform:  /opt/go/src/net/rpc/server.go:481 +0x404
2017/03/21 09:25:52 [DEBUG] plugin: terraform: 
2017/03/21 09:25:52 [DEBUG] plugin: /home/philipl/.local/bin/terraform: plugin process exited

Panic Log

https://gist.github.com/philipl/e5108167c11c17f2b96564e81e34e0fc

Expected Behavior

zip file is created with contents of directory

Actual Behavior

empty zip file is created and crash is reported in debug output

Steps to Reproduce

  1. terraform plan

Important Factoids

This seems a very specific failure. The following scenarios work fine:

  1. source_dir = "${path.module}/src"
  2. source_dir = "/some/path/../src"
  3. source_file = "${path.module}/../src/filename.js"

The specific failure is when ${path.module} is combined with "../"

@hashibot hashibot added the bug label Jun 13, 2017
@brikis98
Copy link

brikis98 commented Aug 4, 2017

I think I just hit the same thing, except the error I'm getting is:

data.archive_file.source_code: unexpected EOF

@thyrion
Copy link

thyrion commented Nov 8, 2017

Still happens with terraform v0.10.7 and archive 1.0.0 provider

@reg0l
Copy link

reg0l commented Mar 8, 2018

Still happening with Terraform v0.11.3

  • provider.archive v1.0.0

Except i get an

unexpected EOF

@katbyte
Copy link
Contributor

katbyte commented May 25, 2018

Can confirm this is still a problem with:

Terraform v0.11.7
+ provider.archive v1.0.3

panic:

panic: runtime error: invalid memory address or nil pointer dereference
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x15af6f2]
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4:
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: goroutine 25 [running]:
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: github.com/terraform-providers/terraform-provider-archive/archive.(*ZipArchiver).ArchiveDir.func1(0xc420206040, 0x3b, 0x0, 0x0, 0x1aeb9a0, 0xc420212360, 0x0, 0x0)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-archive/archive/zip_archiver.go:86 +0x52
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: path/filepath.walk(0xc4202143c0, 0x5a, 0x1af3ca0, 0xc42029e410, 0xc4202086a0, 0x0, 0x20)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/usr/local/go/src/path/filepath/path.go:377 +0x2eb
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: path/filepath.Walk(0xc4202143c0, 0x5a, 0xc4202086a0, 0xc42029e340, 0x0)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/usr/local/go/src/path/filepath/path.go:403 +0x11d
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: github.com/terraform-providers/terraform-provider-archive/archive.(*ZipArchiver).ArchiveDir(0xc420208660, 0xc4202143c0, 0x5a, 0x0, 0x0)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-archive/archive/zip_archiver.go:85 +0x108
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: github.com/terraform-providers/terraform-provider-archive/archive.archive(0xc420294380, 0x21, 0x1af3ca0)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-archive/archive/data_source_archive_file.go:158 +0x18d
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: github.com/terraform-providers/terraform-provider-archive/archive.dataSourceFileRead(0xc420294380, 0x0, 0x0, 0xc42029c1c0, 0xc)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-archive/archive/data_source_archive_file.go:123 +0xb2
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: github.com/terraform-providers/terraform-provider-archive/vendor/github.com/hashicorp/terraform/helper/schema.(*Resource).ReadDataApply(0xc4202ca720, 0xc420208520, 0x0, 0x0, 0xc4202decb8, 0xc420212201, 0x80000000018)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-archive/vendor/github.com/hashicorp/terraform/helper/schema/resource.go:259 +0xb0
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: github.com/terraform-providers/terraform-provider-archive/vendor/github.com/hashicorp/terraform/helper/schema.(*Provider).ReadDataApply(0xc4202c0230, 0xc42020e050, 0xc420208520, 0x1cb8d90, 0x0, 0x18)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-archive/vendor/github.com/hashicorp/terraform/helper/schema/provider.go:398 +0x9a
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: github.com/terraform-providers/terraform-provider-archive/vendor/github.com/hashicorp/terraform/plugin.(*ResourceProviderServer).ReadDataApply(0xc4202bc600, 0xc4202920f0, 0xc420292260, 0x0, 0x0)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/opt/teamcity-agent/work/222ea50a1b4f75f4/src/github.com/terraform-providers/terraform-provider-archive/vendor/github.com/hashicorp/terraform/plugin/resource_provider.go:565 +0x4e
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: reflect.Value.call(0xc420246d80, 0xc420264998, 0x13, 0x1717f5c, 0x4, 0xc420241f20, 0x3, 0x3, 0x0, 0x0, ...)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/usr/local/go/src/reflect/value.go:434 +0x905
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: reflect.Value.Call(0xc420246d80, 0xc420264998, 0x13, 0xc420254f20, 0x3, 0x3, 0x1148bfa, 0x1, 0x0)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/usr/local/go/src/reflect/value.go:302 +0xa4
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: net/rpc.(*service).call(0xc420229940, 0xc4202629b0, 0xc42023b670, 0xc42027b380, 0xc42024ff80, 0x15fbb60, 0xc4202920f0, 0x16, 0x15fbba0, 0xc420292260, ...)
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/usr/local/go/src/net/rpc/server.go:381 +0x142
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: created by net/rpc.(*Server).ServeCodec
2018-05-24T21:57:25.689-0700 [DEBUG] plugin.terraform-provider-archive_v1.0.3_x4: 	/usr/local/go/src/net/rpc/server.go:475 +0x36b
2018/05/24 21:57:25 [ERROR] root.test: eval: *terraform.EvalReadDataApply, err: data.archive_file.function: unexpected EOF
2018/05/24 21:57:25 [ERROR] root.test: eval: *terraform.EvalSequence, err: data.archive_file.function: unexpected EOF
2018/05/24 21:57:25 [TRACE] [walkRefresh] Exiting eval tree: module.test.data.archive_file.function
2018/05/24 21:57:25 [TRACE] dag/walk: upstream errored, not walking "provider.archive (close)"
2018/05/24 21:57:25 [DEBUG] plugin: waiting for all plugin processes to complete...
2018-05-24T21:57:25.691-0700 [WARN ] plugin: error closing client during Kill: err="connection is shut down"
2018-05-24T21:57:25.691-0700 [DEBUG] plugin: plugin process exited: path=/Users/kt/hashi/2018/gh/archive-5/.terraform/plugins/darwin_amd64/terraform-provider-archive_v1.0.3_x4

@katbyte
Copy link
Contributor

katbyte commented May 25, 2018

Hi @reg0l, @thyrion, @brikis98, @philipl,

Thank you for opening and commenting on this issue. I have fixed the panic (#26) and can now see what is actually happening here:

[23:21:56] kt@snowbook:~/hashi/2018/gh/archive-5$ cat main.tf
module "test" {
  source  = "m"
}
[23:22:00] kt@snowbook:~/hashi/2018/gh/archive-5$ cat m/m.tf
data "archive_file" "function" {
    type = "zip"
    source_dir = "${path.module}/../m"
    output_path = "${path.root}/somefile.zip"
}
[23:22:27] kt@snowbook:~/hashi/2018/gh/archive-5$ tfinitautoapply
Initializing modules...
- module.test

[...]

data.archive_file.function: Refreshing state...

Error: Error refreshing state: 1 error(s) occurred:

* module.test.data.archive_file.function: 1 error(s) occurred:

* module.test.data.archive_file.function: data.archive_file.function: error archiving directory: error encountered during file walk: lstat /Users/kt/hashi/2018/gh/archive-5/.terraform/modules/m/m.tf: no such file or directory


[23:22:49] kt@snowbook:~/hashi/2018/gh/archive-5$

It seems ${path.module}../path is confusing terraform because terraform is actually running the module out of a symlink in the .terraform directory:

[23:30:17] kt@snowbook:~/hashi/2018/gh/archive-5/.terraform/modules$ l
total 8
lrwxr-xr-x  1 kt  staff    35B 24 May 22:59 d8e8c5c9d3f2ba57af0e123ff8f18055 -> /Users/kt/hashi/2018/gh/archive-5/m

I've run out of time to spend on this tonight but at least now there will be a reasonable error 🙂

@swoodford
Copy link

swoodford commented Jun 7, 2018

Just ran into this problem:

Terraform code:

data "archive_file" "geodetection" {
  type        = "zip"
  source_dir  = "${path.module}/../../../../dev/lambda-edge-geodetection"
  output_path = "${path.module}/lambda-edge-geodetection.zip"
}

Apply result:

Error: Error refreshing state: 1 error(s) occurred:

* module.edgelambda.data.archive_file.geodetection: 1 error(s) occurred:

* module.edgelambda.data.archive_file.geodetection: data.archive_file.geodetection: unexpected EOF

Had to change to this and it worked...

source_dir  = "${path.root}/../../../dev/lambda-edge-geodetection"

@dumrauf
Copy link

dumrauf commented Jun 20, 2018

I can confirm that @swoodford's solution of replacing ${path.module} with ${path.root} works. It's actually a better solution for my problem but I am wondering if

  1. that's intended behaviour of ${path.root} and what's causing it to act differently than ${path.module}
  2. there are any plans to fix the ${path.module}/../../../this-will-not-work/ problem

Other than that, the archive_file provider is amazing! Keep up the great work!

@rolandjohann
Copy link

came across this issue as well

@codeinaire
Copy link

I've tried both ${path.module} and ${path.root} and neither work for me. I'm able to use ../../apollo-sequelize-terra-test/lambda in the terminal from my terraform's root dir to get to the folder that I want.

When I add ${path.module} or ${path.root} in front of that I get the could not archive missing directory error.

Anyone have a solution?

@mtranter
Copy link

I'm seeing the same as @codeinaire
Neither ${path.module} nor ${path.root} are working.

@ricoms
Copy link

ricoms commented Dec 5, 2019

Terraform v0.12.17 and this is happening, my terminal just hangs at

data.archive_file.deployment_package: Refreshing state...

------------------------------------------------------------------------

no error message stayed like that for 5 minutes

The archive_file is a single python script with 11 lines.
And I tried with and without ${path.module} defined like this:

data "archive_file" "deployment_package" {
  type = "zip"
  source_file = "function.py"
  output_path = "function.zip"
}

I would not maintain #8344 (hashicorp/terraform#8344) closed, although I do not have access.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests