Computed map type variable cuts keys short, when key contains dot

[greg-szabo]

Terraform Version

Terraform v0.11.10

Terraform Custom Provider Code

func dataSourceDir() *schema.Resource {
	return &schema.Resource{
		Read: dataSourceDirRead,

		Schema: map[string]*schema.Schema{
			"rendered": &schema.Schema{
				Type:        schema.TypeMap,
				Computed:    true,
				Description: "rendered files",
			},
		},
	}
}

func dataSourceDirRead(d *schema.ResourceData, meta interface{}) error {
	dirMap := make(map[string]string)
	dirMap["a.b"] = "apple"
	d.Set("rendered", dirMap)
	d.SetId(generateHash(dirMap)) // generates hash, not relevant to the issue
	return nil
}

Terraform Config to test custom provider

data "provider_name" "test" {
}

output "mymap" {
  value = "${data.provider_name.test.rendered}"
}

Debug Output

https://gist.github.com/greg-szabo/9a6393bb35b3e041e96a5dea6d648d93
The debug log is from a bigger custom provider. This custom provider implements template_dir data source for the template provider. (I was planning to submit it when the bugs were all fixed.)

Expected Behavior

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

mymap = {
  a.b = apple
}

Actual Behavior

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

mymap = {
  a = map[b:apple]
}

Steps to Reproduce

1. Create a new provider (I used the template provider and extended that but it should happen in all cases.)
2. Compile the provider (with the above code) and compile it
3. Put the provider under terraform.d/<os_arch>/privder_name
4. Create the test.tf
5. terraform init
6. terraform apply

Additional Context

I was happily finalizing my addition to the template provider (a template_dir data source) to allow slurping in a directory of templates (it would be useful for recursive S3 uploads or multiple config files in a directory) when I realized that my tests are failing. The reason: the rendered map (key = filenames with path ; value = rendered template) is providing the keys without the file extensions (filename: monkey.txt -> rendered["monkey"], instead of rendered["monkey.txt"])
I tried running the binary against a terraform build from master (0.12.0 dev) but I guess the provider boilerplate changed because it was complaining that Incompatible API version with plugin. Plugin version: 4, Client versions: [5]. I guess the provider libraries will need some updates before 0.12 release. (If I can write a provider against 0.12, please give me some pointers.)

I'm happy to dig into the terraform source, if you give some pointers on where the problem could be. Since I only looked at providers yet, I thought I'll ask before I go through everything.

I use Go 1.11.1 on OSX Mojave.

References

It would help resolve hashicorp/terraform-provider-template/issues/34 , hashicorp/terraform-provider-template/issues/37 and most probably it would give a more robust solution than hashicorp/terraform-provider-template/pull/13

[apparentlymart]

Hi @greg-szabo! Sorry for this weird behavior.

Funnily enough, this has actually already been fixed for v0.12.0 in the master branch, and that is actually the reason why your plugin executable didn't work against a build of Terraform from the master branch: fixing this required changes to the provider protocol, which previously would use periods to separate the elements of paths into complex data structures.

If you upgrade the vendored copy of Terraform in the template provider to latest master and rebuild the provider you should find that it is compatible and that the issue you encountered is fixed. The alpha2 prerelease builds have builds of certain providers (including template) bundled in their zip files for this reason: we need to make sure the new protocol implementation is robust before using it in "real" provider releases.

Since the fix for this is already in master, I'm going to close this out. If you find other weirdness with the provider protocol once you have the latest Terraform master branch code in both the terraform program and the provider, please do open other GitHub issues to cover them! Our current efforts are focused on fixing up the remaining issues with the new protocol implementation in preparation for including it in real provider releases, so any extra testing/feedback is very valuable and much appreciated.

---

Regarding the change to the template provider you're working on: returning all the rendered files in outputs might be quite invasive in terms of state file size and might cause problems for people already using this data source, so once you've finished your initial exploration I'd like to talk this through in a github issue in the template provider repository before moving forward to implementation. The template_dir data source is weird and I think there is definitely room for improvement with it, but we need to be mindful of unintended side-effects for existing users. Thanks for working on this!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.