Fix schema path elem traversal

[iwahbe]

Fixes pulumi/pulumi-gcp#1794

This is the second attempt at fixing pulumi/pulumi-gcp#1794, so strict scrutiny would be appreciated.

The core of the issue is correctly interpreting this block:

pulumi-terraform-bridge/pkg/tfshim/shim.go

Lines 102 to 127 in 07b2c1c

	// s.Elem() may return a nil, a Schema value, or a Resource value.
	//
	// The design of Elem() follows Terraform Plugin SDK directly. Case analysis:
	//
	// Case 1: s represents a compound type (s.Type() is one of TypeList, TypeSet or TypeMap), and s.Elem()
	// represents the element of this type as a Schema value. That is, if s ~ List[String] then s.Elem() ~ String.
	//
	// Case 2: s represents a single-nested Terraform block. Logically this is like s having an anonymous object
	// type such as s ~ {"x": Int, "y": String}. In this case s.Type() == TypeMap and s.Elem() is a Resource value.
	// This value is not a real Resource and only implements the Schema field to enable inspecting s.Elem().Schema()
	// to find out the names ("x", "y") and types (Int, String) of the block properties.
	//
	// Case 3: s represents a list or set-nested Terraform block. That is, s ~ List[{"x": Int, "y": String}]. In
	// this case s.Type() is one of TypeList, TypeSet, and s.Elem() is a Resource that encodes the object type
	// similarly to Case 2.
	//
	// Case 4: s.Elem() is nil and s.Type() is a scalar type (none of TypeList, TypeSet, TypeMap).
	//
	// Case 5: s.Elem() is nil but s.Type() is one of TypeList, TypeSet, TypeMap. The element type is unknown.
	//
	// This encoding cannot support map-nested blocks or object types as it would introduce confusion with Case 2,
	// because Map[String, {"x": Int}] and {"x": Int} both have s.Type() = TypeMap and s.Elem() being a Resource.
	// Following the Terraform design, only set and list-nested blocks are supported.
	//
	// See also: https://github.com/hashicorp/terraform-plugin-sdk/blob/main/helper/schema/schema.go#L231
	Elem() interface{}

From what I can tell, both propertyPathToSchemaPathInner and schemaPathToPropertyPathInner confuse case 3 for case 2:

propertyPathToSchemaPathInner and schemaPathToPropertyPathInner correctly handle case 2 ((schema, ps) ~ {x: T}), traversing (schema.Elem().(shim.Resource).Schema(), ps.Fields). This is correct, as the .Elem() in the shim path is a fiction of representation. ps.Fields correctly represents that (schema, ps) is the object ({x: T}) in question.

Currently, both functions incorrectly handle case 3 ((schema, ps) ~ List[{x: T}], (schema.Elem(), ps.Elem) ~ {x: T}), treating it identically to case 2. The correct traversal for the nested object is (schema.Elem().(shim.Resource).Schema(), ps.Elem.Fields) (note the additional .Elem after ps). This PR makes that change, and adds a test case to lock it in.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 59.30%. Comparing base (54802d5) to head (07b2c1c).
Report is 1 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1757      +/-   ##
==========================================
- Coverage   59.79%   59.30%   -0.50%     
==========================================
  Files         300      307       +7     
  Lines       42025    42418     +393     
==========================================
+ Hits        25129    25154      +25     
- Misses      15474    15840     +366     
- Partials     1422     1424       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[VenelinMartinov]

Looks sensible but do we have enough coverage here given the previous regressions? I don't want to block on this if you are certain of the fix but it'd seem prudent to add some coverage if we missed this last time.

[VenelinMartinov]

Does this also need a test? Looks like the tests only cover the other direction.

[iwahbe]

The test is bi-directional. Both changes are covered.

---

I actually only noticed that this needed to be fixed because the test I added was still failing.

[iwahbe]

Looks sensible but do we have enough coverage here given the previous regressions? I don't want to block on this if you are certain of the fix but it'd seem prudent to add some coverage if we missed this last time.

We have coverage over the material that this PR changed. In terms of general coverage of makeTerraformInputs (last regression), more coverage would help, but I'm comfortable merging since the issue was caught by provider tests and didn't make it into a release.

I expect that closing #1282 will give us lots of coverage over makeTerraformInputs.

[t0yv0]

Yeah, very annoying omission. For this case schemaInfo.Elem.Fields makes so much more sense.

[t0yv0]

Thank you!