Fix invoke functions without args #8805

@@           Coverage Diff           @@
##           master    #8805   +/-   ##
=======================================
  Coverage   59.39%   59.39%           
=======================================
  Files         639      639           
  Lines       98051    98066   +15     
  Branches     1386     1386           
=======================================
+ Hits        58235    58248   +13     
  Misses      36533    36533           
- Partials     3283     3285    +2

Impacted Files	Coverage Δ
pkg/codegen/internal/utils/providers.go	`57.14% <0.00%> (-10.95%)`	⬇️
pkg/codegen/pcl/invoke.go	`60.66% <25.00%> (-1.24%)`	⬇️
pkg/codegen/internal/utils/host.go	`85.00% <33.33%> (-15.00%)`	⬇️
pkg/codegen/hcl2/model/type_object.go	`89.18% <0.00%> (-0.55%)`	⬇️
pkg/codegen/hcl2/model/type_eventuals.go	`92.57% <0.00%> (-0.44%)`	⬇️
sdk/nodejs/automation/localWorkspace.ts	`74.03% <0.00%> (-0.39%)`	⬇️
sdk/go/common/resource/properties.go	`83.70% <0.00%> (+0.82%)`	⬆️
sdk/go/common/util/ciutil/github_actions.go	`73.52% <0.00%> (+38.23%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1181311...4667df6. Read the comment docs.

github-actions · 2022-01-21T11:34:43Z

Diff for pulumi-azure-native with merge commit c4c545e

github-actions · 2022-01-21T16:18:20Z

Diff for pulumi-random with merge commit 8a824a2

github-actions · 2022-01-21T16:18:42Z

Diff for pulumi-azuread with merge commit 8a824a2

github-actions · 2022-01-21T16:19:21Z

Diff for pulumi-kubernetes with merge commit 8a824a2

github-actions · 2022-01-21T16:23:39Z

Diff for pulumi-gcp with merge commit 8a824a2

github-actions · 2022-01-21T16:34:02Z

Diff for pulumi-azure with merge commit 8a824a2

github-actions · 2022-01-21T16:37:31Z

Diff for pulumi-aws with merge commit 8a824a2

github-actions · 2022-01-21T16:57:16Z

Diff for pulumi-azuread with merge commit 3f98728

github-actions · 2022-01-21T16:57:55Z

Diff for pulumi-random with merge commit 3f98728

github-actions · 2022-01-21T16:58:37Z

Diff for pulumi-kubernetes with merge commit 3f98728

github-actions · 2022-01-21T17:03:34Z

Diff for pulumi-gcp with merge commit 3f98728

github-actions · 2022-01-21T17:11:36Z

Diff for pulumi-azure-native with merge commit 8a824a2

github-actions · 2022-01-21T17:14:27Z

Diff for pulumi-azure with merge commit 3f98728

github-actions · 2022-01-21T17:17:26Z

Diff for pulumi-aws with merge commit 3f98728

github-actions · 2022-01-21T17:20:24Z

Diff for pulumi-random with merge commit a9af4ee

github-actions · 2022-01-21T17:20:36Z

Diff for pulumi-azuread with merge commit a9af4ee

github-actions · 2022-01-21T17:22:00Z

Diff for pulumi-kubernetes with merge commit a9af4ee

github-actions · 2022-01-21T17:26:03Z

Diff for pulumi-gcp with merge commit a9af4ee

github-actions · 2022-01-21T17:48:34Z

Diff for pulumi-azure-native with merge commit 3f98728

github-actions · 2022-01-21T18:18:23Z

Diff for pulumi-azure-native with merge commit a9af4ee

t0yv0 · 2022-01-21T19:29:30Z

pkg/codegen/internal/test/testdata/aws-native-pseudo-params-pp/aws-native-pseudo-params.pp

+	cloudWatchLogsRoleArn = invoke("aws-native::importValue", {
+		name = "TrailLogGroupRoleTestArn"
+	}).value
+	kmsKeyId = invoke("aws-native::importValue", {


Program generation breaks for me unless I specify "kMSKeyId" here...but only node code generation emits a legible error message. interesting..

t0yv0 · 2022-01-21T19:29:58Z

pkg/codegen/pcl/binder_test.go

@@ -13,37 +13,48 @@ import (
 )

 func TestBindProgram(t *testing.T) {
-	files, err := ioutil.ReadDir(testdataPath)
+	testdata, err := ioutil.ReadDir(testdataPath)


Thanks so much for this! This test was a no-op before your fix. Quite an omission.

t0yv0 · 2022-01-21T19:30:33Z

pkg/codegen/pcl/invoke.go

@@ -114,7 +118,7 @@ func (b *binder) zeroSignature() model.StaticFunctionSignature {
 }

 func (b *binder) signatureForArgs(fn *schema.Function, args model.Expression) (model.StaticFunctionSignature, error) {
-	if b.useOutputVersion(fn, args) {
+	if args != nil && b.useOutputVersion(fn, args) {


The spirit of this is reasonable.

t0yv0 · 2022-01-21T19:33:15Z

@danielrbradley do you have an issue describing the end-user problem with this?

I am a little worried because I've added your example to programgen tests and they are all broken (or almost). Go, TS, Dotnet are broken. Python generates code but based on how others are broken it might have runtime errors I need to check. checking docs generation now.

I will be happy to follow up with PRs that demonstrate these breakages and file issues against them I'm just wondering if this fix of yours is sufficient to unblock the use case so we can backlog fixing those things, or you need program generation to work, in which case you are still blocked?

Example: `invoke("aws-native::getPartition")` has no additional args object passed therefore the args length is 1. Graciously fallback to disable rendering of the output form for this case.

- No sub-tests were run since the folder structure changed in 06a19b5 - Also fix capture of loop variable in closure.

This fails if 735ab2b is reverted.

- Allow example generation to run without panic while we investigate deeper issues. - Remove example which appears to have errors (leave aws-native host there for now).

danielrbradley · 2022-01-24T11:37:02Z

@t0yv0 the current issue is that the latest pulumi/pulumi version in aws-native causes a panic during example generation (due to accessing args[1]), so the impact is not being able to upgrade.

This is my first time looking at example generation, so may well be some additional issues with cf2pulumi that's causing the invalid pcl which I don't yet understand.

I've changed direction a bit and am just emitting diagnostics if len(args) == 1 to avoid the panic and then should be able to investigate more on the aws-native side what's going on. Have left the aws-native test setup code in there for now though.

github-actions · 2022-01-24T11:38:30Z

Diff for pulumi-random with merge commit 34b9a53

github-actions · 2022-01-24T11:38:45Z

Diff for pulumi-azuread with merge commit 34b9a53

github-actions · 2022-01-24T11:39:58Z

Diff for pulumi-kubernetes with merge commit 34b9a53

github-actions · 2022-01-24T11:42:43Z

Diff for pulumi-gcp with merge commit 34b9a53

github-actions · 2022-01-24T11:49:38Z

Diff for pulumi-azure with merge commit 34b9a53

github-actions · 2022-01-24T12:06:16Z

Diff for pulumi-aws with merge commit 34b9a53

github-actions · 2022-01-24T12:36:23Z

Diff for pulumi-azure-native with merge commit 34b9a53

t0yv0 · 2022-01-24T16:13:17Z

pkg/codegen/pcl/invoke.go

@@ -80,6 +80,9 @@ func (b *binder) bindInvokeSignature(args []model.Expression) (model.StaticFunct
 		return b.zeroSignature(), hcl.Diagnostics{unknownFunction(token, tokenRange)}
 	}

+	if len(args) < 2 {
+		return b.zeroSignature(), hcl.Diagnostics{errorf(tokenRange, "missing second arg")}


Since we're warning something actionable here perhaps?

You might expand the comment, e.g. Invoke missing a required "args" parameter, try "invoke('token', {})" instead of invoke('token')

t0yv0

Sorry to see your example being removed? I thought that was very useful. Sorry if I was not being clear here:

your changes should ship I think, if invoke("foo") without args is intended to be supported (I think it is), good changes
the example was uncovering further bugs in specific language programgen backends. PCL was fine but the generated code would not compile. I'd like to add that to our backlog so we can fix, I was just wondering about your use case so I don't miss the actual problem you're having we should prioritize fixing that.

TYVM.

danielrbradley · 2022-01-24T18:50:14Z

Ah ok, shall we get this merged with the basic diagnostics so we can proceed with updating aws-native? Then I'll put my example code back onto a new branch so we can take some more time to figure out what's going on there.

t0yv0 · 2022-01-24T19:16:13Z

Yes no problem merging this at all.

t0yv0 · 2022-01-24T19:16:52Z

You have access right? Sorry if you need me to click "Squash and merge" I can do that just lmk.

danielrbradley self-assigned this Jan 21, 2022

danielrbradley marked this pull request as ready for review January 21, 2022 16:53

danielrbradley requested review from t0yv0, Frassle and mikhailshilkov January 21, 2022 16:53

t0yv0 reviewed Jan 21, 2022

View reviewed changes

danielrbradley added 6 commits January 24, 2022 11:21

Fix invoke functions without args

dae06ff

Example: `invoke("aws-native::getPartition")` has no additional args object passed therefore the args length is 1. Graciously fallback to disable rendering of the output form for this case.

Fix binder tests

5e19b1a

- No sub-tests were run since the folder structure changed in 06a19b5 - Also fix capture of loop variable in closure.

Add binder test for aws-native

94f353c

This fails if 735ab2b is reverted.

Add to pending changelog

c6d85b9

Fix lint warning

1be60ec

Emit diagnostics for missing arg

4667df6

- Allow example generation to run without panic while we investigate deeper issues. - Remove example which appears to have errors (leave aws-native host there for now).

danielrbradley force-pushed the danielrbradley/8775-fix-zero-args-fn branch from ec19f20 to 4667df6 Compare January 24, 2022 11:34

t0yv0 reviewed Jan 24, 2022

View reviewed changes

t0yv0 approved these changes Jan 24, 2022

View reviewed changes

danielrbradley merged commit f935dfb into master Jan 24, 2022

pulumi-bot deleted the danielrbradley/8775-fix-zero-args-fn branch January 24, 2022 20:59

This was referenced Jan 24, 2022

Example generation panic #8775

Closed

Recreate broken programgen tests #8823

Closed

Fix invoke functions without args #8805

Fix invoke functions without args #8805

Conversation

danielrbradley commented Jan 21, 2022 • edited Loading

Description

Checklist

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

codecov bot commented Jan 21, 2022 • edited Loading

Codecov Report

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

github-actions bot commented Jan 21, 2022

t0yv0 Jan 21, 2022

Choose a reason for hiding this comment

t0yv0 Jan 21, 2022

Choose a reason for hiding this comment

t0yv0 Jan 21, 2022

Choose a reason for hiding this comment

t0yv0 commented Jan 21, 2022

danielrbradley commented Jan 24, 2022

github-actions bot commented Jan 24, 2022

github-actions bot commented Jan 24, 2022

github-actions bot commented Jan 24, 2022

github-actions bot commented Jan 24, 2022

github-actions bot commented Jan 24, 2022

github-actions bot commented Jan 24, 2022

github-actions bot commented Jan 24, 2022

t0yv0 Jan 24, 2022

Choose a reason for hiding this comment

t0yv0 left a comment

Choose a reason for hiding this comment

danielrbradley commented Jan 24, 2022

t0yv0 commented Jan 24, 2022

t0yv0 commented Jan 24, 2022

danielrbradley commented Jan 21, 2022 •

edited

Loading

codecov bot commented Jan 21, 2022 •

edited

Loading