[sdk/nodejs] Fix construct to wait for RPC operations to complete

[justinvp]

This change fixes the provider implementation of Construct for multi-lang components written in Node.js to wait for any in-flight RPCs to finish before returning the results, s.t. all registered child resources are created.

Tests fail before the SDK change, and pass after.

Fixes #6365

[justinvp]

I'm generally not a huge fan of introducing delays like this in tests. I was thinking about having the provider spin until it received some signal, but I couldn't think of a good place where to send the signal in the tests. Open to other ideas.

15 seconds seemed to be the least amount of time to reliably reproduce to issue, at least on my machine. Any less and sometimes the tests would pass without the SDK change.

[EvanBoyle]

One question so far. I'll build this change locally and report back my findings as well.

[EvanBoyle]

What is the mapping of provider instance to construct request? One-to-one or one-to-many?

disconnecting the runtime causes the rpc clients for the engine and resmon to disconnect, but I believe those connections can be re-established the next time a mon/engine client is accessed in the node runtime.

[justinvp]

What is the mapping of provider instance to construct request? One-to-one or one-to-many?

Pretty sure one-to-many. The provider process will run and can receive multiple construct calls. We reset the values on each invocation of construct:

pulumi/sdk/nodejs/provider/server.ts

Lines 250 to 275 in ec220de

	public async construct(call: any, callback: any): Promise<void> {
	try {
	const req: any = call.request;
	const type = req.getType();
	const name = req.getName();
	if (!this.provider.construct) {
	callback(new Error(`unknown resource type ${type}`), undefined);
	return;
	}
	// Configure the runtime.
	//
	// NOTE: these are globals! We should ensure that all settings are identical between calls, and eventually
	// refactor so we can avoid the global state.
	runtime.resetOptions(req.getProject(), req.getStack(), req.getParallel(), this.engineAddr,
	req.getMonitorendpoint(), req.getDryrun());
	const pulumiConfig: {[key: string]: string} = {};
	const rpcConfig = req.getConfigMap();
	if (rpcConfig) {
	for (const [k, v] of rpcConfig.entries()) {
	pulumiConfig[k] = v;
	}
	}
	runtime.setAllConfig(pulumiConfig);

We should be able to disconnect from the monitor and engine as the next call to construct will re-establish connections to these.

Though, this is changing globals... Seems like we'll run into problems if there are ever concurrent calls to construct, though not sure offhand if that's even possible given the event loop. @pgavlin, do you know off hand if concurrent calls are possible? Should we be serializing calls to construct in the Node.js implementation so that only one call to construct runs at a time?

[pgavlin]

Seems like we'll run into problems if there are ever concurrent calls to construct, though not sure offhand if that's even possible given the event loop. @pgavlin, do you know off hand if concurrent calls are possible?

Concurrent calls are absolutely possible, yes. Serializing construct calls is a good short-term workaround, but I'm not sure that it's sustainable long-term due to the limits it places on parallelism. We might want to scope the engine connection somehow (or at least the in-flight RPCs). IIRC @lukehoban suggested investigating node's createContext API to see if we could run each invocation of Construct in its own context.

I think that we will also need to await inside of the provider's implementation of Cancel.

[EvanBoyle]

I did a quick check, and this doesn't solve all of my runtime issues. There is one issue in particular where when using nested multi-lang components, one output resolves as undefined on the first update but resolves fine on the second update. This change doesn't seem to fix that.

[justinvp]

There is one issue in particular where when using nested multi-lang components, one output resolves as undefined on the first update but resolves fine on the second update. This change doesn't seem to fix that.

@EvanBoyle, Do you have a small repro? Or point me to the code where you're seeing this?

[EvanBoyle]

Ok, good news is the issue where the program was not waiting for the ECS service/task to get created is no longer an issue. I was able to remove the service and task from the state of the component and it is still working as expected.

I don't have a simple repro for the output issue. But my repro is here:

https://github.com/pulumi/astro/blob/main/provider/nodejs/astro/components/fargateContainerApp.ts#L120

If you change that line to instead be const endpoint = app.endpoint, then that output seems to get dropped from the dependency graph. Other components begin so see astoOutputs resolve as undefined at that point. I'm wondering if there is some issue about serialized outputs from resource references. In that example, FargateApp is a component resource, and LocalApp is a component resource. Trying to use LocalApp.endpoint as part of FargateApp.astroOutputs seems to cause astroOutputs to resolve undefined.

[justinvp]

Let's tackle the nested issue separately (tracked in #6477).

[pgavlin]

The serialization looks good. My only concern is that this will deadlock on nested calls to construct. We can fix that when we fix the more general concurrency issues.