Refresh reorders set elements causing spurious diffs

[t0yv0]

What happened?

When servicing pulumi refresh the provider only gets to influence the result in a Read call, but the diff will be computed outside of the provider by the Pulumi CLI and the engine within; the engine does not understand sets and therefore deals with a list projection. The inputs are ordered arbitrarily according to program order, but the state read from the cloud might be (should be?) ordered by TF-canonical order (using Set function that gives an identity to each element). This makes Pulumi display spurious reordering diffs the are not visible in TF.

What we could do here:

• protocol change that lets bridged provider participate in computing diffs for refresh; make that set-aware

• without a protocol change we can do a pass over Read results that attempts to reorder set element projection to match the order from inputs as much as possible, that is intentionally minimize the spurious diffs within the current model; this is doable quickly I suggest we stat here

Example

A good example for Plugin Framework based resources is provided in pulumi/pulumi-aws#3303

I'm searching for more examples for SDKv2 based resources.

Output of pulumi about

See above.

Additional context

N/A

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[iwahbe]

Thanks for raising the issue. We will prioritize next iteration.

[t0yv0]

Slightly more involved proposal that could also fix for this: pulumi/pulumi#16072 - in this proposal the state out of Read would be threaded through the program again, so the usual Diff machinery would kick in that handles sets already.

[t0yv0]

This might change significantly with pulumi/pulumi#16146 that plumbs refresh results through Diff, though I've not verifed.

[t0yv0]

@mikhailshilkov reminds of an issue pulumi/pulumi-gcp#1935 affecting import (not just refresh). Possibly same fix if Read could canonicalize the ordering of sets.

[guineveresaenger]

pulumi/pulumi-gcp#1935 was fixed via pulumi/pulumi#16024.

[t0yv0]

I'm not sure I agree with pulumi/pulumi#16024 being a fix for this. It's a workaround for Providers returning inconsistent data. Consider an example where the set re-order is coupled with another independent actual change that makes the diff result DIFF_SOME. Then 16024 will not apply and 1935 will be right back.