Fix panic when changing untargetted provider versions

[Frassle]

Description

Fixes #15704.

When doing a targeted run the source evaluator isn't aware of targets but it is responsible for registering default providers. As such on getting a resource event with a new provider version (e.g 1.0 -> 2.0) it will send of a registration for the new version it's seen, which as a default provider the step generator will accept and add to state (this is probably ok).
However when the step generator runs for the resource using this provider it will see it's not targetted and ignore its new goal state just reusing its old state. This old state will be referring to the old version of the provider (e.g "default_aws_1_0_0" rather than "default_aws_2_0_0"), which was causing a panic in the step generator when trying to build the overall stack state for StackAnalyze as the old provider had never been registered.

We now catch this situation when generating the same step for a non-targeted resource and error out that this isn't supported.

Checklist

• I have run make tidy to update any new dependencies
• I have run make lint to verify my code passes the lint check
  • I have formatted my code using gofumpt

• I have added tests that prove my fix is effective or that my feature works

• I have run make changelog and committed the changelog/pending/<file> documenting my change

• Yes, there are changes in this PR that warrants bumping the Pulumi Cloud API version

[pulumi-bot]

Changelog

[uncommitted] (2024-03-19)

Bug Fixes

• [engine] Fix a panic when updating provider version in a run using --target
  #15716

[tgummerer]

Thanks! So I guess it was #15476 that caused this, which was first in 3.109.0.

[Frassle]

So I guess it was #15476 that caused this, which was first in 3.109.0.

yeh maybe but I suspect things were wrong before that anyway because the state would of incorrectly said the resource was using the new provider version.

Change of tack

[Frassle]

So I've spent a couple of days on this and it's a gnarly one. Best I've got so far that's consistent is to just turn this into an error case rather than a panic.

It feels like the engine should be able to deal with this, but the current structure of the source evaluator and step generator makes that very hard, maybe impossible.

The source evaluator has no knowledge of --targets (or URNs for that matter), so when it gets the register resource request with the new provider version it doesn't know any better than to just register it onwards. The first part of that is registering the default provider for new provider version, and setting that as the "provider" field on the goal state that's sent to the step generator.

The step generator then does take --targets and old state into account, but the step generator both doesn't have any connection back to the source evaluator and default provider registrations but also more importantly isn't re-entrant. So even if we did pass a pointer to the default provider manager to the step generator it can't call into to start the default provider registration of the old provider version.

All in all, feels like we ought to be able to do something here to support this but I can't see what.

[tgummerer]

This seems like a good solution for at least providing a better error to the user for now. Might be worth opening a separate issue as a followup for making the user experience better here if we can in the future?