Rename a resource

[lukehoban]

Today, if you rename a resource in code, Pulumi has no idea that the old name and new name are related. It schedules a delete of the old and a create of the new.

A workaround is to manually edit the checkpoint file to change the name in the same way - since these names are just logical names that are only tracked in the checkpoint file, this is technicaly possible.

It might be nice to have a command line way to do this more safely (especially for cases where the checkpoint file is managed remotely).

This is similar to terraform state mv. Generally, we will likely eventually want logical equivalents of all terraform state commands.

[joeduffy]

I was actually dreaming about this last night. (No joke.)

It seems to me there are two approaches to go about this: (1) rename the state in the checkpoint file, or (2) apply renames "during" the preview or update process.

The first seems simpler, but it doesn't fit the workflow I anticipate wanting to use when running into this issue. Namely, you've renamed something in your program and want to match the old state with the new state during an update. Being able to preview that to make sure you got it all correct and then locking it in alongside the update seems to fit our model so much better than an approach that mutates the checkpoint file's URNs prior to actually applying the update in which the rest of the updates are going to take place. The update should be the one atomic advancement of the state.

How might this look? I'm thinking you could just have an option to both preview and update, something like --remap-urn, where you could say --remap-urn x=y; any occurrences of x in the old checkpoint are treated as though they were y. I also think we can consider supporting regular expressions in here to do bulk remappings (like changing the project part of the URN).

We also need to think about component authors making breaking changes to renames and wanting to minimize the impact of said updates. As an extreme example, imagine we rename some components deep in the bowel's of our own cloud framework. To minimize the customer impact, we could redistribute a remapping file that does the bulk renames of any resources being ingested. In theory, we could even have some story for autodetecting during upgrades, based on semver.

[lukehoban]

I'm thinking you could just have an option to both preview and update, something like --remap-urn, where you could say --remap-urn x=y; any occurrences of x in the old checkpoint are treated as though they were y.

That seems reasonable - but not clear why it's much better than separately pulumi rename followed by pulumi plan. The latter is more decoupled, and doesn't make it any harder to see the preview prior prior to doing the update. In fact, it makes it easier to ensure that you don't accidentally do pulumi preview --remap-urn followed by pulumi update - forgetting to do the remap on the update. What's the downside of keeping these separate? (The upside is more overall flexibility, and separation of concerns).

We also need to think about component authors making breaking changes to renames and wanting to minimize the impact of said updates. As an extreme example, imagine we rename some components deep in the bowel's of our own cloud framework. To minimize the customer impact, we could redistribute a remapping file that does the bulk renames of any resources being ingested. In theory, we could even have some story for autodetecting during upgrades, based on semver.

I do worry a lot about this. It feels like just about any change in one of our components will need to be a semver breaking change in the way we are currently approaching things. If we can come up with some reliable way to annotate components with renames they should apply - that could be very helpful (though not yet clear what the impact of implicitly forcing renames on a users checkpoint file will be).

[brandonbloom]

How about adding aliases: [...] to the general resource options? Then a rename would be:

1. move the current name in to the aliases option array.
2. provide a new primary name.
3. run pulumi update
4. later, remove the alias.

This approach means that the rename has a trail in source control where both names exist at once.

As a bonus, code that still uses the old name wouldn't immediately break, but could issue a warning about using a deprecated / non-primary resource name.

[joeduffy]

I really love this idea. It also points out a critical flaw in everything discussed earlier, which is that it would not be tracked in source control, and instead would have been a trail of manually run commands.

[lukehoban]

Thanks @brandonbloom! In fact, @pat @ellismg and myself were discussing almost exactly the same thing just yesterday (though I was calling it previousNames, which is much worse than aliases 😄).

In our discussion yesterday we were very optimistic that this would work.

Also notable that this additional annotation only needs to stay around in source code as long as you have stacks in the wild that use the old names that you might want to be able to migrate forward. Once you know no more of these exist, you can remove it safely.

[lukehoban]

I've opened an initial PR for investigation at #2404.

Here's a quick note on the general approach:

Aliases provide a way for a resource to be reliably renamed without forcing recreation of the underlying cloud resource. This is important for several scenarios:

1. A simpler or different name is desired
2. A resource is moved to be owned by a component (or subcomponent, or out of a component, etc.) - this implicitly changes its fully qualified name (URN)
3. Overall changes to the URN format - for example removing the inclusion of project/stack name in the URN to avoid changes to those from recreating all resources - in this case the aliasing would be managed internally by Pulumi.

The idea of aliases is to allow any resource creation to specify an additional set of names that it should be aliased to. It will also still get a URN allocated as normal based on it's name and the component hierarchy it is part of. But if any of the names it lists as an alias is found in the previous state, the new name will be treated as being the same resource.

Users can add aliases to code whenever they need to change the name, then do an update and they should see no diff, and after doing the deployment on all stacks that might exist using the old names, the aliases can be removed. But by putting this in code (instead of just supporting a pulumi state rename command), the renaming can be codified so that it can be rolled out across multiple stacks (dev/stage/prod) and can be delivered as part of components that may be instantiated in an unbounded number of stacks (and would thus need to keep aliases indefinitely to avoid breaking changes).

For the simple end user case, a resource can change from this:

const res = new Resource("foobar");

to this:

const stackName = pulumi.getStack();
const projectName = pulumi.getProject();
const res = new Resource("baz", { aliases: [`urn:pulumi:${stackName}::${projectName}::my:module:Type::foobar`]});

The complexity of building up the old URN is unfortunate here - but it is in general important to be able to express all the possible changes that might be needed to the URN. We could consider adding shorthands for some common cases, but even better would be to use this feature to simplify the URN scheme itself, so that this is not as difficult.

For components, there are two scenarios:

1. A user setting an alias on a Component at construction time
2. A component refactoring it's internal implementation
3. "Adopting" an individual resource into a component

For the first case, we will want this to cause all children to be implicitly aliased using the parents alias root. This is not implemented yet in the PR, but should be possible.

For the second case, the component can use an alias internally constructed based on the refactoring in question (again, a little complicated with current URN scheme, but possible).

The third case is unfortunately not clear yet. If the component is only used in one place (like during initial implementation) the implementation can be updated to alias to the old child identity (and the child resource can be removed). In practice, this will handle the common cases of initial refactoring into components. But in more complex scenarios users may want to adopt a resource into a component which was defined elsewhere and manages a few resources in a addition to the one being adopted. It's hard to imagine fully general patterns for supporting this, but if important, it could be that component resources get an additional flag for something like childAliases to specify aliases to apply to specific children of the component from the outside.

[ellismg]

It's hard to imagine fully general patterns for supporting this, but if important, it could be that component resources get an additional flag for something like childAliases to specify aliases to apply to specific children of the component from the outside.

I do wonder if it would be possible to have some way to mark "I'd like you to add an alias to me that is based on the URN that would have been assigned if I was not part of a component hierarchy, and instead had been defined at top level". I think that would provide a nice gesture for taking an existing resource and adopting it into a component hierarchy and possibility could be done in a way such that I didn't have to try to build a URN myself.

[lukehoban]

We reviewed the design proposal in #458 (comment) and #2404, and there was consensus to move ahead with this approach.

The biggest concern was with the ability to "rename" a component and have that alias down through children - this is currently difficult because of the (effectively required, but also for many cases practical) pattern of including the name of the component as the prefix of the name of the children resource - either with:

1. new Child(name, { parent: this } or
2. new Child(name+"-child", { parent: this}

This breaks the child aliasing scheme because not just the parent type needs to be updated, but also the child resource name - but we don't know a-priori how to inject the "old name" into the name used for the child resource.

The solution we landed on for this was to automatically replace the new parent resource name in the prefix of the child name with the old name as part of constructing the child aliases. This is a little bit more "convention"-based than we'd ideally like, but practically will work well and will encourage additional consistency. We will add this to component design guidelines.

We also discussed helpers for URN construction. We will likely add one of these to construct a URN from a parent and child Resource object (which can be done synchronously even though the urn properties of the resources are Outputs).

[Sytten]

@lukehoban I know this is an old issue, but the aliases are not fun to use when you create nested components and I would really enjoy having a rename CLI

[GaboFDC]

Also #837 was closed as a dupe of this. But after the actual implementation changing the parent-child relationship is still a braking change, despite if being purely cosmetic/organizational for pulumi, having to delete and create a new resource in order to change, or create this organization (parent-child) marked it prohibitive for various cases.

[lukehoban]

Also #837 was closed as a dupe of this. But after the actual implementation changing the parent-child relationship is still a braking change, despite if being purely cosmetic/organizational for pulumi, having to delete and create a new resource in order to change, or create this organization (parent-child) marked it prohibitive for various cases.

As noted as well in #837 (comment), aliases make it possible to change the parent of a resource without replacement, but you must indicate the previous parent name as an alias. I take the feedback here and from @Sytten as "it should be easier to change a resource parent without needing to add alias". I'll open a new issue to track that. There are some deep and subtle challenges, but it's worth looking into.

[slavafomin]

I was trying to rename an aws.route53.Record resource today using the aliases option. But for some reason Pulumi was trying to just create a new DNS record and got into the record name conflict on the AWS side instead of just renaming the resource without applying any changes.

[Frassle]

@slavafomin I'd suggest opening a new issue for that. If you could add what language SDK and what the alias looked like to that ticket it will be much easier for us to work out what could have gone wrong and help fix.