Add custom groups support to upgrade run resource #1191

ksamoray · 2024-04-18T18:38:40Z

Add the logic to create and update custom host groups while running the upgrade process.

annakhm · 2024-04-19T00:28:17Z

nsxt/resource_nsxt_host_upgrade_group.go

+				Optional:    true,
+				Default:     true,
+			},
+			"extended_configuration": {


I think we can reuse schema for key/value pairs, we already have a func getSwitchingProfileIdsSchema, we might want to rename it and generalize the description. Also advanced_configuration of edge transport node might benefit from this

Switching profile uses a different schema than edge transport node and upgrade - SwitchingProfileTypeIdEntry vs KeyValuePair.
I'll consolidate edge and upgrade and leave switching profiles as is.

annakhm · 2024-04-19T00:32:13Z

nsxt/resource_nsxt_host_upgrade_group_assignment.go

+		return fmt.Errorf("host %s already exists in group %s", hostID, groupID)
+	}
+	group.UpgradeUnits = append(group.UpgradeUnits, model.UpgradeUnit{Id: &hostID})
+	_, err = client.Update(groupID, group)


I feel simultaneous updates might cause errors here, so we might need to wrap this code (including Get) in retry. Auto-retry that we have today will not re-read the revision, so Update would fail again if revision got updated. What do you think?

Makes a lot of sense. Do we have any example for such retry mechanism in the code?
BTW it seems like the _revision attribute is ignored by NSX - I'll open a bug for this.

I think that I've found something.

ksamoray · 2024-04-21T14:18:09Z

/test-all

ksamoray · 2024-04-22T09:55:44Z

/test-all

ksamoray · 2024-05-09T12:01:41Z

/test-all

ksamoray · 2024-05-09T12:08:43Z

/test-all

ksamoray · 2024-05-09T12:18:12Z

/test-all

annakhm · 2024-05-13T22:19:46Z

nsxt/resource_nsxt_upgrade_run.go

+				return fmt.Errorf("couldn't find upgrade unit group without id or display_name")
+			}
+			// This is a custom group, try to find it by name
+			groupList, err := upgradeClientSet.GroupClient.List(nil, nil, nil, nil, nil, nil, nil, nil)


is pagination relevant here?

Do we expect more than 1000 groups?

Doesn't sound feasible, but maybe a comment about it would be helpful

annakhm · 2024-05-13T22:21:33Z

nsxt/resource_nsxt_upgrade_run.go

+		var err error
+		isCreate := false
+		if groupID == "" {
+			groupName := group["display_name"].(string)


can the user change display_name for existing group?

No - this isn't doable as the display_name is used to identify a pre-existing custom group, and as the upgrade support doesn't maintain/support state.
If the upgrade implementation would support state, we could identify the group by id and update the group.

annakhm · 2024-05-13T22:29:48Z

nsxt/resource_nsxt_upgrade_run.go

+						} else if err != nil {
+							return handleDeleteError("Host Upgrade Group Binding", nsxUnit, err)
+						}
+						return addHostToGroup(m, groupID, nsxUnit, false)


Is the return here intentional?

It's a bug...

annakhm · 2024-05-13T22:39:13Z

nsxt/resource_nsxt_upgrade_run.go

+
+		hostIDs := getUnitIDsFromUnits(group.UpgradeUnits)
+		if slices.Contains(hostIDs, hostID) {
+			return fmt.Errorf("host %s already exists in group %s", hostID, groupID)


should we swallow this error?

Indeed, thanks

annakhm · 2024-05-13T22:43:59Z

nsxt/resource_nsxt_upgrade_run.go

+	// it should be assigned to the 'Group 1 for ESXI' group (this value is hardcoded in NSX)
+	groupClient := upgrade.NewUpgradeUnitGroupsClient(connector)
+	componentType := "HOST"
+	hostGroups, err := groupClient.List(&componentType, nil, nil, nil, nil, nil, nil, nil)


It seems wasteful to run this API for each getHostDefaultUpgradeGroup call, can the logic be change to run it once? We only care for default groups here, and default groups are not expected to change throughout the update operation, is that correct?

You mean, load the entire host/group map and cache it for the entire execution?
As the "original" upgrade group for each host could be different according to cluster membership (or lack of cluster association).

Yes, only run the API once and use the list for all calls of this function

annakhm · 2024-05-13T22:48:50Z

I didn't see any update in setUpgradeRunOutput function (sorry if I missed it), but looks like custom group needs to be updated with its new attributes

ksamoray · 2024-05-16T11:46:56Z

I didn't see any update in setUpgradeRunOutput function (sorry if I missed it), but looks like custom group needs to be updated with its new attributes

Yeah I missed this. Although as we do not support the state existence, that's a bit meaningless for now.

ksamoray · 2024-05-16T11:50:04Z

I didn't see any update in setUpgradeRunOutput function (sorry if I missed it), but looks like custom group needs to be updated with its new attributes

But this is only the status type - which doesn't have the extra group attributes. So I'm not sure that a change is required here.

annakhm · 2024-05-17T00:18:01Z

I didn't see any update in setUpgradeRunOutput function (sorry if I missed it), but looks like custom group needs to be updated with its new attributes

Yeah I missed this. Although as we do not support the state existence, that's a bit meaningless for now.

I think that although we ask to remove state post-upgrade, terraform plan should not produce non-empty diff after upgrade. is this true today?

annakhm · 2024-05-23T00:10:39Z

LGTM, except for two doubts:

after upgrade is complete and there are no new changes in config, would terraform plan show non-empty diff?
comment below regarding deleted custom group

annakhm · 2024-05-23T00:18:21Z

nsxt/resource_nsxt_upgrade_run.go

@@ -503,9 +523,83 @@ func updateUpgradeUnitGroups(upgradeClientSet *upgradeClientSet, d *schema.Resou
 	for _, groupI := range d.Get(componentToGroupKey[component]).([]interface{}) {


Looks like the code doesn't delete custom groups that were created with previous apply (which might have failed) but are not present in current config, is that correct?
To solve this, d.GetChange("host_group") might be useful

Add the logic to create and update custom host groups while running the upgrade process. Signed-off-by: Kobi Samoray <kobi.samoray@broadcom.com>

Signed-off-by: Kobi Samoray <kobi.samoray@broadcom.com>

ksamoray · 2024-05-26T13:56:05Z

Terraform doesn't show an empty diff, also without these changes as far as I understand (there are dependencies on computed groups within the upgrade process - I think that this is the reason?).
As for group deletion: it's possible to check the groups for empty groups (with no upgrade units) which do not match the "predefined group pattern" (either have the id of a compute collection, or that hardcoded name which NSX uses for non-clustered hosts). These can be deleted I guess. Is that what you had in mind?

annakhm · 2024-05-28T21:19:58Z

Terraform doesn't show an empty diff, also without these changes as far as I understand (there are dependencies on computed groups within the upgrade process - I think that this is the reason?). As for group deletion: it's possible to check the groups for empty groups (with no upgrade units) which do not match the "predefined group pattern" (either have the id of a compute collection, or that hardcoded name which NSX uses for non-clustered hosts). These can be deleted I guess. Is that what you had in mind?

I would suggest to rely on GetChange routing to check if groups are explicitly deleted + deleting groups that we know are recreated as a result of update.

When a host upgrade group is deleted, clean it from NSX. Signed-off-by: Kobi Samoray <kobi.samoray@broadcom.com>

ksamoray · 2024-06-04T14:45:47Z

/test-all

ksamoray force-pushed the upgrade_groups branch 2 times, most recently from 55d6452 to 67fdfd9 Compare April 18, 2024 19:11

annakhm reviewed Apr 19, 2024

View reviewed changes

ksamoray force-pushed the upgrade_groups branch 2 times, most recently from af8f4a4 to b9f3123 Compare April 21, 2024 14:11

ksamoray force-pushed the upgrade_groups branch from b9f3123 to fb91dfa Compare May 9, 2024 11:19

ksamoray changed the title ~~Implement upgrade groups resources~~ Add custom groups support to upgrade run resource May 9, 2024

ksamoray force-pushed the upgrade_groups branch 3 times, most recently from 8aeb442 to f57ef48 Compare May 9, 2024 11:56

ksamoray force-pushed the upgrade_groups branch from f57ef48 to 637ed16 Compare May 9, 2024 12:08

ksamoray force-pushed the upgrade_groups branch from 637ed16 to b06de98 Compare May 9, 2024 12:17

annakhm reviewed May 13, 2024

View reviewed changes

ksamoray force-pushed the upgrade_groups branch from b06de98 to c646023 Compare May 15, 2024 14:55

ksamoray force-pushed the upgrade_groups branch from c646023 to 8e737ea Compare May 21, 2024 12:05

annakhm reviewed May 23, 2024

View reviewed changes

ksamoray force-pushed the upgrade_groups branch from 8e737ea to 4a9baa6 Compare May 23, 2024 09:26

ksamoray added 2 commits May 26, 2024 14:25

Add custom groups support to upgrade run resource

b2715c4

Add the logic to create and update custom host groups while running the upgrade process. Signed-off-by: Kobi Samoray <kobi.samoray@broadcom.com>

Use one function call to retrieve group list

d235730

Signed-off-by: Kobi Samoray <kobi.samoray@broadcom.com>

ksamoray force-pushed the upgrade_groups branch from 4a9baa6 to d235730 Compare May 26, 2024 11:25

annakhm approved these changes Jun 4, 2024

View reviewed changes

ksamoray force-pushed the upgrade_groups branch 2 times, most recently from 91ff29b to f71f3d6 Compare June 4, 2024 08:44

Host upgrade group cleanup when not in schema

1996125

When a host upgrade group is deleted, clean it from NSX. Signed-off-by: Kobi Samoray <kobi.samoray@broadcom.com>

ksamoray force-pushed the upgrade_groups branch from f71f3d6 to 1996125 Compare June 4, 2024 08:47

ksamoray merged commit b9a7d1f into vmware:master Jun 6, 2024
8 checks passed

ksamoray deleted the upgrade_groups branch June 6, 2024 10:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add custom groups support to upgrade run resource #1191

Add custom groups support to upgrade run resource #1191

ksamoray commented Apr 18, 2024 •

edited

Loading

annakhm Apr 19, 2024

ksamoray Apr 21, 2024

annakhm Apr 19, 2024

ksamoray Apr 21, 2024

ksamoray Apr 21, 2024

ksamoray commented Apr 21, 2024

ksamoray commented Apr 22, 2024

ksamoray commented May 9, 2024

ksamoray commented May 9, 2024

ksamoray commented May 9, 2024

annakhm May 13, 2024

ksamoray May 15, 2024

annakhm May 23, 2024

annakhm May 13, 2024

ksamoray May 15, 2024

annakhm May 13, 2024

ksamoray May 15, 2024

annakhm May 13, 2024

ksamoray May 15, 2024

annakhm May 13, 2024

ksamoray May 15, 2024

annakhm May 15, 2024

annakhm commented May 13, 2024

ksamoray commented May 16, 2024

ksamoray commented May 16, 2024

annakhm commented May 17, 2024

annakhm commented May 23, 2024 •

edited

Loading

annakhm May 23, 2024

ksamoray commented May 26, 2024

annakhm commented May 28, 2024

ksamoray commented Jun 4, 2024

		@@ -503,9 +523,83 @@ func updateUpgradeUnitGroups(upgradeClientSet upgradeClientSet, d schema.Resou
		for _, groupI := range d.Get(componentToGroupKey[component]).([]interface{}) {

Add custom groups support to upgrade run resource #1191

Add custom groups support to upgrade run resource #1191

Conversation

ksamoray commented Apr 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ksamoray commented Apr 21, 2024

ksamoray commented Apr 22, 2024

ksamoray commented May 9, 2024

ksamoray commented May 9, 2024

ksamoray commented May 9, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

annakhm commented May 13, 2024

ksamoray commented May 16, 2024

ksamoray commented May 16, 2024

annakhm commented May 17, 2024

annakhm commented May 23, 2024 • edited Loading

Choose a reason for hiding this comment

ksamoray commented May 26, 2024

annakhm commented May 28, 2024

ksamoray commented Jun 4, 2024

ksamoray commented Apr 18, 2024 •

edited

Loading

annakhm commented May 23, 2024 •

edited

Loading