[release-1.25] fix: decouple vmss from the lb if the backend pool is empty when usin…

[nilo19]

…g ip-based LB

chore: do not set etag to nil when updating lb backend pool

(cherry picked from commit a3fa07b)

What type of PR is this?

/kind bug
/kind cleanup

What this PR does / why we need it:

1. We remove all backend pool refs from the vmss when the backend pool type is migrated from nic to IP. But we only do this when there are vms in the backend pool. If there is no VM in the pool, we ignore this operation, leaving the lb backend pool in the vmss model. In this case, if we scale the vmss up, the new VM will be automatically attached to the backend pool, which is not allowed because the pool is already ip-based. We hit the issue only when there is no node in the cluster and we try to migrate the backend pool type from nic to ip, which is not allowed in AKS, so only self-managed clusters are influenced. This PR ensures the vmss will be decoupled from the backend pool even when the pool is empty. If the vmss has already been decoupled, it will be a no-op, so no extra API calls will be introduced.
2. We used to manually set the etag of the lb to nil to prevent etag mismatch error, but there may be potential race condition because of that. This PR re-construct the lb in the code from arm when necessary.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

fix: decouple vmss from the lb if the backend pool is empty when using ip-based LB
chore: do not set etag to nil when updating lb backend pool

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: nilo19

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [nilo19]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[coveralls]

Coverage decreased (-0.03%) to 80.385% when pulling a65610d on nilo19:fix/cherry-pick-2829-1.25 into 80797f8 on kubernetes-sigs:release-1.25.

[jwtty]

/lgtm