Remove shared nsg rule immediately when no destinations prefixes are left

[MartinForReal]

What type of PR is this?

/kind bug

What this PR does / why we need it:

Remove shared nsg rule immediately when no destinations left

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Fixed: Remove shared nsg rule immediately when no destinations left

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

[netlify]

✅ Deploy Preview for kubernetes-sigs-cloud-provide-azure canceled.

Name	Link
🔨 Latest commit	cb8383b
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-sigs-cloud-provide-azure/deploys/6442639f7c60dd00083b0058

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: MartinForReal

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [MartinForReal]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jwtty]

Do we have a test case where the sg rule should be deleted completely?

[MartinForReal]

Please refer to

cloud-provider-azure/tests/e2e/network/network_security_group.go

Lines 143 to 191 in 3f6d95e

	It("should support service annotation `service.beta.kubernetes.io/azure-shared-securityrule`", func(ctx SpecContext) {
	By("Exposing two services with shared security rule")
	annotation := map[string]string{
	consts.ServiceAnnotationSharedSecurityRule: "true",
	}
	ips1 := createAndExposeDefaultServiceWithAnnotation(cs, tc.IPFamily, serviceName, ns.Name, labels, annotation, ports)
	defer func() {
	err := utils.DeleteService(cs, ns.Name, serviceName)
	Expect(err).NotTo(HaveOccurred())
	}()
	serviceName2 := serviceName + "-share"
	ips2 := createAndExposeDefaultServiceWithAnnotation(cs, tc.IPFamily, serviceName2, ns.Name, labels, annotation, ports)
	defer func() {
	By("Cleaning up")
	err := utils.DeleteService(cs, ns.Name, serviceName2)
	Expect(err).NotTo(HaveOccurred())
	}()
	By("Validate shared security rule exists")
	port := fmt.Sprintf("%d", serverPort)
	nsgs, err := tc.GetClusterSecurityGroups()
	Expect(err).NotTo(HaveOccurred())
	ipList := append(ips1, ips2...)
	Expect(validateSharedSecurityRuleExists(nsgs, ipList, port)).To(BeTrue(), "Security rule for service %s not exists", serviceName)
	By("Validate automatically adjust or delete the rule, when service is deleted")
	Expect(utils.DeleteService(cs, ns.Name, serviceName)).NotTo(HaveOccurred())
	ipList = ips2
	Eventually(func() (bool, error) {
	nsgs, err = tc.GetClusterSecurityGroups()
	if err != nil {
	return false, err
	}
	return validateSharedSecurityRuleExists(nsgs, ips2, port) && !validateSharedSecurityRuleExists(nsgs, ips1, port), nil
	}).WithContext(ctx).Should(BeTrue(), "Security rule should be modified to only contain service %s", serviceName2)
	Expect(utils.DeleteService(cs, ns.Name, serviceName2)).NotTo(HaveOccurred())
	Eventually(func() (bool, error) {
	nsgs, err := tc.GetClusterSecurityGroups()
	if err != nil {
	return false, err
	}
	return validateSharedSecurityRuleExists(nsgs, ipList, port), nil
	}).WithContext(ctx).Should(BeFalse(), "Fail to automatically delete the shared security rule")
	})

[feiskyer]

/lgtm

[MartinForReal]

/retest

[MartinForReal]

/cherrypick release-1.27

[MartinForReal]

/cherrypick release-1.26

[MartinForReal]

/cherrypick release-1.25

[MartinForReal]

/cherrypick release-1.24

[k8s-infra-cherrypick-robot]

@MartinForReal: new pull request created: #3797

In response to this:

/cherrypick release-1.27

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-infra-cherrypick-robot]

@MartinForReal: new pull request created: #3798

In response to this:

/cherrypick release-1.26

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-infra-cherrypick-robot]

@MartinForReal: new pull request created: #3799

In response to this:

/cherrypick release-1.25

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-infra-cherrypick-robot]

@MartinForReal: new pull request created: #3800

In response to this:

/cherrypick release-1.24

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.