Add Dual-Stack support to L4 ILB

[panslava]

Add support for spec.ipFamilies and spec.ipFamiliyPolicy in L4 ILB

• Add new feature-flag --enable-l4ilb-dual-stack, all the new logic is hidden behind it
• Support all the transitions between ipFamilies, controller sync will provide resources for required ipFamily and clean resources for not required, but only if they exist in annotations
• On service deletion, clean up all the resources, no matter if they exist in annotations

[k8s-ci-robot]

@panslava: GitHub didn't allow me to assign the following users: czawadka.

Note that only kubernetes members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time.
For more information please see the contributor guide

In response to this:

/assign czawadka

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[panslava]

/assign cezarygerard

[cezarygerard]

let's discuss some way to slice the l4 struct

[cezarygerard]

Please see my comments

I still didn't review the following packages:

pkg/healthchecksl4
pkg/l4lb
pkg/loadbalancers

[panslava]

/hold don't merge to the current release

[cezarygerard]

Slavik, we need some unit tests in
pkg/healthchecksl4/healthchecksl4_test.go

[cezarygerard]

can you move all this code to private method:

	existingBS, err := l4.backendPool.Get(name, meta.VersionGA, l4.scope)
	err = utils.IgnoreHTTPNotFound(err)
	if err != nil {
		klog.Errorf("Failed to lookup existing backend service, ignoring err: %v", err)
	}

	existingFR, err := l4.forwardingRules.Get(l4.GetFRName())
	if existingBS != nil && existingBS.Protocol != string(protocol) {
		klog.Infof("Protocol changed from %q to %q for service %s", existingBS.Protocol, string(protocol), l4.NamespacedName)
		// Delete forwarding rule if it exists
		frName := l4.getFRNameWithProtocol(existingBS.Protocol)
		existingFR, err = l4.forwardingRules.Get(frName)
		if err != nil {
			klog.Errorf("Failed to get forwarding rule %s, err %v", frName, err)
		}
		err = l4.forwardingRules.Delete(frName)
		if err != nil {
			klog.Errorf("Failed to delete forwarding rule %s, err %v", frName, err)
		}

[cezarygerard]

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cezarygerard, panslava

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [cezarygerard,panslava]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[panslava]

/unhold