Support for configurable Network Interfaces #2411

brianlieberman · 2022-06-22T20:43:15Z

/kind feature

What this PR does / why we need it:
This PR adds the ability to configure one or more NetworkInterfaces for AzureMachines and AzureMachinePools, as well as pre-warming multiple IPConfigs on each interface.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #2327

TODOs:

[ x] squashed commits
[ x] includes documentation
[ x] adds unit tests

Release note:

Adds additional fields for AzureMachine, AzureMachineTemplate, and AzureMachinePool to configure multiple NetworkInterfaces. An example configuration:

         - subnetName: node-subnet
           privateIPConfigs: 2

linux-foundation-easycla · 2022-06-22T20:43:20Z

The committers listed above are authorized under a signed CLA.

✅ login: brianlieberman / name: Brian Lieberman (a9b3b46, c19ddea, 6e20ef6, 2a6e944, 9afbebe, 3e5a2f6, 7bd6b24, 9ab3955)
✅ login: dthorsen / name: Dane Thorsen (3800b8b, 6095d7b, 5a45826)

k8s-ci-robot · 2022-06-22T20:43:24Z

Hi @brianlieberman. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

CecileRobertMichon · 2022-06-22T21:39:33Z

Haven't looked at the code yet, but thanks for opening this!

One question about the UX:

 ipConfigs:
  - {}
  - {}

If a user needs to pre-allocate say 250 IPs to work with Azure CNI in a 250 max pods per node configuration, how would this scale? It might make sense to allow the user to specify an IP count instead?

brianlieberman · 2022-06-22T22:01:37Z

Yeah ,as currently implemented it doesn't scale to that level well from a UX perspective, there's a tradeoff since currently you can specify a configuration for each IPConfig (publicIP, privateIP, allocatePublicIP). In the example we leave them empty to just provision a default of a dynamic privateIP, but that's the tradeoff we might make in order to make something like an int we can iterate instead. Open to implementation suggestions for improvement.

dthorsen · 2022-06-22T22:36:55Z

I like the idea of simply exposing a number of ipConfigs too. Definitely makes for a nicer UX. Maybe just have a couple integers? privateIPConfigs: 30 and publicIPConfigs: 1 or something like that?

jackfrancis · 2022-06-23T09:37:51Z

api/v1beta1/azuremachine_webhook.go

@@ -136,6 +136,20 @@ func (m *AzureMachine) ValidateUpdate(oldRaw runtime.Object) error {
 		)
 	}

+	if !reflect.DeepEqual(m.Spec.SubnetName, old.Spec.SubnetName) {


Is this new immutability requirement for SubnetName closely related to this change? Should it be a separate PR?

The immutability isn't strictly changed with these changes, however it appeared to be an oversight I believe while looking at managing and validating the new interface level field?

api/v1beta1/azuremachine_webhook_test.go

azure/scope/machine.go

jackfrancis · 2022-06-23T10:11:00Z

azure/scope/machine.go

+		spec.SubnetName = n.SubnetName
+
+		// Check for control plane interface setup on interface 0
+		if m.Role() == infrav1.ControlPlane && i == 0 {


Do we have a guarantee that the first NetworkInterface in the m.AzureMachine.Spec.NetworkInterfaces slice will always be eth0?

I wondered about this as well. However, in my testing so far, I have been unable to cause the Linux interface order within the default CAPZ Ubuntu image to be anything other than matching the interface order in the Azure APIs. I have not had any luck so far on identifying the mechanism that is ensuring this though. I believe the ordering is consistent due to the behavior of the kernel module for the virtual network interface card, but I haven't specifically found the logic to say that definitively.

Is there any other more reliable indication (e.g., n.Name == "eth0") that could be used?

Is there a feature of the kernel driver, cloud-init, or netplan that allows us to take advantage of that convention? Do you happen to have a link to some docs or code that can lend clarity there?

We do have a guarantee that the first network interface in the list from the user-specified yaml is the first interface in the Azure API. We are taking advantage of the inherent stability of Go slice element order to ensure that part doesn't get mixed up. The only part I can't say for certain is that the first interface in the slice will always be eth0 in the Linux OS. I do believe it to be the case, and it is the most reliable mechanism I have found so far. I have been unable to trigger (even after trying many times) eth0 in the Linux OS being anything other than the first NetworkInterface in the slice. I think this is likely the best way to signal to the OS the desired linux interface order, since the order in the API would then map to the ethX index in the OS 1 to 1.

Yeah I would definitely prefer a more reliable indication. Maybe we could make a helper function in the scope for getting eth0? That way it's less error prone and more readable.

Can you clarify what you are proposing? Utilizing the interface index appears to be the only current reliable way of ensuring your intended interface order in the OS.

Oh re-reading your comment I think I understand. The intent is to improve the code readability. Makes sense to me to have a helper function.

I agree that if we are able to guarantee ordinality, that the first interface reported from the linux API is going to be eth0. So a convenience func + UT that ensures that ordinality should be sufficient.

Thanks!

brianlieberman · 2022-06-24T15:32:58Z

Reworked the IPConfig functionality to use integer quantities and updated the example in the PR description

Jont828

Thanks for working on this PR, something that touches this much code definitely takes a while to put together. Here are some suggestions on some tweaks to make.

Jont828 · 2022-06-27T23:00:31Z

api/v1beta1/azuremachine_types.go

@@ -114,6 +114,8 @@ type AzureMachineSpec struct {
 	// SubnetName selects the Subnet where the VM will be placed
 	// +optional
 	SubnetName string `json:"subnetName,omitempty"`
+
+	NetworkInterfaces []AzureNetworkInterface `json:"networkInterfaces,omitempty"`


Should this be an optional field? Just to clarify, if apply an existing template w/o the NetworkInterfaces field do we want it to give an error or just create an empty slice?

Also we're going to want to add a header comment for this as well.

+1 to making this optional

Can you also please add a clarifying comment about what happens with Network Interfaces are not specified?

Jont828 · 2022-06-27T23:07:31Z

api/v1beta1/types.go

+// Network Interfaces to attach to each VM
+// +optional
+type AzureNetworkInterface struct {
+	// The subnet to place the interface in


Suggested change

// The subnet to place the interface in

// The subnet to place the interface in.

Could we end the comments with periods? Not sure if the linters will complain but it's more consistent in any case.

Jont828 · 2022-06-27T23:09:38Z

api/v1beta1/types.go

+
+// IP Configuration defines options to confiure a network interface.
+type AzureIPConfig struct {
+	PrivateIP       string `json:"privateIP,omitempty"`


Can we add comments for these fields as well?

Jont828 · 2022-06-27T23:09:52Z

api/v1beta1/types.go

+	ID string `json:"id,omitempty"`
+}
+
+// IP Configuration defines options to confiure a network interface.


Suggested change

// IP Configuration defines options to confiure a network interface.

// AzureIPConfig defines options to configure a network interface.

Jont828 · 2022-06-27T23:10:58Z

api/v1beta1/types.go

@@ -579,6 +579,37 @@ type SubnetSpec struct {
 	SubnetClassSpec `json:",inline"`
 }

+// Network Interfaces to attach to each VM


Suggested change

// Network Interfaces to attach to each VM

// AzureNetworkInterface defineds a network interface to attach to each VM.

Typically we want to start the documentation comment with the name of the type/function. I believe the linters will complain about it if it doesn't match.

Jont828 · 2022-06-27T23:27:27Z

azure/services/scalesets/scalesets_test.go

+							Subnet: &compute.APIEntityReference{
+								ID: to.StringPtr("/subscriptions/123/resourceGroups/my-rg/providers/Microsoft.Network/virtualNetworks/my-vnet/subnets/subnet2"),
+							},
+							//LoadBalancerBackendAddressPools: &[]compute.SubResource{{ID: to.StringPtr("/subscriptions/123/resourceGroups/my-rg/providers/Microsoft.Network/loadBalancers/capz-lb/backendAddressPools/backendPool")}},


Jont828 · 2022-06-27T23:29:39Z

exp/api/v1alpha4/azuremachinepool_conversion.go

@@ -17,6 +17,7 @@ limitations under the License.
 package v1alpha4

 import (
+	machineryConversion "k8s.io/apimachinery/pkg/conversion"


Suggested change

machineryConversion "k8s.io/apimachinery/pkg/conversion"

apiMachineryConversion "k8s.io/apimachinery/pkg/conversion"

Nit: for consistency with import name in other conversions.

Jont828 · 2022-06-27T23:30:57Z

exp/api/v1beta1/azuremachinepool_types.go

@@ -87,6 +87,10 @@ type (
 		// SubnetName selects the Subnet where the VMSS will be placed
 		// +optional
 		SubnetName string `json:"subnetName,omitempty"`
+
+		// Network Interfaces to attach to the to a virtual machine


Suggested change

// Network Interfaces to attach to the to a virtual machine

// NetworkInterfaces specifies the network interfaces to attach to the to a virtual machine.

exp/api/v1beta1/azuremachinepool_webhook.go

azure/services/networkinterfaces/spec.go

invidian · 2022-07-05T12:32:29Z

/ok-to-test

CecileRobertMichon · 2022-07-05T22:06:22Z

api/v1beta1/types.go

@@ -579,6 +579,42 @@ type SubnetSpec struct {
 	SubnetClassSpec `json:",inline"`
 }

+// AzureNetworkInterface defineds a network interface.
+// +optional
+type AzureNetworkInterface struct {


Let's just call this NetworkInterface. It's assumed that everything in here is in the context of Azure as this is the Azure provider and the types are used in AzureMachines and AzureClusters.

CecileRobertMichon · 2022-07-05T22:06:32Z

api/v1beta1/types.go

@@ -579,6 +579,42 @@ type SubnetSpec struct {
 	SubnetClassSpec `json:",inline"`
 }

+// AzureNetworkInterface defineds a network interface.


Suggested change

// AzureNetworkInterface defineds a network interface.

// AzureNetworkInterface defines a network interface.

CecileRobertMichon · 2022-07-05T22:07:03Z

api/v1beta1/types.go

+}
+
+// AzureIPConfig defines options to confiure a network interface.
+type AzureIPConfig struct {


same here, preferrence forIPConfig for consistency

CecileRobertMichon · 2022-07-05T22:07:20Z

api/v1beta1/types.go

+	ID string `json:"id,omitempty"`
+}
+
+// AzureIPConfig defines options to confiure a network interface.


Suggested change

// AzureIPConfig defines options to confiure a network interface.

// AzureIPConfig defines options to configure a network interface.

CecileRobertMichon · 2022-07-05T22:08:50Z

api/v1beta1/azuremachine_types.go

@@ -114,6 +114,8 @@ type AzureMachineSpec struct {
 	// SubnetName selects the Subnet where the VM will be placed
 	// +optional
 	SubnetName string `json:"subnetName,omitempty"`
+
+	NetworkInterfaces []AzureNetworkInterface `json:"networkInterfaces,omitempty"`


+1 to making this optional

Can you also please add a clarifying comment about what happens with Network Interfaces are not specified?

CecileRobertMichon · 2022-07-05T22:39:32Z

api/v1beta1/types.go

+	// The subnet to place the interface in.
+	SubnetName string `json:"subnetName,omitempty"`
+
+	// Number of private IP address to attach to the interface.


Suggested change

// Number of private IP address to attach to the interface.

// Number of private IP addresses to attach to the interface.

CecileRobertMichon · 2022-07-05T22:40:34Z

api/v1beta1/types.go

+	// +optional
+	PrivateIPConfigs int `json:"privateIPConfigs,omitempty"`
+
+	// Number of public IP addresses to attach to the interface.


Nit: comment name should always start with field name (not sure if linter will catch this?)

same thing applies to other type comments in the file

CecileRobertMichon · 2022-07-05T22:43:06Z

api/v1beta1/types.go

+}
+
+// AzureIPConfig defines options to confiure a network interface.
+type AzureIPConfig struct {


where is this type being used? I can't find a reference to it

CecileRobertMichon · 2022-07-05T22:46:30Z

azure/scope/machine.go

@@ -660,7 +716,7 @@ func (m *MachineScope) SetSubnetName() error {
 				subnetName = subnet.Name
 			}
 		}
-		if subnetCount == 0 || subnetCount > 1 || subnetName == "" {
+		if (subnetCount == 0 || subnetCount > 1 || subnetName == "") && len(m.AzureMachine.Spec.NetworkInterfaces) == 0 {


if len(m.AzureMachine.Spec.NetworkInterfaces) != 0, this function would cause the AzureMachine subnet name to get set, which is not a valid per webhook validation as subnet name cannot be set if network interfaces are defined. Am I missing something?

(feel free to ignore this comment if you implement my above comment about deprecating AzureMachine.Spec.SubnetName)

CecileRobertMichon · 2022-07-05T22:49:19Z

azure/scope/machine.go

 		AcceleratedNetworking: m.AzureMachine.Spec.AcceleratedNetworking,
 		IPv6Enabled:           m.IsIPv6Enabled(),
 		EnableIPForwarding:    m.AzureMachine.Spec.EnableIPForwarding,
+		SubnetName:            m.Subnet().Name,


not sure why this is preferable, we're looping over all the subnets to find the one whose Name matches the AzureMachine SubnetName so it's equivalent but more costly than just using m.AzureMachine.Spec.SubnetName

CecileRobertMichon · 2022-07-05T23:02:52Z

azure/scope/machine.go

+			continue
+		}
+		if i == 0 {
+			spec = m.DefaultNICSpec()


it's unclear to me why we're using the defaults for index 0 but not for the others. What's significant about the order the network interfaces are specified in? What's different between the default spec and the fields we're setting below in the else statement?

CecileRobertMichon · 2022-07-05T23:07:34Z

azure/scope/machine.go

+	for i, n := range m.AzureMachine.Spec.NetworkInterfaces {
+		var spec *networkinterfaces.NICSpec
+		if n.ID != "" {
+			continue


this deviates significantly from the pattern we've established for other BYO existing resource in CAPZ. The way it works for other resources is that the reconciler will add a tag on each resource it creates to allow it to identify the resources it created vs. the ones the user pre-created. See the virtual networks as an example. There are pros and cons for both ways of doing it, however because this is creating an inconsistent user experience I would recommend putting BYO network interface as out of scope for this PR and tackling it in its own issue/PR so we can iterate on the design.

linux-foundation-easycla · 2022-07-15T15:25:17Z

The committers listed above are authorized under a signed CLA.

✅ login: brianlieberman / name: Brian Lieberman (a9b3b46, c19ddea, 6e20ef6, 2a6e944, 9afbebe, 3e5a2f6, 7bd6b24, 9ab3955, 3685380, 3a90933, 4ff1d2a, 77e3bfc, 6567927, 6a18007, a38a8d3, a6ba4e9, 24b09a8, 7cf1d09, ff3eae1, 2723c89, c902221, d90488f, 230d88f)
✅ login: dthorsen / name: Dane Thorsen (3800b8b, 6095d7b, 5a45826, be0b6cb, 05fc0d6)
✅ login: mytunguyen / name: Mytu (a362d56, 89f0f71)
✅ login: BrennenMM7 / name: Brennen Murray (4600f18)

CecileRobertMichon

LGTM pending few minor comments

/assign @nawazkh for another round of review

Docs are missing but will work on docs + e2e tests with @nawazkh as part of #467

@dthorsen thanks so much for doing the refactor and staying with us. Let's get this merged before January 10th so it can be included in v1.7.0. Could you please squash commits?

api/v1beta1/azuremachine_webhook.go

azure/defaults.go

azure/scope/machine.go

azure/types.go

dthorsen · 2022-12-23T20:00:42Z

/retest

nawazkh

Jotted down my thoughts on another pass. Please take a look!

api/v1beta1/azuremachine_types.go

api/v1beta1/types.go

api/v1beta1/azuremachine_default.go

nawazkh · 2022-12-28T23:10:56Z

api/v1beta1/azuremachinetemplate_webhook.go

@@ -128,5 +132,6 @@ func (r *AzureMachineTemplate) Default(ctx context.Context, obj runtime.Object)
 	}
 	t.Spec.Template.Spec.SetDefaultCachingType()
 	t.Spec.Template.Spec.SetDataDisksDefaults()
+	t.Spec.Template.Spec.SetNetworkInterfacesDefaults()


Did we miss updating the validateUpdate function for the new NetworkInterface updates?

azuremachinetemplate is immutable. And the new defaulting behavior should be permitted already through the immutability by the logic here

Thanks for pinning that logic!

nawazkh · 2022-12-28T23:12:09Z

api/v1beta1/azuremachinetemplate_webhook_test.go

+							DataDisks:             []DataDisk{},
+							SSHPublicKey:          "fake ssh key",
+							SubnetName:            "subnet1",
+							AcceleratedNetworking: to.BoolPtr(true),


Can we also add a test with AcceleratedNetworking: to.BoolPtr(true) and SubnetName is '' and NetworkInterface is not nil?

nawazkh · 2022-12-29T01:27:20Z

azure/services/networkinterfaces/spec.go

+	PrivateIP       string
+	PublicIP        bool
+	PublicIPAddress string


Can we instead convert IPConfig struct as below? We will have one less field to manage and a nil pointer will intrinsicly imply weather it is assigned or not?

Suggested change

PrivateIP string

PublicIP bool

PublicIPAddress string

PrivateIP *string

PublicIPAddress *string

controllers/azuremachine_reconciler.go

exp/controllers/azuremachinepool_reconciler.go

CecileRobertMichon · 2023-01-03T19:33:02Z

@nawazkh thank you for the review 🙏

@dthorsen happy new year! please let me know once all comments have been addressed and this is ready for a final pass

dthorsen · 2023-01-05T23:00:56Z

/retest

CecileRobertMichon

/lgtm

CecileRobertMichon · 2023-01-06T02:01:56Z

@nawazkh are all your comments resolved?

If so @dthorsen can you please squash one more time?

dthorsen · 2023-01-06T15:53:51Z

@nawazkh I believe I addressed all of your comments. Let me know if you have any other comments before I squash and push commits. Thanks!

nawazkh

Mega job @dthorsen ! Thank you for patiently working on this and addressing all the nits 🙌🏼
/lgtm

Co-authored-by: Dane Thorsen <dthorsen@users.noreply.github.com>

nawazkh · 2023-01-06T19:21:03Z

Persisting lgtm
/lgtm

dthorsen · 2023-01-06T19:33:14Z

/retest

k8s-ci-robot · 2023-01-06T20:27:43Z

@brianlieberman: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cluster-api-provider-azure-coverage	`230d88f`	link	false	`/test pull-cluster-api-provider-azure-coverage`
pull-cluster-api-provider-azure-e2e-exp	`93305ad`	link	false	`/test pull-cluster-api-provider-azure-e2e-exp`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

CecileRobertMichon · 2023-01-06T23:11:25Z

both of these test failures are known flakes getting worked on independently :(

/retest
/approve

k8s-ci-robot · 2023-01-06T23:11:36Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CecileRobertMichon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [CecileRobertMichon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. kind/feature Categorizes issue or PR as related to a new feature. labels Jun 22, 2022

k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jun 22, 2022

k8s-ci-robot requested review from CecileRobertMichon and Jont828 June 22, 2022 20:43

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Jun 22, 2022

jackfrancis reviewed Jun 23, 2022

View reviewed changes

api/v1beta1/azuremachine_webhook_test.go Outdated Show resolved Hide resolved

jackfrancis reviewed Jun 23, 2022

View reviewed changes

azure/scope/machine.go Outdated Show resolved Hide resolved

jackfrancis reviewed Jun 23, 2022

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jun 27, 2022

Jont828 requested changes Jun 27, 2022

View reviewed changes

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 1, 2022

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 5, 2022

CecileRobertMichon reviewed Jul 5, 2022

View reviewed changes

k8s-ci-robot removed the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jul 15, 2022

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 22, 2022

CecileRobertMichon reviewed Dec 22, 2022

View reviewed changes

dthorsen force-pushed the main branch from bd78d75 to 4bec8b8 Compare December 22, 2022 21:20

nawazkh reviewed Dec 29, 2022

View reviewed changes

nawazkh mentioned this pull request Jan 3, 2023

Log error when a SKU does not support AccelaratedNetworking #2979

Open

CecileRobertMichon reviewed Jan 6, 2023

View reviewed changes

k8s-ci-robot assigned CecileRobertMichon Jan 6, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 6, 2023

nawazkh approved these changes Jan 6, 2023

View reviewed changes

✨ Add support for multiple network interfaces

93305ad

Co-authored-by: Dane Thorsen <dthorsen@users.noreply.github.com>

dthorsen force-pushed the main branch from 13530e7 to 93305ad Compare January 6, 2023 19:15

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 6, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 6, 2023

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 6, 2023

k8s-ci-robot merged commit 6563b7d into kubernetes-sigs:main Jan 7, 2023

nawazkh mentioned this pull request Feb 1, 2023

Add subnetName support to ammp #3132

Merged

3 tasks

mweibel mentioned this pull request Mar 6, 2023

MachinePools/VMSS NetworkInterfaces wrong configuration #3224

Closed

CecileRobertMichon mentioned this pull request Mar 20, 2023

Refactor scalesets NIC config #3188

Merged

3 tasks

This was referenced Mar 21, 2023

Update Azure VM Images to enable DHCP on eth0 instead of using Azures IMDS endpoint #3342

Closed

add e2e test coverage for Azure CNI v1 using single NIC per linux node #3343

Closed

	// The subnet to place the interface in
	// The subnet to place the interface in.

	// IP Configuration defines options to confiure a network interface.
	// AzureIPConfig defines options to configure a network interface.

	// Network Interfaces to attach to each VM
	// AzureNetworkInterface defineds a network interface to attach to each VM.

	machineryConversion "k8s.io/apimachinery/pkg/conversion"
	apiMachineryConversion "k8s.io/apimachinery/pkg/conversion"

	// Network Interfaces to attach to the to a virtual machine
	// NetworkInterfaces specifies the network interfaces to attach to the to a virtual machine.

	// AzureNetworkInterface defineds a network interface.
	// AzureNetworkInterface defines a network interface.

	// Number of private IP address to attach to the interface.
	// Number of private IP addresses to attach to the interface.

Support for configurable Network Interfaces #2411

Support for configurable Network Interfaces #2411

Conversation

brianlieberman commented Jun 22, 2022 • edited by CecileRobertMichon

linux-foundation-easycla bot commented Jun 22, 2022 • edited

k8s-ci-robot commented Jun 22, 2022

CecileRobertMichon commented Jun 22, 2022

brianlieberman commented Jun 22, 2022

dthorsen commented Jun 22, 2022 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

brianlieberman commented Jun 24, 2022

Jont828 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

invidian commented Jul 5, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

linux-foundation-easycla bot commented Jul 15, 2022 • edited

CecileRobertMichon left a comment

Choose a reason for hiding this comment

dthorsen commented Dec 23, 2022

nawazkh left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

CecileRobertMichon commented Jan 3, 2023

dthorsen commented Jan 5, 2023

CecileRobertMichon left a comment

Choose a reason for hiding this comment

CecileRobertMichon commented Jan 6, 2023

dthorsen commented Jan 6, 2023

nawazkh left a comment

Choose a reason for hiding this comment

nawazkh commented Jan 6, 2023

dthorsen commented Jan 6, 2023

k8s-ci-robot commented Jan 6, 2023 • edited

CecileRobertMichon commented Jan 6, 2023

k8s-ci-robot commented Jan 6, 2023

brianlieberman commented Jun 22, 2022 •

edited by CecileRobertMichon

linux-foundation-easycla bot commented Jun 22, 2022 •

edited

dthorsen commented Jun 22, 2022 •

edited

linux-foundation-easycla bot commented Jul 15, 2022 •

edited

k8s-ci-robot commented Jan 6, 2023 •

edited