Add support for running in a virtual machine -- Remove PF dependency #369

atyronesmith · 2020-10-07T20:00:52Z

This PR adds support for running (initially) in an OpenShift on OpenStack environment. In a VM environment, only VFs are attached (no PF) and the VFs represent an underlying OpenStack network. This PR removes the necessity of accessing a PF and uses additional information passed into the VM by the OpenStack metadata service to include underlying network mappings in the sriovnetworknodestate. In addition, the exposed underlying network information can be used to select networks for a policy.

This PR is in conjunction with a PR in k8snetworkplumbingwg/sriov-network-device-plugin repository.

This PR is to allow a review of the concept.

zshi-redhat

Thanks Aaron for working on the patches, added few comments inline.
I think the overall design looks good to me:

A virtual plugin for VM use case
Auto detect platform type and invoke the virtual plugin

deploy/crds/sriovnetwork.openshift.io_sriovnetworknodepolicies_crd.yaml

pkg/utils/utils_virtual.go

zshi-redhat · 2020-10-21T04:58:36Z

pkg/utils/utils_virtual.go

+		iface.TotalVfs = 1
+		iface.NumVfs = 1
+
+		vf := sriovnetworkv1.VirtualFunction{


Should we consider the VM VF as a individual device (just like a regular PF device in the host) and not populate the vf fields?

We could modify current numVfs range to allow 0 numVfs be configured.

I think it would be good to treat VFs as individual devices. The problem is that the current code seems to be keyed to TotalVFs/NumVfs and I took the approach of setting to 1 to reduce the number code changes. I can take a look at your suggestion.

pkg/utils/utils.go

zshi-redhat

Looks generally good to me.
I think we might want to do two things in a follow-up PRs:

enable use of numVfs=0 so that virtual-plugin doesn't need to populate InterfaceExt.VFs
move platform specific code into a separate file.

controllers/sriovnetworknodepolicy_controller.go

pkg/daemon/daemon.go

zshi-redhat · 2020-11-16T02:58:41Z

/cc @pliurh

pkg/plugins/virtual/virtual_plugin.go

atyronesmith · 2020-11-17T14:22:17Z

/retest

pkg/daemon/plugin.go

pliurh · 2020-11-18T06:36:12Z

pkg/daemon/writer.go

+	var err error
+
+	if platformType == Virtual {
+		iface, err = utils.DiscoverSriovDevicesVirtual()


Why not use one function to discover devices for both baremetal and virtual nodes? We can pass the platformType to DiscoverSriovDevices so that the VFs can be treated differently on a virtual node.

I wanted to completely separate the virtual path from the baremetal path. I didn't think it was clean to include a bunch of checks in the discover function. I can change if you feel strongly about it...

pkg/plugins/virtual/virtual_plugin.go

pkg/utils/utils_virtual.go

format

atyronesmith

Completed

pkg/daemon/daemon.go

pkg/plugins/virtual/virtual_plugin.go

atyronesmith · 2020-11-18T14:26:50Z

pkg/daemon/writer.go

+	var err error
+
+	if platformType == Virtual {
+		iface, err = utils.DiscoverSriovDevicesVirtual()


I wanted to completely separate the virtual path from the baremetal path. I didn't think it was clean to include a bunch of checks in the discover function. I can change if you feel strongly about it...

pkg/plugins/virtual/virtual_plugin.go

pkg/utils/utils_virtual.go

pliurh · 2020-11-25T07:05:47Z

/lgtm

openshift-ci-robot · 2020-11-25T07:06:10Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: atyronesmith, pliurh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [pliurh]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

config-daemon: fix NM udev path

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 7, 2020

openshift-ci-robot requested review from fepan and zshi-redhat October 7, 2020 20:01

atyronesmith mentioned this pull request Oct 7, 2020

Add support for running inside a VM with vfio-noiommu k8snetworkplumbingwg/sriov-network-device-plugin#272

Merged

zshi-redhat reviewed Oct 21, 2020

View reviewed changes

martinkennelly reviewed Oct 23, 2020

View reviewed changes

pkg/utils/utils.go Outdated Show resolved Hide resolved

atyronesmith mentioned this pull request Oct 26, 2020

Add additional extended Selectors to for networks in virtual deployments. k8snetworkplumbingwg/sriov-network-device-plugin#277

Closed

atyronesmith changed the title ~~WIP -- Add support for running in a virtual machine -- Remove PF dependency~~ Add support for running in a virtual machine -- Remove PF dependency Nov 6, 2020

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Nov 6, 2020

atyronesmith mentioned this pull request Nov 6, 2020

Add support for incorporating metadata from an OpenStack-based virtual deployment #399

Closed

atyronesmith requested a review from zshi-redhat November 10, 2020 01:42

atyronesmith force-pushed the master branch 3 times, most recently from 6c9ddd6 to 29a1f46 Compare November 12, 2020 15:23

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 12, 2020

atyronesmith force-pushed the master branch 6 times, most recently from 794990d to b641804 Compare November 12, 2020 15:58

openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 12, 2020

zshi-redhat reviewed Nov 16, 2020

View reviewed changes

controllers/sriovnetworknodepolicy_controller.go Outdated Show resolved Hide resolved

pkg/daemon/daemon.go Outdated Show resolved Hide resolved

pkg/daemon/daemon.go Show resolved Hide resolved

atyronesmith force-pushed the master branch 2 times, most recently from ec2b90a to 69376a8 Compare November 16, 2020 02:30

openshift-ci-robot requested a review from pliurh November 16, 2020 02:58

zshi-redhat reviewed Nov 16, 2020

View reviewed changes

pkg/plugins/virtual/virtual_plugin.go Show resolved Hide resolved

pliurh reviewed Nov 18, 2020

View reviewed changes

Add support for VMS -- Remove PF dependency

dfcbe28

atyronesmith force-pushed the master branch from 69376a8 to dfcbe28 Compare November 18, 2020 20:27

Additional cleanup

2f8a221

format

atyronesmith force-pushed the master branch from 77a59db to 2f8a221 Compare November 20, 2020 15:47

atyronesmith commented Nov 20, 2020

View reviewed changes

openshift-ci-robot assigned pliurh Nov 25, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Nov 25, 2020

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 25, 2020

openshift-merge-robot merged commit b6967d0 into openshift:master Nov 25, 2020

jerpeter1 pushed a commit to jerpeter1/sriov-network-operator that referenced this pull request Mar 7, 2023

Merge pull request openshift#369 from zeeke/udev-fix

54c377f

config-daemon: fix NM udev path

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for running in a virtual machine -- Remove PF dependency #369

Add support for running in a virtual machine -- Remove PF dependency #369

atyronesmith commented Oct 7, 2020

zshi-redhat left a comment

zshi-redhat Oct 21, 2020

zshi-redhat Oct 21, 2020

atyronesmith Oct 21, 2020

zshi-redhat left a comment

zshi-redhat commented Nov 16, 2020

atyronesmith commented Nov 17, 2020

pliurh Nov 18, 2020

atyronesmith Nov 18, 2020

atyronesmith left a comment

atyronesmith Nov 18, 2020

pliurh commented Nov 25, 2020

openshift-ci-robot commented Nov 25, 2020

Add support for running in a virtual machine -- Remove PF dependency #369

Add support for running in a virtual machine -- Remove PF dependency #369

Conversation

atyronesmith commented Oct 7, 2020

zshi-redhat left a comment

Choose a reason for hiding this comment

zshi-redhat Oct 21, 2020

Choose a reason for hiding this comment

zshi-redhat Oct 21, 2020

Choose a reason for hiding this comment

atyronesmith Oct 21, 2020

Choose a reason for hiding this comment

zshi-redhat left a comment

Choose a reason for hiding this comment

zshi-redhat commented Nov 16, 2020

atyronesmith commented Nov 17, 2020

pliurh Nov 18, 2020

Choose a reason for hiding this comment

atyronesmith Nov 18, 2020

Choose a reason for hiding this comment

atyronesmith left a comment

Choose a reason for hiding this comment

atyronesmith Nov 18, 2020

Choose a reason for hiding this comment

pliurh commented Nov 25, 2020

openshift-ci-robot commented Nov 25, 2020