Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reload VF driver only when rdma is enabled #207

Merged
merged 1 commit into from Dec 2, 2021

Conversation

pliurh
Copy link
Collaborator

@pliurh pliurh commented Nov 22, 2021

To set the GUID to VF, we need to set the admin MAC and reload the VF driver. It will take a long time if users set big 'numVFs'. This PR changes the logic to only reload the VF driver when isRdma is set. So that, users not using ROCE will no longer need to wait for the VF driver reload.

@pliurh
Copy link
Collaborator Author

pliurh commented Nov 22, 2021

/cc @Mmduh-483

@Mmduh-483
Copy link
Contributor

@moshe010 @adrianchiris

@zshi-redhat
Copy link
Collaborator

@pliurh Will this fix be backported to lower versions? If so, I'd prefer to not changing API. My understanding is we will eventually remove this once kernel fix is in place.

@pliurh
Copy link
Collaborator Author

pliurh commented Nov 22, 2021

@pliurh Will this fix be backported to lower versions? If so, I'd prefer to not changing API. My understanding is we will eventually remove this once kernel fix is in place.

If we want to fix that 4.6 bug, we need to backport it. Without modifying API, the config daemon cannot know if a VF will be used for ROCE. I don't know when it will be fixed in the kernel.

func setVfsGuid(iface *sriovnetworkv1.Interface) error {
glog.Infof("setVfsGuid(): device %s", iface.PciAddress)
pfLink, err := netlink.LinkByName(iface.Name)
func setVfsGuid(vfAddr string, pfLink netlink.Link) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo the name: setVfGUID or setVfGuid is more appropriate now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

func setVfsAdminMac(iface *sriovnetworkv1.InterfaceExt) error {
glog.Infof("setVfsAdminMac(): device %s", iface.PciAddress)
pfLink, err := netlink.LinkByName(iface.Name)
func setVfsAdminMac(vfAddr string, pfLink netlink.Link, isRdma bool) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo the name: setVFAdminMac or setVFAdminMAC is more appropriate now.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

return err
}
if isRdma {
glog.Infof("setVfsAdminMac(): unbind driver for %s", vfAddr)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add a method to handle the driver unbind and move the unbind logic out of SetVfAdminMac and SetVfGuid ?
so that these functions do exactly what their name say ?

e.g add a function : UnbindDriverIfNeeded() and call it from configureSriovDevice ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Collaborator

@adrianchiris adrianchiris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Peng !

When merging please add the description of this PR to the merge commit so its clear why this change was introduced

To set the GUID to VF, we need to set the admin MAC and reload the
VF driver. It will take a long time if users set big 'numVFs'. This
PR changes the logic to only reload the VF driver when isRdma is
set. So that, users not using ROCE will no longer need to wait for
the VF driver reload.
@pliurh pliurh merged commit ff8c2f1 into k8snetworkplumbingwg:master Dec 2, 2021
e0ne added a commit to e0ne/sriov-network-operator that referenced this pull request Mar 28, 2022
SriovNetworkNodeState is outdated. IsRdma field was introduced
in the PR k8snetworkplumbingwg#207.

This patch syncs CRDs between config/crd/bases and helm chart.
zeeke pushed a commit to zeeke/sriov-network-operator-1 that referenced this pull request May 3, 2022
SriovNetworkNodeState is outdated. IsRdma field was introduced
in the PR k8snetworkplumbingwg#207.

This patch syncs CRDs between config/crd/bases and helm chart.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants