verbs: Add an API to retrieve IB device number of ports via netlink

[yishaih]

This series adds a verbs API to retrieve IB device number of ports via netlink.

The new API is used by mlx5 in the DR area.

[jgunthorpe]

Why not make the verb return the query device value if netlink isn't support and forget about the dv API?

[mark-bloch]

Why not make the verb return the query device value if netlink isn't support and forget about the dv API?

What dv API?
Are you talking about the DR code? this is just internal implementation which isn't exposed.

About the verb, the current definition of "netlink based number of ports" is very clear and creates a separate API from the regular query device (phys_port_cnt) which is limited to u8.

Of course doing that (your comment) will allow us to say "don't user phys_port_cnt/ibv_query_device() and move to use ibv_get_device_num_ports()) are we okay with saying that?

[jgunthorpe]

Yes move the logic to call to the device information to the ibv_query_device() and get rid of the dr_ function that tries to do that. We probably need to pass in the ibv_context instead of the device to make that work. Netlink or not netlink is too complicated a detail to expose as an API. This should always return the port count and be the only way to get a port count if it is larger than u8

[mark-bloch]

Yes move the logic to call to the device information to the ibv_query_device() and get rid of the dr_ function that tries to do that. We probably need to pass in the ibv_context instead of the device to make that work. Netlink or not netlink is too complicated a detail to expose as an API. This should always return the port count and be the only way to get a port count if it is larger than u8

Okay, so as ibv_query_device_ex() can fail as well so DR will need some logic anyway.
I think I better expose a new field (phys_port_cnt_ex) in struct ibv_device_attr_ex which is u32 bits.
If the netlink value is valid (!= 0) fill that, if not just fill phys_port_cnt.

This way an application can always use the new phys_port_cnt_ex value and there is no need for a new verb.

[rleon]

Yes move the logic to call to the device information to the ibv_query_device() and get rid of the dr_ function that tries to do that. We probably need to pass in the ibv_context instead of the device to make that work. Netlink or not netlink is too complicated a detail to expose as an API. This should always return the port count and be the only way to get a port count if it is larger than u8

Okay, so as ibv_query_device_ex() can fail as well so DR will need some logic anyway.
I think I better expose a new field (phys_port_cnt_ex) in struct ibv_device_attr_ex which is u32 bits.
If the netlink value is valid (!= 0) fill that, if not just fill phys_port_cnt.

This way an application can always use the new phys_port_cnt_ex value and there is no need for a new verb.

Always fill that new field and encourage users to always read.

Something like phys_port_cnt_real ???

[mark-bloch]

Yes move the logic to call to the device information to the ibv_query_device() and get rid of the dr_ function that tries to do that. We probably need to pass in the ibv_context instead of the device to make that work. Netlink or not netlink is too complicated a detail to expose as an API. This should always return the port count and be the only way to get a port count if it is larger than u8

Okay, so as ibv_query_device_ex() can fail as well so DR will need some logic anyway.
I think I better expose a new field (phys_port_cnt_ex) in struct ibv_device_attr_ex which is u32 bits.
If the netlink value is valid (!= 0) fill that, if not just fill phys_port_cnt.
This way an application can always use the new phys_port_cnt_ex value and there is no need for a new verb.

Always fill that new field and encourage users to always read.

Something like phys_port_cnt_real ???

Yea, that's the idea, it will look something like this (in ibv_cmd_query_device_any()):

	if (attr_size >= offsetofend(struct ibv_device_attr_ex, phys_port_cnt_ex)) {
		struct verbs_sysfs_dev *sysfs_dev = verbs_get_device(context->device)->sysfs;

		if (sysfs_dev->num_ports)
			attr->phys_port_cnt_ex = sysfs_dev->num_ports;
		else
			attr->phys_port_cnt_ex = attr->orig_attr.phys_port_cnt;
	}

so if the netlink value is there (sysfs_dev->num_ports) use it, otherwise just use the legacy phys_port_cnt.
About the name I was thinking about phys_port_cnt_ex.

[yishaih]

The PR was updated as was discussed above, thanks.