How to attach Click-DPDK to ovs-DPDK via dpdkring?

[kthkaya]

Hi

As a continuation of #54, and the same approach as this, I am trying to attach my click process (two ports)
to two ports of ovs.

Following ovs guide, I I add the following two lines and create two ports in form of dpdkring.

ovs-vsctl add-port br0 dpdkr0 -- set Interface dpdkr0 type=dpdkr
ovs-vsctl add-port br0 dpdkr1 -- set Interface dpdkr1 type=dpdkr

As for click side,

fastclick/elements/userlevel/todpdkring.hh

Lines 38 to 48 in cbd995f

	=item FROM_PROC
	String. The name of the Click-DPDK process that uses this element.
	It can be a made-up name, however, the other end (a FromDPDKRing)
	must use the same name at its own TO_PROC parameter.
	=item TO_PROC
	String. The name of the Click-DPDK process that is attached to our process.
	It can be a made-up name, however, the other end (a FromDPDKRing)
	must use the same name at its own FROM_PROC parameter.

above comment leaves the impression that the other process must be a click process too. In ovs case, should I type in the name of the ovs process?

I would appreciate if you could share sample commands/configuration related to this scenario.

[gkatsikas]

I can only answer to your first question, as I haven't tried OVS.
The two processes need not be Click-based. All you need is that both call the same DPDK functions for creating (primary) or attaching to (secondary) a memory pool. To do so, each process' function call needs to use consistent names that characterize the memory pool. This is symmetric as the memory pool's name of the Tx side needs to be the same as the name of the memory pool of the other process' Rx side.

[kthkaya]

Thanks Georgios, I will investigate and see if I can manage connecting to OVS.

Does click/fastclick support DPDK 17.11? I tried to compile ovs on 17.05 but no avail. Fastclick did compile on 17.11 and ran a sample DPDK click process successfully, so can I take it as a yes?

How have you worked around this? veth?

[gkatsikas]

FastClick supports DPDK 17.11 and 18.02 is coming soon.

That topic from mainstream Click was the reason that I implemented the From/ToDPDKRing elements.
However I did not try to use these elements with OVS; instead I implemented service chaining only in FastClick (by connecting NFs through these shared ring buffers).

The floor is yours now to find out how to connect FastClick with OVS :)

[tbarbette]

I'm actually pretty sure I did it :p But two years ago... I'm going to give it a shot ASAP.

[kthkaya]

Fingers crossed...

[kthkaya]

Alternatively, vhost-user ports look like an acceptable option.

ovs-vsctl add-port br0 vhost-user-1 -- set Interface vhost-user-1 type=dpdkvhostuser
This action creates a socket located at /usr/local/var/run/openvswitch/vhost-user-1

Following is how dpdk testpmd picks up the socket

testpmd ....... --vdev=virtio_user0,path=/usr/local/var/run/openvswitch/vhost-user-1

Is there any element in click to treat vhost-user socket as a device and read/write to it?

Imho it would be a good idea to have one From/ToDevice element for all, then specify the type as an argument like From/ToDevice( TYPE x, ...) where x can be kernel,dpdk, dpdkring, vhost etc.

[tbarbette]

The ToDevice and ToDPDKDevice have different port behaviours, this was one of the main reason why it has been made different. Mainly, ToDPDKDevice allows full push.
ToDevice behave exactly as you say, you can choose METHOD NETMAP/PCAP/UNIX etc. What we could do is group DPDK modes under From/ToDPDKDevice. Eg. physical, ring or KNI for now.

I'm not familiar with vhost-user. Does it involves copying? Or is it only a kind of ring?
FromDevice can open "normal" sockets, but it is far from efficient

[kthkaya]

I am not familiar with it at low-level either. Been trying to to get my head around this to come up with an educated guess, but it is a bit confusing. It claims to be shared-memory based, but the app is attaching to a socket file.

[tbarbette]

It seems they achieved this in "Considerations on Deploying High-Performance Container-based NFV". They use vhost as you said and Click/FromDPDKDevice (the mainline one, backported from FastClick) inside the container. As a switch between containers it seems they use BESS, though FastClick could also be used. Vhost seems to be pure DPDK wireing to pass as dpdk arguments, that makes a "virtual" device appear that can be used as any physical device, requiring no change in the code.

[kthkaya]

Wow thanks for the link to the research, it was amazingly helpful. I guess i have a better understanding now. I believe the command dpdk-app-testpmd testpmd -l 6-7 -n 4 -m 1024 --no-pci \ --vdev=virtio_user0,path=/var/run/usvhost makes a virtual device appear in the container. I assumed it was some test application (the term "test" got me there) running in the container, and implementing the pmd on the socket file attached. The fact that they were able to use FromDPDKDevice(n) within the container proves my assumption wrong.

I will implement this and return with some feedback. Btw do you think connecting click to ovs via dpdkring would be noticeably faster due to the lack of a virtual device abstraction?

[kthkaya]

Here is a short guide for a simple Traffic source -> DPDClick Bridge -> Traffic Sink scenario

OVS

ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev
ovs-vsctl add-port br0 SRC-eth0 -- set Interface SRC-eth0 type=dpdkvhostuser ofport_request=1
ovs-vsctl add-port br0 NF-eth0 -- set Interface NF-eth0 type=dpdkvhostuser ofport_request=2
ovs-vsctl add-port br0 NF-eth1 -- set Interface NF-eth1 type=dpdkvhostuser ofport_request=3
ovs-vsctl add-port br0 SNK-eth0 -- set Interface SNK-eth0 type=dpdkvhostuser ofport_request=4

Above rules will create unix socket files under /usr/local/var/run/openvswitch/. Next, install rules so that traffic flow is strictly defined as SRC->NF->SNK and SRC<-NF<-SNK

ovs-ofctl add-flow br0 "in_port=1 action=output:2" 
ovs-ofctl add-flow br0 "in_port=2 action=output:1" 
ovs-ofctl add-flow br0 "in_port=3 action=output:4"
ovs-ofctl add-flow br0 "in_port=4 action=output:3"

dpdkbridge.click

VIRTIOUSR0_IN   ::  FromDPDKDevice(0, PROMISC true);
VIRTIOUSR0_OUT  ::  ToDPDKDevice(0);

VIRTIOUSR1_IN  ::  FromDPDKDevice(1, PROMISC true);
VIRTIOUSR1_OUT ::  ToDPDKDevice(1);

//-------------Logic----------------- 
VIRTIOUSR0_IN -> VIRTIOUSR1_OUT;
VIRTIOUSR1_IN -> VIRTIOUSR0_OUT;

We define the DPDK EAL arguments that we need to pass to click as following.

export EAL_PARAMS="-l 5-7 --master-lcore 5 -n 1 \
-m 1024 --file-prefix=nf --no-pci \
--vdev=virtio_user0,path=/usr/local/var/run/openvswitch/NF-eth0,mac=00:00:91:00:00:01 \
--vdev=virtio_user1,path=/usr/local/var/run/openvswitch/NF-eth1,mac=00:00:91:00:00:02"

--vdev argument has to start with "virtio_user" followed by a number as if index.
Finally, run click

click --dpdk $EAL_PARAMS -- dpdkbridge.click

Providing the virtio user interface as a From/ToDPDKDevice is a matter of specifying the path to the socket file that OVS created, in the EAL parameters. So if you want to containerize click, just map the /usr/local/var/run/openvswitch folder on the host to the container (additionally you need to map the hugepages mount to the container as well).

I will get back with some performance results as soon as I am finished. Hope it helps someone and thanks for your input guys!

[tbarbette]

That's fantastic news ! If you can push a little working config in the conf/fastclick folder with the above comments, that would be even better !

[kthkaya]

I will get to it as soon as I wrap things up.