Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Vhost config for netronome-packetgen #253

Closed
sarsanaee opened this issue Aug 7, 2019 · 12 comments
Closed

Vhost config for netronome-packetgen #253

sarsanaee opened this issue Aug 7, 2019 · 12 comments

Comments

@sarsanaee
Copy link

Hello

I want to config the netronome-packetgen to work with a vhost PMD port(using a vdev parameter), however, I have some issues in configuring it.

Here is the command I run from the MOONGEN_ROOT:
sudo ./build/MoonGen examples/netronome-packetgen/packetgen.lua --dpdk-config=libmoon/dpdk-conf.lua --tx-slave 0

Here is my dpdk-config:

DPDKConfig {
	-- configure the CPU cores to use, default: all cores
	--cores = {0, 10, 11, 12, 13, 14, 15},
	cores = {0, 1},
	
	-- max number of shared tasks running on core 0
	sharedCores = 2,

	-- black or whitelist devices to limit which PCI devs are used by DPDK
	-- only one of the following examples can be used
	pciBlacklist = {"0000:81:00.0","0000:81:00.1"},
	--pciWhitelist = {"0000:81:00.3","0000:81:00.1"},
	--
	-- arbitrary DPDK command line options
	-- the following configuration allows multiple DPDK instances (use together with pciWhitelist)
	-- cf. http://dpdk.org/doc/guides/prog_guide/multi_proc_support.html#running-multiple-independent-dpdk-applications
	cli = {
		--"--file-prefix", "m1",
		"--socket-mem", "1024,1024",
		"--log-level", "8",
		--"--proc-type", "auto",
		"--vdev", "virtio_user0,path=/users/alireza/my_vhost1.sock,queues=1"
	}

}

It actually recognizes my vhost port, here is the dpdk log:

EAL: TSC frequency is ~1995388 KHz
EAL: Master lcore 0 is ready (tid=b1181940;cpuset=[0])
EAL: lcore 3 is ready (tid=ad193700;cpuset=[0])
EAL: lcore 2 is ready (tid=ad994700;cpuset=[0])
EAL: lcore 1 is ready (tid=ae195700;cpuset=[1])
EAL: PCI device 0000:81:00.0 on NUMA socket 1
EAL:   Device is blacklisted, not initializing
EAL: PCI device 0000:81:00.1 on NUMA socket 1
EAL:   Device is blacklisted, not initializing
EAL: Search driver virtio_user0 to probe device virtio_user0
[INFO]  Found 1 usable devices:
   Device 0: CA:53:6D:1B:E3:34 (unknown NIC (PCI ID 7fcf:0))

However, I still get the following error:

[FATAL] Lua error in task master
/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/device.lua:196: Could not configure device 0: error Operation not supported
Stack Traceback
===============
(2) Lua method 'fatal' at file '/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/log.lua:129'
	Local variables:
	 self = table: 0x400bff98  {DEBUG:0, fatal:function: 0x40a98848, writeToLog:function: 0x40a98828, INFO:1 (more...)}
	 str = string: "Could not configure device 0: error Operation not supported"
(3) Lua field 'config' at file '/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/device.lua:196'
	Local variables:
	 args = table: 0x41704020  {rxQueues:1, stripVlan:true, dropEnable:true, mempools:table: 0x40837a58 (more...)}
	 driverInfo = table: 0x40a573d8  {}
	 rc = number: -95
(4) Lua function 'master' at file 'examples/netronome-packetgen/packetgen.lua:250' (best guess)
	Local variables:
	 continue = boolean: true
	 totalSlaves = number: 1
	 portList = table: 0x41703fb0  {1:0}
	 devices = table: 0x41703ff8  {}
	 (for generator) = C function: builtin#6
	 (for state) = table: 0x41703fb0  {1:0}
	 (for control) = number: 1
	 _ = number: 1
	 port = number: 0
	 deviceIdx = number: 1
(5) global C function 'xpcall'
(6) Lua upvalue 'master' at file '/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/main.lua:96'
	Local variables:
	 _ = string: "./build/MoonGen"
	 file = string: "examples/netronome-packetgen/packetgen.lua"
	 args = table: 0x419b85d0  {1:--tx-slave, 2:0}
	 cfgFile = nil
	 ok = boolean: true
	 parsedArgs = table: 0x40e6a930  {}
(7) Lua function 'main' at file '/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/main.lua:146' (best guess)
	Local variables:
	 task = string: "master"
	/proj/uic-dcs-PG0/MoonGen/build/../libmoon/lua/device.lua:196: Could not configure device 0: error Operation not supported

I couldn't figure out where the issue is in this case!

Thanks,
Alireza
@emmericp
Copy link
Owner

That error means its trying to configure some feature not supported by the (virtual) device, I unfortunately don't know much about the netronome packetgen and vhost devices, so I'm afraid it could be literally anything.

Two ways to debug this:

  1. start with a simpler script, try the disableOffloads option

  2. figure out where exactly in device.c it is failing

@NetronomeMoongen
Copy link
Contributor

NetronomeMoongen commented Aug 13, 2019 via email

@sarsanaee
Copy link
Author

Hello,

Thank you for the response, the problem was the disableOffload, which is false by default. For vdev devices, I think we should set it true somewhere.

@immadhavv
Copy link

@sarsanaee @emmericp
Hi,
I'm trying to set up a system where MoonGen(in a container) send line rate traffic to testpmd(in a container) through virtual ports (dpdkvhostuser type) connected to an ovs-dpdk bridge. I have been able to initialize one vdev parameter like you've mentioned here, and its getting recognized, but I'm not sure how to add more than one vdev parameter.

Can someone help me understand how this dpdk-config.lua file works and how to manipulate it and make use of it for my purpose?

@NetronomeMoongen
Copy link
Contributor

NetronomeMoongen commented Apr 15, 2020 via email

@immadhavv
Copy link

Hi @NetronomeMoongen
Thanks for the pointers. Actually, I've already been able to send packets from pktgen using the similar setup, I've mentioned before.

But now I want to send using MoonGen , as I need it to resemble a DDOS attack. I've been able to configure one Vhost device in the dpdk-config file like @sarsanaee mentioned. It's getting recognized as Device 0. But I need to configure another vhost device too as Device 1 so that I can send packets.

@NetronomeMoongen
Copy link
Contributor

NetronomeMoongen commented Apr 15, 2020 via email

@sarsanaee
Copy link
Author

sarsanaee commented Apr 15, 2020 via email

@sarsanaee
Copy link
Author

I find this issue pretty misleading although it is created by myself since I eventually realized I had some misunderstandings regarding DPDK. But still, the responses helped me figure that out.

@immadhavv
Copy link

immadhavv commented Apr 15, 2020 •
edited

@sarsanaee
Thanks! I'm able to initialize multiple virtual ports now. The issue was that I was passing the mac address argument too previously which worked with pktgen but it was a problem here. Also,it was only because of this issue I was able to get started with this. Thanks for that actually.

DPDKConfig { cores={0,3,4}, cli = { "--file-prefix", "moongen", "--socket-mem", "512", --"--rxq","1", --"--txq","1", "--vdev","net_virtio_user1,path=/var/run/openvswitch/vhost-user1,queues=1", "--vdev","net_virtio_user2,path=/var/run/openvswitch/vhost-user2,queues=1", "--log-level","8", } }
I've turned the disbleoffloads option to true in device.lua file too.
But now, the compiler results in this. says I've requested for 3 rx and tx queues while my devide has only 1 rx and tx queue..but I didn't really ask for 3. I'm trying to pass the rxq and txq arguments here but it won't get accepted(says invalid arguments). It's supposed to be set as q1 by default anyway.
`EAL: Virtual area found at 0x7f8505400000 (size = 0xb9000000)
EAL: Ask a virtual area of 0x200000 bytes
EAL: Virtual area found at 0x7f8505000000 (size = 0x200000)
EAL: Requesting 256 pages of size 2MB from socket 0
EAL: TSC frequency is ~2397223 KHz
EAL: Master lcore 0 is ready (tid=ead10940;cpuset=[0])
EAL: lcore 6 is ready (tid=c91fd700;cpuset=[0])
EAL: lcore 7 is ready (tid=c89fc700;cpuset=[0])
EAL: lcore 5 is ready (tid=c99fe700;cpuset=[0])
EAL: lcore 8 is ready (tid=c33ff700;cpuset=[0])
EAL: lcore 9 is ready (tid=c2bfe700;cpuset=[0])
EAL: lcore 10 is ready (tid=c23fd700;cpuset=[0])
EAL: lcore 11 is ready (tid=c1bfc700;cpuset=[0])
EAL: lcore 4 is ready (tid=ca1ff700;cpuset=[4])
EAL: lcore 3 is ready (tid=e7510700;cpuset=[3])
EAL: lcore 2 is ready (tid=e7d11700;cpuset=[2])
EAL: lcore 1 is ready (tid=e8512700;cpuset=[1])
EAL: lcore 12 is ready (tid=c13fb700;cpuset=[0])
EAL: PCI device 0000:00:03.0 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 8086:100e net_e1000_em
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:00:08.0 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 8086:100e net_e1000_em
EAL: Not managed by a supported kernel driver, skipped
EAL: PCI device 0000:00:09.0 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 8086:100e net_e1000_em
EAL: Not managed by a supported kernel driver, skipped
EAL: Search driver net_virtio_user1 to probe device net_virtio_user1
EAL: Search driver net_virtio_user2 to probe device net_virtio_user2
[INFO] Found 2 usable devices:
Device 0: 8A:E9:DD:7B:A3:89 (unknown NIC (PCI ID 7f85:0))
Device 1: 66:3B:CF:5A:FC:AC (unknown NIC (PCI ID 7830:7030))
[INFO] Flow load-latency => 0x1
[FATAL] Lua error in task master
/home/MoonGen/build/../libmoon/lua/device.lua:120: device supports only 1 tx queues, requested 3
Stack Traceback

(2) Lua method 'fatal' at file '/home/MoonGen/build/../libmoon/lua/log.lua:129'
Local variables:
self = table: 0x40ad6c00 {DEBUG:0, fatal:function: 0x40ad6d30, writeToLog:function: 0x40ad6d10, INFO:1 (more...)}
str = string: "device supports only 1 tx queues, requested 3"
(3) Lua field 'config' at file '/home/MoonGen/build/../libmoon/lua/device.lua:120'
Local variables:
args = table: 0x40ee86d8 {bufSize:2048, rxQueues:1, txDescs:1024, txQueues:3, rssQueues:0, numBufs:2047 (more...)}
driverInfo = table: 0x40eeb3e0 {}
(4) Lua method 'configure' at file './interface/devmgr.lua:40'
Local variables:
self = table: 0x40ee64e8 {0:table: 0x40ee7b40, 1:table: 0x40ee7c50}
(for generator) = C function: next
(for state) = table: 0x40ee64e8 {0:table: 0x40ee7b40, 1:table: 0x40ee7c50}
(for control) = number: nan
i = number: 0
v = table: 0x40ee7b40 {rxq:1, rxqi:0, txq:3, txqi:0, rsqi:0, rsq:0}
txq = number: 3
rxq = number: 1
(5) Lua function 'master' at file './interface/init.lua:66' (best guess)
Local variables:
args = table: 0x419e35a0 {output:., config:flows, start:true, flows:table: 0x419e3f30}
devices = table: 0x40ee64e8 {0:table: 0x40ee7b40, 1:table: 0x40ee7c50}
flows = table: 0x40ee6598 {1:table: 0x41daf388}
(6) global C function 'xpcall'
(7) Lua upvalue 'master' at file '/home/MoonGen/build/../libmoon/lua/main.lua:96'
Local variables:
_ = string: "./build/MoonGen"
file = string: "./interface/init.lua"
args = table: 0x41855948 {1:start, 2:load-latency:0:1:rate=10Mp/s}
cfgFile = string: "libmoon/dpdk-conf.lua"
ok = boolean: true
parsedArgs = table: 0x419e3538 {1:table: 0x419e35a0}
(8) Lua function 'main' at file '/home/MoonGen/build/../libmoon/lua/main.lua:146' (best guess)
Local variables:
task = string: "master"
/home/MoonGen/build/../libmoon/lua/device.lua:120: device supports only 1 tx
queues, requested 3`

@immadhavv
Copy link

@sarsanaee @NetronomeMoongen @emmericp
Could someone take a look at the above issue?
I'd like to know how to add the number of rx queues and tx queues as parameters in the dpdk-conf.lua file.
I went through the list of EAL parameters but couldn't find how to.
It's supposed to be defaulted to 1, but the compiler says I have requested for 3 queues, when I didn't. :/
I tried to manually change the code in device.lua also to set the args.rxQueues and args.txQueues to 1, but it throws another error saying "device is configured with rx queues 0 to 0, tried to get rx queue number 1"

Can someone help me with this and maybe explain how to use the dpdk-conf.lua file to add parameters(a list of all parameters available especially for vdev would help..not just the list of EAL parameters in the dpdk.org)

@immadhavv immadhavv mentioned this issue Apr 18, 2020
Open
@sarsanaee
Copy link
Author

Did you get it going?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@sarsanaee @emmericp @NetronomeMoongen @immadhavv