Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

设置一些参数配置后client必定会回一个ack #442

Closed
alongpan opened this issue May 10, 2024 · 38 comments
Closed

设置一些参数配置后client必定会回一个ack #442

alongpan opened this issue May 10, 2024 · 38 comments

Comments

@alongpan
Copy link

当我设置keepalive大于2s的,或者是keepalive后面配置了num,或者配置protocol为http,client必定会确认一个ack,如果这些不配置client和server是一个请求一个响应,请问这个client的ack可以取消吗?
image

@pengjianzhang
Copy link
Collaborator

pengjianzhang commented May 10, 2024

请详细说明一下不ACK的理由。
你试一下早期的版本,可能不会回复ACK。
我看下能不能给个配置参数关闭ACK

@alongpan
Copy link
Author

有几种场景,如果不是测试其他服务器,只是dperf client和服务器对发数据
1、client回应ack会导致数据包多3/1(主要测pps的时候,这个倒是可以写公式解决,如果能参数控制回不回ack更好)
2、固定包长的时候回的ack是64字节,这个没法更改,比如固定测试128字节数据,出现1/3的64字节数据包

@alongpan
Copy link
Author

这是我的配置,试了几次没有重传
#daemon
mode client
socket_mem 12000
tx_burst 8
launch_num 8
cpu 0 2 4 6
rss auto l3l4

payload_random

vlan 1234

packet_size 56
duration 3000h
cps 40000
cc 1000000
keepalive 1320ms

port 0000:04:00.0 192.168.31.10 192.168.99.240
client 192.168.31.10 200
server 192.168.99.240 1
listen 80 2
lport_range 63026-65535
http_method POST

#daemon
mode server
socket_mem 12000
tx_burst 8
payload_random
cpu 12 14 16 18
rss auto l3l4
vlan 1234

duration 3000h
packet_size 56
keepalive 10s

port 0000:04:00.1 192.168.99.240 192.168.31.10
client 192.168.31.10 200
server 192.168.99.240 1
listen 80 2

@pengjianzhang
Copy link
Collaborator

http协议回复ACK太快了,应该是一个Bug
keepalive超过2秒,dperf会快速回复ACK,可以做一个开关关掉

@alongpan
Copy link
Author

我这个配置没超过2s,看数据包是不回ack的。
麻烦在基于post那个版本做个开关,谢谢。

@pengjianzhang
Copy link
Collaborator

http post已经合入到主干了
请在周围小伙伴里面宣传一下dperf 谢谢

@pengjianzhang
Copy link
Collaborator

请测一下这个分支 https://github.com/pengjianzhang/dperf/tree/disable_ack
client上配置:
disable_ack

@alongpan
Copy link
Author

arpRx /home/dperf/build/dperf: unrecognized option '--no-telemetry'
EAL: Detected 24 lcore(s)
EAL: Detected 2 NUMA nodes
EAL: FATAL: Invalid 'command line' arguments.
EAL: Invalid 'command line' arguments.

Usage: /home/dperf/build/dperf [options]

EAL common options:
-c COREMASK Hexadecimal bitmask of cores to run on
-l CORELIST List of cores to run on
The argument format is [-c2][,c3[-c4],...]
where c1, c2, etc are core indexes between 0 and 128
--lcores COREMAP Map lcore set to physical cpu set
The argument format is
'<lcores[@CPUs]>[<,lcores[@CPUs]>...]'
lcores and cpus list are grouped by '(' and ')'
Within the group, '-' is used for range separator,
',' is used for single number separator.
'( )' can be omitted for single element group,
'@' can be omitted if cpus and lcores have the same value
-s SERVICE COREMASK Hexadecimal bitmask of cores to be used as service cores
--master-lcore ID Core ID that is used as master
--mbuf-pool-ops-name Pool ops name for mbuf to use
-n CHANNELS Number of memory channels
-m MB Memory to allocate (see also --socket-mem)
-r RANKS Force number of memory ranks (don't detect)
-b, --pci-blacklist Add a PCI device in black list.
Prevent EAL from using this PCI device. The argument
format is domain:bus:devid.func.
-w, --pci-whitelist Add a PCI device in white list.
Only use the specified PCI devices. The argument format
is <[domain:]bus:devid.func>. This option can be present
several times (once per device).
[NOTE: PCI whitelist cannot be used with -b option]
--vdev Add a virtual device.
The argument format is [,key=val,...]
(ex: --vdev=net_pcap0,iface=eth2).
--iova-mode Set IOVA mode. 'pa' for IOVA_PA
'va' for IOVA_VA
-d LIB.so|DIR Add a driver or driver directory
(can be used multiple times)
--vmware-tsc-map Use VMware TSC map instead of native RDTSC
--proc-type Type of this process (primary|secondary|auto)
--syslog Set syslog facility
--log-level= Set global log level
--log-level=:
Set specific log level
-v Display version information on startup
-h, --help This help
--in-memory Operate entirely in memory. This will
disable secondary process support
--base-virtaddr Base virtual address

EAL options for DEBUG use only:
--huge-unlink Unlink hugepage files after init
--no-huge Use malloc instead of hugetlbfs
--no-pci Disable PCI
--no-hpet Disable HPET
--no-shconf No shared config (mmap'd files)

EAL Linux options:
--socket-mem Memory to allocate on sockets (comma separated values)
--socket-limit Limit memory allocation on sockets (comma separated values)
--huge-dir Directory where hugetlbfs is mounted
--file-prefix Prefix for hugepage filenames
--create-uio-dev Create /dev/uioX (usually done by hotplug)
--vfio-intr Interrupt mode for VFIO (legacy|msi|msix)
--legacy-mem Legacy memory mode (no dynamic allocation, contiguous segments)
--single-file-segments Put all hugepage memory in single files
--match-allocations Free hugepages exactly as allocated

rte_eal_init fail
dpdk_eal_init fail
dpdk init fail

@alongpan
Copy link
Author

这个版本启动报错,一摸一样的配置回退就可以了
mode client
socket_mem 12000
cpu 0 2 4 6
rss auto l3l4
tx_burst 8
launch_num 8
duration 3000h
cps 40000
cc 2000010
vlan 1234
client 192.168.31.10 200
server 192.168.99.240 1
listen 80 2
http_method POST
packet_size 128
keepalive 5980ms
lport_range 60526-65535
port 0000:04:00.0 192.168.31.10 192.168.99.240

@pengjianzhang
Copy link
Collaborator

你用的是哪个dpdk版本

@alongpan
Copy link
Author

19.11.10

@pengjianzhang
Copy link
Collaborator

修复了,更新一下代码,再试一下

@alongpan
Copy link
Author

还是有问题,无法启动
Error: bad gateway. dperf cannot find gateway's MAC address. Please check the link.

@pengjianzhang
Copy link
Collaborator

port 0000:04:00.1 192.168.99.240 192.168.31.10
这两个IP在同一个网段吗 192.168.99.240 192.168.31.10 ?

@alongpan
Copy link
Author

不是同一个网段,没过三层的,改成老版本可以跑通

@pengjianzhang
Copy link
Collaborator

192.168.31.10 是LB地址?
如果不是同一网段,那就要配MAC地址
这不是BUG

@alongpan
Copy link
Author

不是LB,我们这个配置不写掩码,实际下发配置会带掩码吗?我的环境非常简单,就是client和server对发,中间过二层交换机

@alongpan
Copy link
Author

client和server都改成192.168.31.x也不行,用回老版本192.168.99.240 192.168.31.10也可以通

@pengjianzhang
Copy link
Collaborator

如果client server互打,那么配置就该这样

client A 200
server B 1
port 0000:04:00.0 A B

@alongpan
Copy link
Author

是的,两边的ip和网关是互指的

@pengjianzhang
Copy link
Collaborator

你把client server都改成31网段,把两边配置发一下

@alongpan
Copy link
Author

mode client
socket_mem 12000
cpu 0 2 4 6
rss auto l3l4
tx_burst 8
launch_num 8
duration 3000h
cps 40000
cc 2000010
vlan 1234
client 192.168.31.10 200
server 192.168.31.250 1
listen 80 2
packet_size 128
keepalive 5980ms
lport_range 60526-65535
port 0000:04:00.0 192.168.31.10 192.168.31.250
http_method POST
disable_ack

@alongpan
Copy link
Author

mode server
socket_mem 12000
tx_burst 8
payload_random
cpu 12 14 16 18
rss auto l3l4
vlan 1234

duration 3000h
packet_size 128
keepalive 10s

port 0000:04:00.1 192.168.31.250 192.168.31.10
client 192.168.31.10 200
server 192.168.31.250 1
listen 80 2

@pengjianzhang
Copy link
Collaborator

  1. 用disable_ack分支,这个配置会出现找不到网关MAC吗?
  2. 用哪个版本是没有错的?

@alongpan
Copy link
Author

1、是的
2、我上一个版本是用的post那个分支

@pengjianzhang
Copy link
Collaborator

两边都同时启动看看,把最后的输出贴一下,主要是 看下arp统计 如:
Total Numbers:
...
...
arpRx 2 arpTx 2

@pengjianzhang
Copy link
Collaborator

pengjianzhang commented May 20, 2024

另外,请试一下主干 是否有错 https://github.com/baidu/dperf/tree/main

@alongpan
Copy link
Author

没看到arp统计,怎么看呢?
感觉是启动太慢,要么client或者是server,启动接近20秒,一端已经结束了,两外一端还没开始。
image

@alongpan
Copy link
Author

arpRx /home/dperf/build/dperf: unrecognized option '--no-telemetry' EAL: Detected 24 lcore(s) EAL: Detected 2 NUMA nodes EAL: FATAL: Invalid 'command line' arguments. EAL: Invalid 'command line' arguments.

Usage: /home/dperf/build/dperf [options]

EAL common options: -c COREMASK Hexadecimal bitmask of cores to run on -l CORELIST List of cores to run on The argument format is [-c2][,c3[-c4],...] where c1, c2, etc are core indexes between 0 and 128 --lcores COREMAP Map lcore set to physical cpu set The argument format is '<lcores[@CPUs]>[<,lcores[@CPUs]>...]' lcores and cpus list are grouped by '(' and ')' Within the group, '-' is used for range separator, ',' is used for single number separator. '( )' can be omitted for single element group, '@' can be omitted if cpus and lcores have the same value -s SERVICE COREMASK Hexadecimal bitmask of cores to be used as service cores --master-lcore ID Core ID that is used as master --mbuf-pool-ops-name Pool ops name for mbuf to use -n CHANNELS Number of memory channels -m MB Memory to allocate (see also --socket-mem) -r RANKS Force number of memory ranks (don't detect) -b, --pci-blacklist Add a PCI device in black list. Prevent EAL from using this PCI device. The argument format is domain:bus:devid.func. -w, --pci-whitelist Add a PCI device in white list. Only use the specified PCI devices. The argument format is <[domain:]bus:devid.func>. This option can be present several times (once per device). [NOTE: PCI whitelist cannot be used with -b option] --vdev Add a virtual device. The argument format is [,key=val,...] (ex: --vdev=net_pcap0,iface=eth2). --iova-mode Set IOVA mode. 'pa' for IOVA_PA 'va' for IOVA_VA -d LIB.so|DIR Add a driver or driver directory (can be used multiple times) --vmware-tsc-map Use VMware TSC map instead of native RDTSC --proc-type Type of this process (primary|secondary|auto) --syslog Set syslog facility --log-level= Set global log level --log-level=: Set specific log level -v Display version information on startup -h, --help This help --in-memory Operate entirely in memory. This will disable secondary process support --base-virtaddr Base virtual address

EAL options for DEBUG use only: --huge-unlink Unlink hugepage files after init --no-huge Use malloc instead of hugetlbfs --no-pci Disable PCI --no-hpet Disable HPET --no-shconf No shared config (mmap'd files)

EAL Linux options: --socket-mem Memory to allocate on sockets (comma separated values) --socket-limit Limit memory allocation on sockets (comma separated values) --huge-dir Directory where hugetlbfs is mounted --file-prefix Prefix for hugepage filenames --create-uio-dev Create /dev/uioX (usually done by hotplug) --vfio-intr Interrupt mode for VFIO (legacy|msi|msix) --legacy-mem Legacy memory mode (no dynamic allocation, contiguous segments) --single-file-segments Put all hugepage memory in single files --match-allocations Free hugepages exactly as allocated

rte_eal_init fail dpdk_eal_init fail dpdk init fail

主线也报这个错

@pengjianzhang
Copy link
Collaborator

给我邮箱发个邮件 pengjianzhang@gmail.com 会议讨论一下吧

@pengjianzhang
Copy link
Collaborator

pengjianzhang commented May 20, 2024

再更新一下主干

@pengjianzhang
Copy link
Collaborator

@alongpan
Copy link
Author

还是有几个问题:
1、启动时间感觉太长,配置protocol 为HTTP也会提示找不到网关
2、配置disable_ack之后,服务器触发了重传,这个是否也能关掉
3、实际发出的数据包比配置的pcap_size多10byte(这个历史版本都一样)

@alongpan
Copy link
Author

image

@pengjianzhang
Copy link
Collaborator

看邮件 会议讨论一下

@pengjianzhang
Copy link
Collaborator

pengjianzhang commented May 29, 2024

请测试 https://github.com/pengjianzhang/dperf/tree/disable_ack

配置参考:

keepalive 6s
retransmit_timeout 10
disable_ack

@alongpan
Copy link
Author

可以了

@pengjianzhang
Copy link
Collaborator

pengjianzhang commented May 30, 2024

这个特性已经合入了主干,请帮忙测试一下主干,近期会发布新版本,谢谢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants