Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IP defrag does not works for UDP VLAN traffic #747

Open
ates opened this issue Aug 12, 2021 · 1 comment
Open

IP defrag does not works for UDP VLAN traffic #747

ates opened this issue Aug 12, 2021 · 1 comment

Comments

@ates
Copy link

ates commented Aug 12, 2021

Hello,

I've tried to send UDP data which size is more than MTU via VLAN interface and found what IP defragmentation does not works in this case.

Tested on:

  • PF_RING 7.8, CentOS 8.3.2011, kernel 4.18.0-240.el8.x86_64
  • PF_RING dev:c449372b7f3d6fb4fe2b86c8197814f2e0a14e2c, Ubuntu 20.04, kernel 5.8.0-63-generic

options passed to pf_ring.ko: enable_debug=2 enable_ip_defrag=1

[687783.079029] [PF_RING] Welcome to PF_RING 8.1.0 ($Revision: dev:c449372b7f3d6fb4fe2b86c8197814f2e0a14e2c$)
                (C) 2004-20 ntop.org
[687783.079031] LINUX_VERSION_CODE 00050812
[687783.079031] [PF_RING] Min # ring slots 4096
[687783.079032] [PF_RING] Slot version     18
[687783.079032] [PF_RING] Capture TX       Yes [RX+TX]
[687783.079032] [PF_RING] IP Defragment    Yes
[687783.079039] NET: Registered protocol family 27

Logs:

[687863.363816] [PF_RING][DEBUG] ring_create:4515 [pid=805720]
[687863.363820] [PF_RING][DEBUG] ring_insert:2025 ring_insert
[687863.363821] [PF_RING][DEBUG] lockless_list_add:638 BEGIN [total=0]
[687863.363822] [PF_RING][DEBUG] lockless_list_add:664 END [total=1][id=0][top_element_id=0]
[687863.363823] [PF_RING][DEBUG] lockless_list_add:669 [slot 0 is full]
[687863.363827] [PF_RING][DEBUG] ring_proc_add:1196 Added /proc/net/pf_ring/805720-none.1
[687863.363827] [PF_RING][DEBUG] ring_create:4589 created
[687863.363832] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=107)
[687863.363834] [PF_RING][DEBUG] ring_setsockopt:7320 --> SO_RING_BUCKET_LEN=65535
[687863.363835] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=127)
[687863.363847] [PF_RING][DEBUG] ring_bind:5611 ring_bind() called
[687863.363848] [PF_RING][DEBUG] ring_bind:5628 searching device wlp3s0
[687863.363850] [PF_RING][DEBUG] packet_ring_bind:5568 packet_ring_bind(wlp3s0, bucket_len=65535) called
[687863.363851] [PF_RING][DEBUG] ring_proc_remove:1215 Removing /proc/net/pf_ring/805720-none.1
[687863.363852] [PF_RING][DEBUG] ring_proc_remove:1219 Removed /proc/net/pf_ring/805720-none.1
[687863.363854] [PF_RING][DEBUG] ring_proc_add:1196 Added /proc/net/pf_ring/805720-wlp3s0.1
[687863.363855] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=108)
[687863.363856] [PF_RING][DEBUG] ring_setsockopt:7005 Setting channel 0
[687863.363857] [PF_RING][DEBUG] ring_setsockopt:7019 [channel_id_mask=FFFFFFFFFFFFFFFF]
[687863.363858] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=141)
[687863.383754] [PF_RING][DEBUG] ring_mmap:5688 called
[687863.383756] [PF_RING][DEBUG] ring_mmap:5696 called, size: 4096 bytes [bucket_len=65535]
[687863.383757] [PF_RING][DEBUG] ring_alloc_mem:1927 ring_alloc_mem(bucket_len=65535)
[687863.383758] [PF_RING][DEBUG] ring_alloc_mem:1970 [PF_RING] Warning: jumbo mtu or snaplen (65535), resizing slots.. (num_slots = 512 x slot_len = 65592)
[687863.388479] [PF_RING][DEBUG] ring_alloc_mem:1985 successfully allocated 33595392 bytes at 0xffffba434de69000
[687863.388481] [PF_RING][DEBUG] ring_alloc_mem:2002 allocated 512 slots [slot_len=65592][tot_mem=33595392]
[687863.388482] [PF_RING][DEBUG] ring_mmap:5747 mmap [slot_len=65592][tot_slots=512] for ring on device wlp3s0
[687863.388484] [PF_RING][DEBUG] do_memory_mmap:5645 mode=0, size=4096, ptr=00000000e571badb
[687863.388489] [PF_RING][DEBUG] ring_mmap:5812 succeeded
[687863.388511] [PF_RING][DEBUG] ring_mmap:5688 called
[687863.388512] [PF_RING][DEBUG] ring_mmap:5696 called, size: 33595392 bytes [bucket_len=65535]
[687863.388513] [PF_RING][DEBUG] ring_mmap:5747 mmap [slot_len=65592][tot_slots=512] for ring on device wlp3s0
[687863.388513] [PF_RING][DEBUG] do_memory_mmap:5645 mode=0, size=33595392, ptr=00000000e571badb
[687863.389182] [PF_RING][DEBUG] ring_mmap:5812 succeeded
[687863.389188] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=140)
[687863.389190] [PF_RING][DEBUG] is_netdev_promisc:5210 checking promisc for wlp3s0
[687863.389192] [PF_RING][DEBUG] set_netdev_promisc:5224 setting promisc for wlp3s0
[687863.389201] device wlp3s0 entered promiscuous mode
[687863.389374] [PF_RING][DEBUG] ring_getsockopt:7632 --> getsockopt(179)
[687863.389376] [PF_RING][DEBUG] ring_getsockopt:7632 --> getsockopt(182)
[687863.389381] [PF_RING][DEBUG] ring_getsockopt:7632 --> getsockopt(184)
[687863.389386] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=117)
[687863.389387] [PF_RING][DEBUG] ring_setsockopt:7299 --> SO_SET_POLL_WATERMARK=1
[687863.390435] [PF_RING][DEBUG] ring_getsockopt:7632 --> getsockopt(184)
[687863.390439] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=26)
[687863.390440] [PF_RING][DEBUG] ring_setsockopt:6897 BPF filter
[687863.390441] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=26)
[687863.390442] [PF_RING][DEBUG] ring_setsockopt:6897 BPF filter
[687863.390475] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=106)
[687863.390476] [PF_RING][DEBUG] ring_setsockopt:7237 * SO_ACTIVATE_RING *
[687900.256909] [PF_RING][DEBUG] add_skb_to_ring:3814 ring_id=1 pfr->filtering_sample_rate=0 pfr->filtering_sampling_size=0
[687900.256946] [PF_RING][DEBUG] add_skb_to_ring:3814 ring_id=1 pfr->filtering_sample_rate=0 pfr->filtering_sampling_size=0
[687903.952629] [PF_RING][DEBUG] ring_setsockopt:6891 --> ring_setsockopt(optname=124)
[687903.953426] [PF_RING][DEBUG] ring_release:5362 called ring_release(wlp3s0)
[687903.953428] [PF_RING][DEBUG] ring_proc_remove:1215 Removing /proc/net/pf_ring/805720-wlp3s0.1
[687903.953430] [PF_RING][DEBUG] ring_proc_remove:1219 Removed /proc/net/pf_ring/805720-wlp3s0.1
[687903.953433] [PF_RING][DEBUG] ring_remove:2058 ring_remove()
[687903.953434] [PF_RING][DEBUG] ring_remove:2072 Found socket to remove
[687903.953435] [PF_RING][DEBUG] lockless_list_remove:692 BEGIN [total=1]
[687903.953436] [PF_RING][DEBUG] lockless_list_remove:711 END [total=0][top_element_id=0]
[687903.953437] [PF_RING][DEBUG] ring_remove:2091 leaving ring_remove()
[687903.954067] [PF_RING][DEBUG] unset_netdev_promisc:5247 resetting promisc for wlp3s0
[687903.954073] device wlp3s0 left promiscuous mode
[687904.059835] [PF_RING][DEBUG] ring_release:5513 ring_release: done

How to reproduce:

Interface with VLAN configured on machine 1:

enp0s3.1@enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 08:00:27:db:5d:1f brd ff:ff:ff:ff:ff:ff
    inet 10.4.0.2/8 brd 10.255.255.255 scope global enp0s3.1
       valid_lft forever preferred_lft forever
    inet6 fe80::a00:27ff:fedb:5d1f/64 scope link 
       valid_lft forever preferred_lft forever

Interface with VLAN configured on machine 2:

wlp3s0.1@wlp3s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 5c:e0:c5:a5:da:66 brd ff:ff:ff:ff:ff:ff
    inet 10.4.0.1/8 brd 10.255.255.255 scope global wlp3s0.1
       valid_lft forever preferred_lft forever
    inet6 fe80::5ee0:c5ff:fea5:da66/64 scope link 
       valid_lft forever preferred_lft forever

Send data from machine 1 to machine 2:

cat /etc/sysctl.conf | nc -u 10.4.0.1 50000

Run tcpdump on machine 2:

tcpdump -i wlp3s0.1 -e -n -vvv
tcpdump: listening on wlp3s0.1, link-type EN10MB (Ethernet), capture size 262144 bytes
15:24:45.441002 08:00:27:db:5d:1f > 5c:e0:c5:a5:da:66, ethertype 802.1Q (0x8100), length 1518: vlan 1, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 32461, offset 0, flags [+], proto UDP (17), length 1500)
    10.4.0.2.41556 > 10.4.0.1.50000: UDP, bad length 2683 > 1472
15:24:45.441036 08:00:27:db:5d:1f > 5c:e0:c5:a5:da:66, ethertype 802.1Q (0x8100), length 1249: vlan 1, p 0, ethertype IPv4, (tos 0x0, ttl 64, id 32461, offset 1480, flags [none], proto UDP (17), length 1231)
    10.4.0.2 > 10.4.0.1: ip-proto-17

Also tested on interfaces without VLAN configured and it's works (sent the same data using nc):

tcpdump -i wlp3s0 -e -n -vvv "udp port 50000"
tcpdump: listening on wlp3s0, link-type EN10MB (Ethernet), capture size 262144 bytes
15:27:18.141203 08:00:27:db:5d:1f > 5c:e0:c5:a5:da:66, ethertype IPv4 (0x0800), length 2725: (tos 0x0, ttl 64, id 22009, offset 0, flags [none], proto UDP (17), length 2711)
    10.10.1.41.54100 > 10.10.1.111.50000: [udp sum ok] UDP, length 2683

Here is pcaps for both cases:
pf_ring_defrag_vlan.zip
@ates
Copy link
Author

ates commented Aug 16, 2021 •
edited

VLAN stripping offload is disabled

rx-vlan-offload: off
tx-vlan-offload: off [requested on]
rx-vlan-filter: on [fixed]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@ates