Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot send packets in one direction (other works) #78

Open
mirabilos opened this issue Jul 3, 2022 · 22 comments
Open

Cannot send packets in one direction (other works) #78

mirabilos opened this issue Jul 3, 2022 · 22 comments

Comments

@mirabilos
Copy link

mirabilos commented Jul 3, 2022 •
edited

Downstream bugtracker link: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1016129

I have a virtual machine on a host-only network. The configuration is thus:

host$ ip a show dev virbr1
23: virbr1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 52:54:00:76:0b:a6 brd ff:ff:ff:ff:ff:ff
    inet 10.82.222.129/25 brd 10.82.222.255 scope global virbr1
       valid_lft forever preferred_lft forever
    inet6 fec0::1/64 scope site 
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:fe76:ba6/64 scope link 
       valid_lft forever preferred_lft forever

guest$ ip a show dev eth1
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc jhtb state UP group default qlen 1000
    link/ether 52:54:00:b7:47:b9 brd ff:ff:ff:ff:ff:ff
    inet 10.82.222.130/25 brd 10.82.222.255 scope global eth1
       valid_lft forever preferred_lft forever
    inet6 fec0::2/64 scope site 
       valid_lft forever preferred_lft forever
    inet6 fe80::5054:ff:feb7:47b9/64 scope link 
       valid_lft forever preferred_lft forever

I can do this:

guest$ sudo tcp6 -i eth1 -y 1000 -d fec0::1 -a 22 -P 1200

But I cannot send packets in the other direction:

host$ sudo tcp6 -i virbr1 -y 1000 -d fec0::2 -a 22 -P 1200
Error while performing Neighbor Discovery for the Destination Address
Error while learning Souce Address and Next Hop
host$ sudo tcp6 -i virbr1 -y 1000 -s fec0::1 -d fec0::2 -a 22 -P 1200 -S 52:54:00:76:0b:a6 -D 52:54:00:b7:47:b9
Error while performing Neighbor Discovery for the Destination Address
Error while learning Souce Address and Next Hop
host$ sudo tcp6 -i virbr1 -y 1000 -s fe80::5054:ff:fe76:ba6 -d fe80::5054:ff:feb7:47b9 -a 22 -P 1200 -S 52:54:00:76:0b:a6 -D 52:54:00:b7:47:b9
Error while performing Neighbor Discovery for the Destination Address
Error while learning Souce Address and Next Hop

Misspelt error messages aside, why is this, even if I pass the source address? Neighbour discovery should work because…

guest$ ping6 -c 1 fec0::1
PING fec0::1(fec0::1) 56 data bytes
64 bytes from fec0::1: icmp_seq=1 ttl=64 time=0.182 ms

--- fec0::1 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.182/0.182/0.182/0.000 ms
host$ ping6 -c 1 fec0::2
PING fec0::2(fec0::2) 56 data bytes
64 bytes from fec0::2: icmp_seq=1 ttl=64 time=0.384 ms

--- fec0::2 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.384/0.384/0.384/0.000 ms

It also doesn’t work when using link-local addresses, see the last attempt above.
@mirabilos
Copy link
Author

It stopped working in the other direction as well, after I upgraded the VM from buster to bullseye. I suspect this upgrade to be at fault.

Downgrading just the ipv6toolkit binary package does not fix the problem.

@mirabilos
Copy link
Author

Booting with the buster kernel, Linux 4.19.0-21-amd64 (4.19.249-2), does not magically fix this either.

@mirabilos
Copy link
Author

Incidentally… on the host system running bullseye, switching to a buster chroot does work!

host:~ $ sudo tcp6 -i virbr1 -d fec0::2 -y 500 -a 22 -P 600; echo = $?
Error while performing Neighbor Discovery for the Destination Address
Error while learning Souce Address and Next Hop
= 1
host:~ $ schroot -prc buster
(buster-i386)host:~ $ sudo tcp6 -i virbr1 -d fec0::2 -y 500 -a 22 -P 600; echo = $?
= 0

@mirabilos
Copy link
Author

(Workaround comment for https://github.com/orgs/community/discussions/23319 - please ignore.)

@mirabilos
Copy link
Author

mirabilos commented Jul 27, 2022 via email

@mirabilos mirabilos mentioned this issue Jul 27, 2022
Closed
@guyharris
Copy link

What happens if you run with -vv which, it appears, will cause the programs to report errors from libpcap calls?

(And why are those not ALWAYS reported? "Error while performing Neighbor Discovery for the Destination Address" and "Error while learning Souce Address and Next Hop" amount to "Something bad happened, but I'm not going to tell you what it was", which aren't very helpful if you want to try to make something bad not happen.)

@mirabilos
Copy link
Author

mirabilos commented Jul 27, 2022 via email

@mcr
Copy link

mcr commented Jul 27, 2022

On the bullseye kernel, a libpcap 1.8 works?
I'm curious if on a buster kernel, a libpcap 1.10 works?
It might also matter which kernel capture mechanism is compiled it.

@mirabilos
Copy link
Author

mirabilos commented Jul 27, 2022 via email

@guyharris
Copy link

guyharris commented Jul 27, 2022 •
edited

Not more, unfortunately:

So either 1) running with four -vs doesn't cause idata->verbose_f in ipv6_to_ether() to be set to a value > 1 or 2) foundaddr isn't set to "true" at the end of ipv6_to_ether().

If it's 1), that would imply unlikely brokenness, so I'll assume it's 2), and, therefore, that no libpcap call reported an error.

That means that the Neighbor Advertisement packet wasn't seen, either because 1) it wasn't sent, 2) it wasn't received, 3) it was received but didn't make it to the PF_PACKET socket or was dropped before the filtering, or 4) it was received, made it to the PF_PACKET socket, but didn't pass the capture filter.

Did you run tcpdump or Wireshark while running tcp6, to see what traffic was sent by and received on the machine running tcp6?

@mirabilos
Copy link
Author

mirabilos commented Jul 27, 2022 via email

@guyharris
Copy link

OK, try that again, but with tcpdump running with a filter of "icmp6 and ip6[7]==255 and ip6[40]==136 and ip6[41]==0".

@mirabilos
Copy link
Author

@guyharris here:

$ sudo tcpdump -evvlns 2000 -Xi eth1 "icmp6 and ip6[7]==255 and ip6[40]==136 and ip6[41]==>
tcpdump: listening on eth1, link-type EN10MB (Ethernet), snapshot length 2000 bytes
00:09:25.590900 52:54:00:76:0b:a6 > 52:54:00:b7:47:b9, ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32) fec0::1 > fe80::5054:ff:feb7:47b9: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is fec0::1, Flags [router, solicited, override]
          destination link-address option (2), length 8 (1): 52:54:00:76:0b:a6
            0x0000:  5254 0076 0ba6
        0x0000:  6000 0000 0020 3aff fec0 0000 0000 0000  `.....:.........
        0x0010:  0000 0000 0000 0001 fe80 0000 0000 0000  ................
        0x0020:  5054 00ff feb7 47b9 8800 a369 e000 0000  PT....G....i....
        0x0030:  fec0 0000 0000 0000 0000 0000 0000 0001  ................
        0x0040:  0201 5254 0076 0ba6                      ..RT.v..
00:09:26.591051 52:54:00:76:0b:a6 > 52:54:00:b7:47:b9, ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32) fec0::1 > fe80::5054:ff:feb7:47b9: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is fec0::1, Flags [router, solicited, override]
          destination link-address option (2), length 8 (1): 52:54:00:76:0b:a6
            0x0000:  5254 0076 0ba6
        0x0000:  6000 0000 0020 3aff fec0 0000 0000 0000  `.....:.........
        0x0010:  0000 0000 0000 0001 fe80 0000 0000 0000  ................
        0x0020:  5054 00ff feb7 47b9 8800 a369 e000 0000  PT....G....i....
        0x0030:  fec0 0000 0000 0000 0000 0000 0000 0001  ................
        0x0040:  0201 5254 0076 0ba6                      ..RT.v..
00:09:27.591131 52:54:00:76:0b:a6 > 52:54:00:b7:47:b9, ethertype IPv6 (0x86dd), length 86: (hlim 255, next-header ICMPv6 (58) payload length: 32) fec0::1 > fe80::5054:ff:feb7:47b9: [icmp6 sum ok] ICMP6, neighbor advertisement, length 32, tgt is fec0::1, Flags [router, solicited, override]
          destination link-address option (2), length 8 (1): 52:54:00:76:0b:a6
            0x0000:  5254 0076 0ba6
        0x0000:  6000 0000 0020 3aff fec0 0000 0000 0000  `.....:.........
        0x0010:  0000 0000 0000 0001 fe80 0000 0000 0000  ................
        0x0020:  5054 00ff feb7 47b9 8800 a369 e000 0000  PT....G....i....
        0x0030:  fec0 0000 0000 0000 0000 0000 0000 0001  ................
        0x0040:  0201 5254 0076 0ba6                      ..RT.v..
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel

@mcr
Copy link

mcr commented Jul 27, 2022

It sounds like if we have the capture that we can build a regression test that shows the different between 1.8 and 1.10, or are there kernel issues I'm missing?

@guyharris
Copy link

guyharris commented Jul 27, 2022 •
edited

tcpdump with the filter - which is also the filter used by ipv6tools when looking for a Neighbor Advertisement - saw the Neighbor Advertisements, so the filter doesn't seem to be the problem.

Presumably the version of ipv6tools used here has commit 03b0fdd from 2020, so it's not providing a timeout of 0 to pcap_open_live(). (If it doesn't have that commit, that's a bug that could cause the code not to work with libpcap 1.10 on Linux, as 1.10 has a commit to fix the behavior of a timeout of 0 to match the documentation.)

@mirabilos
Copy link
Author

Gotcha. It doesn’t:

https://sources.debian.org/src/ipv6toolkit/2.0%2Bds.1-1/tools/libipv6.h/#L126

I’ll test locally if that fixes the issue; if so, it’s easy to backport, and I’ll take that to the maintainer.

Updating ipv6toolkit in distros hits quite the snag in that ipv6toolkit upstream doesn’t publish released versions any more, apparently :/

Thank you for the debugging! Much appreciated.

@guyharris
Copy link

By the way, if you're willing to require libpcap 1.5 or later:

The various packet batching mechanisms are oriented towards packet capture, where immediate delivery isn't a priority but reducing per-packet overhead is, so they accumulate a collection of packets and deliver them with one wakeup per collection (and, for non-memory-mapped capture, one copy per collection) rather than one per packet.

What you're doing is implementing a tiny bit of ICMPv6 and directly sending packets on, and receiving packets from, a network interface; for that, you want immediate delivery and don't expect to get a huge number of packets, so reducing per-packet overhead isn't as important.

You could use the pcap_create()/pcap_activate() API - create a handle with pcap_create() set immediate mode with pcap_set_immediate_mode(), set any other parameters that are relevant (note that the timeout is not relevant in immediate mode, so you needn't set it), and activate the handle with pcap_activate().

@mcr
Copy link

mcr commented Jul 28, 2022

Updating ipv6toolkit in distros hits quite the snag in that ipv6toolkit upstream doesn’t publish released versions any more, apparently :/

@fgont is seen regularly at IETF, so I'll bug him this week.

@fgont
Copy link
Owner

fgont commented Jul 28, 2022

Apologies for the delay in getting back to you guys (shame on me!). I'll take a look at this tommorrow.

@mirabilos
Copy link
Author

mirabilos commented Jul 31, 2022 via email

@mirabilos
Copy link
Author

mirabilos commented Aug 3, 2022 via email

@mirabilos
Copy link
Author

mirabilos commented Aug 3, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@mcr @mirabilos @fgont @guyharris