Unable to communicate over IP between guest and Host using PCAP backend on Linux

[synexo]

Code of Conduct & Contributing Guidelines

• I agree to follow the code of conduct and the contributing guidelines.

Have you checked that no other similar bug report(s) already exists?

• I have searched and didn't find any similar issues.

What operating system(s) this bug have occurred on?

Ubuntu 20.04.2

What version(s) of DOSBox-X have this bug?

DOSBox-X version 0.83.18 SDL2

Describe the bug

Cannot ping to/from Linux host when using pcap backend.

I have initial success with the pcap backend, confirming in the log that NE2000 emulation is working on interface eth1 w/ NE2000 enabled. Within DOS, the NE2000 gets the MAC I assigned (de:ad:be:ef:12:34) and MTCP DHCP is even able to get an IP address from the DHCP server running on same Linux host (62:6e:fd:1e:ca:a2). I cannot however ping to/from the Linux host. In tcpdump, I see the ARP requests but they are never responded to, except during the DHCP dialogue:

Successful DHCP:

listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
21:38:18.411623 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 305: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from de:ad:be:ef:12:34, length 263
21:38:19.412098 62:6e:fd:1e:ca:a2 > de:ad:be:ef:12:34, ethertype IPv4 (0x0800), length 342: 10.116.0.3.67 > 10.116.0.5.68: BOOTP/DHCP, Reply, length 300
21:38:22.325558 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype IPv4 (0x0800), length 312: 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from de:ad:be:ef:12:34, length 270
21:38:22.325845 62:6e:fd:1e:ca:a2 > de:ad:be:ef:12:34, ethertype IPv4 (0x0800), length 342: 10.116.0.3.67 > 10.116.0.5.68: BOOTP/DHCP, Reply, length 300

Unsuccessful ping attempts from DOS to Host:

21:38:39.955225 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.5 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46
21:38:40.439892 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.3 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46
21:38:40.980241 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.3 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46
21:38:41.519731 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.3 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46
21:38:42.061926 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.3 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46
21:38:42.062333 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.3 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46
21:38:42.600311 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.3 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46
21:38:43.143166 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.3 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46
21:38:43.683539 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.3 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46
21:38:43.683814 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.3 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46
21:38:44.223910 de:ad:be:ef:12:34 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 60: Request who-has 10.116.0.3 (ff:ff:ff:ff:ff:ff) tell 10.116.0.5, length 46

Interface configuration on host:

eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 10.116.0.3 netmask 255.255.255.255 broadcast 0.0.0.0
ether 62:6e:fd:1e:ca:a2 txqueuelen 1000 (Ethernet)
RX packets 86 bytes 5428 (5.4 KB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 1880 bytes 156740 (156.7 KB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

The other direction, ping'ing from Host the DOS, sees responses from the DOS server to the ARP but the Host appears to ignore them:

21:40:55.617326 62:6e:fd:1e:ca:a2 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.116.0.5 tell 10.116.0.3, length 28
21:40:55.925655 de:ad:be:ef:12:34 > 62:6e:fd:1e:ca:a2, ethertype ARP (0x0806), length 60: Reply 10.116.0.5 is-at de:ad:be:ef:12:34, length 46
21:40:56.627823 62:6e:fd:1e:ca:a2 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.116.0.5 tell 10.116.0.3, length 28
21:40:57.651826 62:6e:fd:1e:ca:a2 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.116.0.5 tell 10.116.0.3, length 28
21:40:58.676280 62:6e:fd:1e:ca:a2 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.116.0.5 tell 10.116.0.3, length 28
21:40:58.997396 de:ad:be:ef:12:34 > 62:6e:fd:1e:ca:a2, ethertype ARP (0x0806), length 60: Reply 10.116.0.5 is-at de:ad:be:ef:12:34, length 46
21:40:58.998993 de:ad:be:ef:12:34 > 62:6e:fd:1e:ca:a2, ethertype ARP (0x0806), length 60: Reply 10.116.0.5 is-at de:ad:be:ef:12:34, length 46
21:40:58.999445 de:ad:be:ef:12:34 > 62:6e:fd:1e:ca:a2, ethertype ARP (0x0806), length 60: Reply 10.116.0.5 is-at de:ad:be:ef:12:34, length 46
21:40:59.699823 62:6e:fd:1e:ca:a2 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.116.0.5 tell 10.116.0.3, length 28
21:41:00.723823 62:6e:fd:1e:ca:a2 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.116.0.5 tell 10.116.0.3, length 28
21:41:01.748662 62:6e:fd:1e:ca:a2 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 10.116.0.5 tell 10.116.0.3, length 28

Expected behavior

Able to ping between host and guest w/ pcap backend, ARP responded to from host and DOS ARP used.

Steps to reproduce the behaviour

Use pcap backend as described on linux host.

Used configuration

[ne2000]
#  ne2000: Enable NE2000 Ethernet emulation.
# nicbase: The base address of the NE2000 board.
#  nicirq: The interrupt it uses. Note serial2 uses IRQ3 as default.
# macaddr: The MAC address the emulator will use for its network adapter.
#            If you have multiple DOSBox-Xes running on the same network,
#            this has to be changed for each. AC:DE:48 is an address range reserved for
#            private use, so modify the last three number blocks, e.g. AC:DE:48:88:99:AB.
# backend: The backend (either pcap or slirp is supported) used for the NE2000 Ethernet emulation.
#            If set to "auto", then "slirp" is selected when available, otherwise "pcap" is selected when available.
#            NE2000 Ethernet emulation will be disabled if no backend is available (or the specified backend if unavailble).
#            Possible values: pcap, slirp, auto, none.
ne2000  = true
nicbase = 300
nicirq  = 10
macaddr = DE:AD:BE:EF:12:34
backend = pcap

[ethernet, pcap]
# realnic: Specifies which of your host network interfaces is used for pcap.
#            Write 'list' here to see the list of devices from the Help
#            menu ('List network interfaces') or from the Status Window.
#            Then make your choice and put either the interface number
#            (e.g. 2) or a part of your adapters name (e.g. VIA here).
# timeout: Specifies the read timeout for the device in milliseconds for the pcap backend, or the default value will be used.
realnic = eth1
timeout = default

C:
NE2000 0x60 10 0x300
set MTCPCFG=\dosbox.cfg
cd MTCP
dhcp
ping 10.116.0.3

Emulator log

No response

Additional context

No response

[rderooy]

I just quickly tried it, and I see similar behaviour on my Linux system (F35).

In addition, in the logs I see regular dma read underrun errors when running mtcp ping.

LOG: BX_ERROR: ne2K: dma read underrun iolen=2 remote_bytes=1

[darkstar]

This is a well-known limitation of PCAP, it is not possible to communicate between host and guest. You should be able to get around this on Linux by using a virtual ethernet interface (veth) and bridging that to your physical interface.

[rderooy]

So I thought this was perhaps related to slirp (introduced in 0.83.12), so I did a build of 0.83.11, but I still see a ~75% package loss on mtcp ping.

Also, this is not restricted to guest <-> host, it also effects pinging from the guest anything else on the network, or the internet.

[rderooy]

I did some git bisecting and this commit seems to have broken PCAP.

af3e81ed400b6fd331cfdbd77304c05b66afe5e3 is the first bad commit
commit af3e81ed400b6fd331cfdbd77304c05b66afe5e3
Author: Jookia <contact@jookia.org>
Date:   Tue Jan 5 12:46:41 2021 +1100

    ne2000: Use 3 second timeout when opening pcap
    
    Currently the value is set to -1, which is invalid and breaks on macOS.
    Change this to something sane like 3000 milliseconds.

 src/hardware/ne2000.cpp | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

[rderooy]

@Jookia the PCAP fix you did for macOS, broke PCAP on Linux. With your fix in place, about 2/3 (or more) of packets are lost.

[rderooy]

As both winpcap and npcap documentation show examples where the read timeout is set to 1000, I tried the same setting on Linux. This improved matters, but there were still dropped packets (about 1 in 5). Changing the setting back to the original -1 fixed problems.

fyi, due to the SLIRP refactoring the value is now set in src/misc/ethernet_pcap.cpp on line 217.