UDP message transmission failure when destination IP address is broadcast

[fmgomes]

Basic Infos

Hardware

Hardware: ESP-12E (NodeMCU)
Core Version: 2.2.0 (also happens with 2.1.0)

Description

I'm sending audio data using UDP (ADPCM audio sampled at 8kHz). It transmits an average of 8 packets/second, each packet with 512 bytes. It is working well if I use a unicast address as destination (I'm in a class C network). If I use a broadcast address as destination (192.168.1.255), I found that the receiver looses many messages (I've included a message counter in the payload, and noticed a failure rate measured on the receiver side of around 50% to 60%). I used WireShark to analyse the network traffic and it also doesn't see the same messages the receiver looses, so it seems that the messages are not transmitted when i use a broadcast destination IP:

  IPAddress TxIp(192, 168, 1, 255);
  ....
  udp.beginPacket(TxIp, PORT_TX);
  udp.write(recBuf, REC_BUFFER_SIZE);
  Serial.println(udp.endPacket())

But works flawlessly when I use a unicast destination IP (exactly the same code, except the IP address):

  IPAddress TxIp(192, 168, 1, 196);
  ...
  udp.beginPacket(TxIp, PORT_TX);
  udp.write(recBuf, REC_BUFFER_SIZE);
  Serial.println(udp.endPacket())

Any idea of the reason? Is it necessary any special initialization for transmitting in broadcast?

Thanks!

Fernando

Settings in IDE

Module: ESP12E (NodeMCU)
Flash Size: 4MB
CPU Frequency: 80Mhz
Flash Mode: ?qio?
Flash Frequency: ?40Mhz?
Upload Using: SERIAL
Reset Method: ?ck / nodemcu?

[Bluebie]

The WiFi standard specifies unicast packets are transmitted reliably, with automatic retransmits when packet loss occurs. Your network protocols need to expect loss like this if they're ever going to run over WiFi with broadcast packets. This affects all WiFi devices, and is in no way specific to esp8266. You're expected to work around this limitation in your network protocol design.

If you're intending to keep using unmodified WiFi, I don't know of any way to request lower network layers reliably transmit broadcasts.

If you're transmitting ADPCM for audio, you'll have heaps of spare bandwidth. An esp8266 should send several megabytes per second, so unless you have a lot of receivers, you maybe fine just having receivers subscribe and transmitting once per receiver with unicast packets, gaining the built in retransmit system. If you're operating on a shared network you should be doing this anyway - broadcasting pointlessly at random smartphones and laptops sharing the network is going to hurt their battery life.

If you have too many receivers to unicast to each one, the simplest network protocol to work around this would be something that retransmits old packets for a few milliseconds in the gaps between new ones, soaking up that extra bandwidth and adding a time delay to the retransmits so hopefully any transient radio interference in the open unlicensed 2.4ghz spectrum will have passed by the time the retransmit happens. Keeping track of which receivers are missing which packets is going to be crazy complicated and likely not worth doing. You'll probably end up needing to retransmit nearly everything anyway because some wireless receivers will end up missing packets regardless.

Also keep in mind when designing broadcast protocols, if you have devices sending back acks, they're likely to create more in air collisions with the important transfers! Be very careful with timing to minimise this, or avoid frequent acks and response packets as much as you can if you're sensitive to broadcast data loss.

[Bluebie]

(this issue should be closed, unless there is some strong evidence it's specifically worse on esp8266)

[fmgomes]

I'm using an 'unreliable' UDP transmission and the number of packets leaving the node are the same in unicast and broadcast test scenarios. What I noticed was that on the broadcast scenario, about half of the packets aren't transmitted (the udp.endPacket() returns true, but no receiver, including wireshark in my PC sees these messages). So for the same traffic pattern, sending in unicast I didn't saw a single failure in hundreds of thousands of messages, sending in broadcast I saw 50% to 60% message failures, this is why I think there is some issue sending in broadcast, maybe I'm not doing it right. For my specific application I have implemented an alternative way, but the broadcast could be interesting for other projects, this is why I reported the issue

[dogmaphobic]

Shouldn't udp.write(recBuf, REC_BUFFER_SIZE) return the number of bytes actually written to the output buffer? Check and see if it matches REC_BUFFER_SIZE. I'm having similar issues.

[Bluebie]

@fmgomes Your test scenario isn't valid. UDP is transmitted reliably over WiFi by it's nature when packets are addressed to a unicast destination. In the real world your unicast messages are failing just as often, but the wifi drivers on all devices involved (including esp8266 and anything else it talks to) are automatically retransmitting lost packets and hiding this loss from you. This is not a bug. It's certainly not a bug in esp8266. It is in compliance with the WiFi specifications and is how all WiFi devices are expected to behave. Unless you are violating the WiFi specification and deeply messing with WiFi drivers, you have no means to transmit unicast udp packets unreliably. You cannot compare the two.

[fmgomes]

@Bluebie Ok, so in WiFi, the UDP messages (unacknowledged at the IP and upper levels) are acknowledged at the WiFi level (if they are unicast)? In the same test scenario, if the sender is my PC using WiFi I also don't see this high number of lost messages I observe with the ESP8266 as a sender, perhaps a different behavior of the WiFi stack? Nevertheless, we could close the issue.

[Bluebie]

@fmgomes Yeah that's right. WiFi will retransmit lost packets when those packets are addressed to a unicast destination at the IP layer, regardless of IP protocol (TCP, UDP, whatevs). This exists to ensure TCP-like protocols (including rudp, utp, and other TCP-over-UDP type protocols) will have mostly reliable transmission. That's super important because network routers when their small buffers are full will drop packets and usually not notify sending devices of the packets being dropped. TCP interprets packet loss as network being overloaded, and slows itself down drastically, then speeds up gradually until it looses data again, to try and stick close to the maximum rate the network can tolerate at any time. Broadcast protocols can't expect reliability, even over switched wired ethernet, because any TCP users bottlenecked by a network point like a modem or slower wired link will cause the router to drop packets sometimes as they detect network capacity and collectively share that capacity equally among heavy bandwidth users. Those packet drops affect all network applications whenever the router's buffer happens to be full.

If TCP experienced 50% packet loss it would slow to the minimum rate of the operating system, which is usually something like 0.5kb/s, so the lower layer retransmit system is essential to make it work. No important general purpose broadcast protocols expect reliable transmission - mostly it's used for very infrequent discovery requests with stuff like MDNS and BitTorrent Local Peer Discovery, which don't matter if they fail sometimes. Weird side effect - on a WiFi device, moving closer to the WiFi AP makes discovery apps like file shares on other pc's appear more quickly and reliably.

A cool side effect is cheap little embedded WiFi devices like the esp8266 can ignore the network for a while and do other stuff like disable interrupts and stream out colours to NeoPixels, because if they miss some packets while they're doing something else, the AP will retransmit the unicast ones later anyway, which are the only important ones.