Strange WiFiClient.remoteIP() == 0

[kirillandy]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: [ESP8266]
• Core Version: [2.7.1]
• Development Env: [Visual Micro (Visual Studio)]
• Operating System: [Windows 10]

Settings in IDE

• Module: [Generic ESP8266 Module]
• Flash Mode: [dout]
• Flash Size: [1MB]
• lwip Variant: [v2 Lower Memory]
• Reset Method: [nodemcu]
• Flash Frequency: [40Mhz]
• CPU Frequency: [80Mhz]
• Upload Using: [SERIAL]
• Upload Speed: [115200]

Problem Description

EDIT:
After trying out different options with my original attempts at a reproducible example, I finally forced the behavior I was encountering. I've updated my core and IDE and re-written the summary of the problem:
I have a WifiClient communicating with a WiFiServer. The WifiClient connects->sends->disconnects (I call this 'notifying', since the client only sends the server a little something and doesn't expect any response). The WiFiServer never disconnects but only waits for the Client to disconnect (to avoid the server hanging for a few seconds, this is a TCP thing). Occasionally, when I retrieve the WiFiClient using server.available(), the remoteIP() function returns an address of 0.0.0.0, even though all the transmitted data is available for me to read. This is a problem because I heavily rely on being able to identify the client that sent the data in the first place. Getting an address of 0 prevents me from doing this...

I cannot 100% guarantee that the example below will re-create this behavior because I have noticed very strange variations. My first experiments were done using 2 boards (say A and B). These would trigger the problem 3-4 times depending on the delay in the Client code (see below).
I then tried the exact same experiment on 2 other boards (C and D). I only managed to get 1 NULL IP address out of a grand total of ~2000 (!!!) transmissions (connect->send->disconnect sessions)

HOW DOES THE EXAMPLE WORK?
There are two setup-loop pairs: one for the client and one for the server (comment out either one when uploading). There are also a few functions. Leave them. Change cicadaIP to the IP address of the server in your network before uploading the client code (due to DHCP)
The client waits for you to send 2 bytes over Serial: a character from 0x30 - 0x33 and a number.
The character will be used as an index to choose a string in the m array. The number is used as the loop limit.
The client will send the character 'number' times by connecting->sending->disconnecting over and over with a delay X.
The server will receive the sent character and use it to choose a message from array m, sending it over Serial. If the remoteIP() == 0, it will also send a "IP address == 0!" message as well as the session number.
In my first experiment, I managed to get the described behavior with delay X == 30 and 60. X == 30 gave slightly more common occurrences. My second experiment only managed to get it 1 time for X == 30 (a significant but unexplained improvement)
I will follow up with a few screenshots capturing the problem

###MCVE

#define LARGEST_LONG 0xFFFFFFFFL
#define FLUSH_TIMEOUT 500000
#include <ESP8266WiFi.h>

IPAddress cicadaIP(0x1F00A8C0);
WiFiServer cicadaServer(1);
WiFiClient serverClient;

char router_ssid[] = [YOUR ROUTER SSID], router_pass[] = [YOUR ROUTER PASSWORD];

//These are just silly strings that I use as messages on Serial to show that something is actually happening on the server side (don't ask how I picked them)
char m1[] = "message1";
char m2[] = "note2";
char m3[] = "a formal invitation 3";
char m4[] = "a strange hieroglyph 4";
char def[] = "This is not a recognized command";
char* m[] = { m1,m2,m3,m4 };
unsigned int sizes[] = { sizeof(m1) - 1, sizeof(m2) - 1, sizeof(m3) - 1, sizeof(m4) - 1 };

//I use this to calculate how much time has elapsed after a call to delay()
unsigned long calcTimeDifference(unsigned long timeStart, unsigned long timeStop) {
	if (timeStop < timeStart) {
		return((LARGEST_LONG - timeStart) + timeStop);
	}
	else {
		return(timeStop - timeStart);
	}
}
//These are my own functions that I use to connect/disconnect/write/read
unsigned int wifiConnect(WiFiClient* client, unsigned int myPort, unsigned int ipAddr, unsigned int port) {
	unsigned int ret;

	IPAddress* addr = new IPAddress(ipAddr);
	WiFiClient::setLocalPortStart(myPort);
	ret = 1;
	if ((*client).connect(*addr, port)) { ret = 0; }
	delete addr;
	return (ret);
}

unsigned int wifiSend(WiFiClient* client, char* buf, unsigned int len) {
	unsigned long time;
	unsigned i;

	time = millis();
Mloop:
	if ((*client).connected())
	{
		i = (*client).write(buf, len);
		if (i == len) { return(0); }

		while (!((*client).flush(0)))
		{
			if (calcTimeDifference(time, millis()) >= FLUSH_TIMEOUT) {
				return (1);
			}
			delay(60);	// yield();
		}
		return 0;
	}
	if (calcTimeDifference(time, millis()) >= FLUSH_TIMEOUT) {
		delay(20);
		goto Mloop;
	}
	return (2);
}

unsigned int wifiRead(WiFiClient* client, char* buf, unsigned int len, unsigned int timeout) {
	unsigned long time;
	int j;

	time = millis();

	while ((*client).available() < len)
	{
		if (calcTimeDifference(time, millis()) > timeout)
		{
		Merr:
			j = (*client).available();
			if (j == 0) { j = 0xFFFFFFFF; }
			return(j);
		}

		if (!((*client).connected())) {
			goto Merr;
		}
		delay(60);	// yield();
	}

	(*client).readBytes(buf, len);
	return 0;
}

unsigned int wifiDisconnect(WiFiClient* client) {
	(*client).stop();
	while ((*client).status() != CLOSED) {
		delay(60);	// yield(); 
	}
	return 0;
}
/*********************************************
Send message to a server with no reply (connect -> send -> disconnect).
I:
	client - client to use for transmission
	myPort - TCP port of client
	ipAddr - address of server
	port - port of server
	buf - buffer with data to send
	len - length of data
*********************************************/
unsigned int wifiNotify(WiFiClient* client, unsigned int myPort, unsigned int ipAddr, unsigned int port, char* buf, unsigned int len) {
	unsigned int i, er;

	// Just an extra close if it's necessary
	if ((*client).status() != CLOSED) { wifiDisconnect(client); Serial.println("Needed to disconnect"); }

	er = wifiConnect(client, myPort, ipAddr, port);
	if (er != 0) { goto Mret; }
	Serial.println("Connected!");

	if ((*client).status() != ESTABLISHED) { goto Mret; }
	er = wifiSend(client, buf, len);
	if (er != 0) { goto Mret; }
	Serial.println("Sent data");

	wifiDisconnect(client);
	Serial.println("Disconnected");
	er = 0;

Mret:
	if (er != 0) {
		Serial.printf("Couldn't communicate, error %d", er);
		Serial.println();
	}
	return er;
}
//I'm using DHCP, so change the server IP in the client code accordingly
//CLIENT SIDE CODE//
void setup(){
	WiFiClient::setDefaultNoDelay(true);
	WiFiClient::setDefaultSync(true);

	delay(5000);
	Serial.begin(115200);
	WiFi.begin(router_ssid, router_pass, 6);
	while (WiFi.status() != WL_CONNECTED) {
		Serial.print(".");
		delay(500);
	}
	Serial.println("Connected!");
	Serial.print("IP address, gateway: ");
	Serial.println(WiFi.localIP());
	Serial.println(WiFi.gatewayIP());
}

void loop(){
	unsigned char buf[10]; unsigned int i, work1;
Mretry:
	while (Serial.available() < 2) { //Only begin execution when I tell you to by sending a byte over Serial
		delay(200);
		goto Mretry;
	}
	
	//The byte is a symbolic index into the 'm' array of messages (see above)
	//It must be a HEX value from 0x30-0x33, otherwise you'll jump back to the wait loop above
	Serial.println("Got a command over Serial!");
	Serial.read((char*)buf, 2);
	buf[0] ^= 0x30;
	if (buf[0] > 3) { 
		Serial.println("Invalid command!");
		goto Mretry;
	}

	//Send the chosen message over buf[1] TCP sessions with a delay in between
	for (i = 0; i < buf[1]; i++) {
		work1 = wifiNotify(&serverClient, 2, cicadaIP, 1, m[buf[0]], sizes[buf[0]]);
		delay(30); //<- THIS DELAY IS IMPORTANT!!!!!! I talk about it in the description
	}
	
	goto Mretry;
}

//SERVER SIDE CODE
/*void setup()
{
	WiFiClient::setDefaultNoDelay(true);
	WiFiClient::setDefaultSync(true);

	delay(5000);
	Serial.begin(115200);
	WiFi.begin(router_ssid, router_pass, 6);
	while (WiFi.status() != WL_CONNECTED) {
		Serial.print(".");
		delay(500);
	}
	Serial.println("Connected!");
	Serial.print("IP address, gateway: ");
	Serial.println(WiFi.localIP());
	Serial.println(WiFi.gatewayIP());

	cicadaServer.begin();
	while (cicadaServer.status() != LISTEN) {
		delay(100);
	}
}

void loop()
{
	unsigned int work1, counter = 0;
	unsigned char buf[30]; //A receive buffer, just 10 bytes long
Mretry2:
	serverClient = cicadaServer.available(); //Try getting a connected client
	if (!serverClient) {
		delay(100);
		goto Mretry2;
	}

	work1 = serverClient.remoteIP();
	if (work1 == 0) {
		Serial.print("IP address == 0! ");
		Serial.printf("Counter %d\r\n", counter);
	}

	work1 = serverClient.available(); //Check receive buffer and place first 10 bytes into our buffer. Send bytes over Serial
	if (work1 != 0) {
		if (work1 > 30)
			work1 = 30;
		serverClient.readBytes(buf, serverClient.available());
		Serial.write(buf, work1);
		Serial.println();
	}
	else {
		Serial.println("No bytes in reception buffer!");
	}
	
	while(serverClient.status() != CLOSED){
		yield();
	}
	counter++;
	goto Mretry2; //Go back to the Serial wait loop above
}*/

[kirillandy]

Here are the results from my first experiment: boards A and B communicating.
Transmitting a request 50 times with a delay of X == 30 gave 1 reset IP address during session 39

Transmitting a request 100 times more gave 2 zero IPs (session 50 and 53). What's remarkable is the fact that the VERY FIRST transmission was a hiccup! How this happens I have no idea. I spent a good few seconds changing the client parameters in between the first 50 messages and the next 100 so there was enough time for things to 'settle down'.

Using 2 different boards (C and D) with a delay of 30 only gave 1 (!!!) wrong IP over ~2000 sessions (session 336, I only captured a part of them in the screenshot but I kept at it for >2000 transmissions with no further errors)

The bottom line (I think) is: what could cause the tcp_pcb of the WiFiClient ClientContext to be reset? Studying the ClientContext.h file showed me that the tcp_pcb structure is the source of the IP address when .remoteIP() is called.

[devyte]

@kirillandy your client is connect-send-closing without expecting any response. If the connection is closed before you handle it on the server, the server-side client won't have a remote side (it's already done with being notified as far as TCP is concerned), so you get a remoteIP of 0.0.0.0.
You can still read the data out of the server-side client because it's contained in the internal buffers, but at that point you can't get info on the remote client because it's already gone.
What you see is working as intended.
Why that happens only sometimes has to do with how you structured your code. Essentially, you have a race condition between your client and your server.

At a glance, I see several choices for you:

• change the notify to expect a reply, change the server to send it. The client would know that the server handled the message, so can freely close the connection.
• package the remoteIP as part of the message, so that it's available aftwr reading feom tje server-side client.
• ignore the lack of remoteIP (maybe you don't need it?)

Closing due to not an issue.

[kirillandy]

Ah, I see. And to think I spent hours poking around trying to get it to give a 0 🤦
I've modified the test a bit and made the server print out the received messages after the Client is done sending all of them and I got zeroes all across the board.
I wonder: is this also something done in other TCP APIs like WinSock? I should keep that in mind and check once I get there...

As for your possible solutions:

1. That's what I had before and it was even nastier. I have a "wifi_exchange" function that does exactly that and makes both sides send 1 message. The only problem was that it caused a really painful deadlock in the 'proper' circumstances (I won't go into the details), so I'd much rather deal with a race condition 😆
2. Yeah, that crossed my mind as soon as I got my first 0 IPs, but it felt like a patch for improper TCP behavior so I decided to determine whether this was in fact a bug before changing my message formats. It is a bit unusual to be able to get a message but to have lost the ID of the client that sent it. Seems like I might opt to do this now.
3. No, not an option. I've got a network of devices communicating with one another and they trigger physical events depending on who sent the message so it's imperative that I know who it was. Otherwise it might cause a bit of chaos 😋

Thank you for looking into this @devyte, much appreciated 👍