SOLVED - ESP Webserver sometimes returns a blank white page, loop does not always mange handleClient() completely #5802

pedros89 · 2021-10-25T09:34:35Z

I'm trying to make a simple access point with ESP32 with a webpage displaying networks found; user can connect to a network and webserver displays connection status and power signal of that network.

The ESP32 launch a webserver
You connect to the access point
You see the webpage on your browser
You scan the wifi networks
you connect the ESP to a network and store in ESP memory the wifi credentials

See pictures

Hardware and system:

SparkFun ESP32 Thing Plus. Flash size: 16MB
Windows 10 64 bit with Arduino IDE 1.8.15
Espressif Arduino ESP32 Core 2.0.0 → library Webserver

This problem is becoming a nightmare because it happens only sometimes. It is a kind of instability. If I refresh the page often the white blank page remains but after some time it goes away without restarting the ESP, but sometimes it comes back.
I have no console error on the browser, just an empty file at root but my server is never supposed to send an empty string.

Steps:

When I start the ESP I can connect to the AP and everything is working well.
I press on "scan" and the webserver receive the request alright and shows me the network (sometimes is a bit slow, up to 6 seconds) but it always responds and the browser shows me the networks like in the pictures I posted
when I go back to the homepage from wifi scanner page button with a simple<a href='/' class='button1'>Homepage</a>
I get the blank white page

-there are no error messages on the browser console

I have done verbose debugging and the result is that at one point the client is not managed completely and from that point I have the blank page. From that point handleClient() is not completely managed anymore (see pic below). The thing is that if I go back retrying after a while without restarting the ESP I can find that the white page has gone and server is back to normal.

If I refresh the webpage it reloads exactly the same output on console, initially it handleclient() and then it does not and get stuck there where I put the red comment in the picture
I have also tried on another network and still sometimes the blank white page nightmare shows. I say that on my network I have a firewall.
I am also printing the free RAM and I see I have plenty.

Hardware:

Board: SparkFun Thing Plus 16M
Core Installation version: 2.0.0
IDE name: Arduino IDE
Flash Frequency: 40Mhz
PSRAM enabled: no (no external RAM)
Upload Speed: 512000
Computer OS: Windows 10

I placed in loop the print of every cycle only for debugging.
If I higher the delay in loop from 2 to 5 or bigger number the problem always represent. When I remove that delay the problem still represent

void handleRequest() {                        
  if (server.hasArg("ssid")&& server.hasArg("password")) {
    Serial.println(" wifi request arrived ");
    Serial.println(server.arg("ssid"));
    Serial.println(server.arg("password"));
    handleSubmitWifi();
  }
 
  if (server.hasArg("scan")) {
    Serial.println(" scan request ");
    handleScan();
  }
  
  checkConnGetSignal();

  String WEBPAGE = "<html><head><title>Big Website String here</title></head><body>IP address: "+IP+" Signal power is: "+signalPower+" Wifi On:"+wifiStateOn" >Wifi Off: "+wifiStateOff+" </body></html>";

  Serial.println(ESP.getFreeHeap());
  
  server.send(200, "text/html", WEBPAGE);
  
}

//function that convert the dBm to %
bool checkConnGetSignal(){
  if(WiFi.status() == WL_CONNECTED){
    wifiStateOn = "";
    wifiStateOff = "none";
    
    long dBm = WiFi.RSSI();
    long quality = 0;
    if(dBm <= -100){
        quality = 0;
    }
    if(dBm >= -50){
        quality = 100;
    }
    quality = 2 * (dBm + 100);
    if(quality > 100){
      quality = 100;
    }
    if(quality < 0){
      quality = 0;
    }
    signalPower = String(quality);
    signalPower = signalPower + "%";
    iAmConn = true;
    return true;
  }
  else{
    wifiStateOn = "none";
    wifiStateOff = "";
    iAmConn = false;
    return false;
  }
}


void handleScan(){
  Serial.println("scan start");
  Serial.println(millis());
  // WiFi.scanNetworks will return the number of networks found
  int n = WiFi.scanNetworks();
  Serial.println("scan done");
  Serial.println(millis());
  if (n == 0) {
      Serial.println("no networks found");
      String response_failure PROGMEM = "<html><head><title>WiFi Scanner Fail</title></head><body>WiFi Scanner Fail</body></html>";
      server.send(200, "text/html", response_failure);
  } else {
    Serial.print(n);
    Serial.println(" networks found");
    for (int i = 0; i < n; ++i) {
      // Print SSID and RSSI for each network found
      Serial.print(i + 1);
      Serial.print(": ");
      Serial.print(WiFi.SSID(i));
      if(i==0){network1 = String(WiFi.SSID(i));}     
      if(i==1){network2 = String(WiFi.SSID(i));}
      if(i==2){network3 = String(WiFi.SSID(i));}
      if(i==3){network4 = String(WiFi.SSID(i));}
      if(i==4){network5 = String(WiFi.SSID(i));}
      Serial.print(" (");
      Serial.print(WiFi.RSSI(i));
      Serial.print(")");
      Serial.println((WiFi.encryptionType(i) == WIFI_AUTH_OPEN)?" ":"*");
      delay(10);
    }
    String response="<html><head><title>WiFi Scanner Network list: "+network1+" "+network2+"  "+network3+"</title></head><body></body></html>";
    
    server.send(200, "text/html", response);
    Serial.println("page sent");
    Serial.println(millis());
  }
  Serial.println("");
}

void setup() 
  {
 
    String wifiMacString = WiFi.macAddress();
    Serial.println("Unique MAC is:");
    Serial.println(wifiMacString);
    String code = wifiMacString.substring(12);
    Serial.println(code);     
    ssidAPStr = ssidAPStr + code;
    Serial.println(ssidAPStr);
 
  int ssid_AP_len = ssidAPStr.length()+1;
  char ssidAP[ssid_AP_len];
  ssidAPStr.toCharArray(ssidAP, ssid_AP_len);     //convert the unique ssid code from string to char

  //configure the Acces point with the 192.168.123.123 as requested
  WiFi.mode(WIFI_AP);                               //don't do set ESP32 as access point and wifi scanner at the same time
  WiFi.softAP(ssidAP, passwordAP);                  //launch the access point
  Serial.println("Wait 100 ms for AP_START...");    //important delay so Esp has time to set IP to 192.168.123.123
  delay(100);
  Serial.println("Setting the AP");
  IPAddress Ip(192, 168, 123, 123);    //set IP AP
  IPAddress NMask(255, 255, 255, 0);
  WiFi.softAPConfig(Ip, Ip, NMask);
  
  IPAddress myIP = WiFi.softAPIP();
  Serial.println(myIP);             //print the IP of access point
  String AP_IP = myIP.toString();

 connectToWifi();

    Serial.println("\nLocal IP is:");
    Serial.println(WiFi.localIP());         // Print ESP32 Local IP Address
 
    IP = WiFi.localIP().toString();
    if((IP!="0.0.0.0")&&(IP!="not connected")&&(IP!=AP_IP)){
      wifiStateOn = "";
      wifiStateOff = "none";

    }
    if((IP=="0.0.0.0")||(IP=="not connected")){
      wifiStateOn = "none";
      wifiStateOff = "";
      IP="not connected";
    }    

  Serial.println(wifiStateOn);
  Serial.println(wifiStateOff);

//launch server  
  server.on("/", handleRequest);

  server.onNotFound(handleNotFound);
  
  server.begin();
  
  Serial.println("HTTP server launched");
//end launch server

}


void loop() {

  server.handleClient(); 
  //Serial.println("I loop");
  delay(2);
  
}

Debug Messages:

189583
[ 54103][V][WebServer.cpp:294] handleClient(): New client: client.localIP()=192.168.123.123
[ 57387][V][Parsing.cpp:121] _parseRequest(): method: GET url: / search: 
[ 57387][V][Parsing.cpp:227] _parseRequest(): headerName: Host
[ 57388][V][Parsing.cpp:228] _parseRequest(): headerValue: 192.168.123.123
[ 57394][V][Parsing.cpp:227] _parseRequest(): headerName: Connection
[ 57400][V][Parsing.cpp:228] _parseRequest(): headerValue: keep-alive
[ 57407][V][Parsing.cpp:227] _parseRequest(): headerName: Cache-Control
[ 57413][V][Parsing.cpp:228] _parseRequest(): headerValue: max-age=0
[ 57419][V][Parsing.cpp:227] _parseRequest(): headerName: Upgrade-Insecure-Requests
[ 57426][V][Parsing.cpp:228] _parseRequest(): headerValue: 1
[ 57432][V][Parsing.cpp:227] _parseRequest(): headerName: User-Agent
[ 57438][V][Parsing.cpp:228] _parseRequest(): headerValue: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36
[ 57453][V][Parsing.cpp:227] _parseRequest(): headerName: Accept
[ 57459][V][Parsing.cpp:228] _parseRequest(): headerValue: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9
[ 57476][V][Parsing.cpp:227] _parseRequest(): headerName: Accept-Encoding
[ 57482][V][Parsing.cpp:228] _parseRequest(): headerValue: gzip, deflate
[ 57489][V][Parsing.cpp:227] _parseRequest(): headerName: Accept-Language
[ 57495][V][Parsing.cpp:228] _parseRequest(): headerValue: it-IT,it;q=0.9,en-GB;q=0.8,en;q=0.7,en-US;q=0.6
[ 57505][V][Parsing.cpp:255] _parseArguments(): args: 
[ 57510][V][Parsing.cpp:238] _parseRequest(): Request: /
[ 57514][V][Parsing.cpp:239] _parseRequest():  Arguments: 
1383136727


[ 57546][V][WebServer.cpp:294] handleClient(): New client: client.localIP()=192.168.123.123

The text was updated successfully, but these errors were encountered:

pedros89 · 2021-11-16T16:10:36Z

I have tried to print the string to the serial monitor with .
Serial.println(WEBPAGE);

I noticed the error I got is
E (49837) uart: uart_write_bytes(1159): buffer null
so this is the reason why the string feeded to the page becomes an empty string and the page displays a blank page.
I started form the assumption that my string is too long because with other shorter strings I don't have that error, my string is about 60k characters and the result of many concatenations like below:

String WEBPAGE = "a lot of html here, and many in line conversions like"+Stirng(multiArray[5][6])+"quite a lot of javascript here"+someStirngVar+"so it ends very long";

I have looked at the server.send() function (here)
The sendContent is the function that actually sprintf and of course it places a /r/n at the end as footer to end HTTP transmission.
I thought I could send data manually, I tried unsuccessfully like this:

WiFiClient client;
while (client.connected(){
    if (client.available()){
  client.println("HTTP/1.1 200 OK");
  client.println("Content-Type: text/html");
  client.println("Connection: close");
  client.print(WEBPAGE_PAR1);  
  client.println(WEBPAGE_PAR2);
  }
}

The idea above inspired form Random Nerd Tutorial does not work.

I also realized that the problem was not having a 60k chars string because I generated one and sent it with server.send(200, "text/html", BIGSTRING); but the chars where displayed correcly on the browser without E (49837) uart: uart_write_bytes(1159): buffer null error
The problem seems to be when we use very big strings that concatenate with other strings variables (sometimes with the String(intVariableToConvert) function as shown above) and we concatenate in one single massive string on one single line.

After some reaseraching I landed here and I found the solution. For which I am extremely grateful towards mr @Pablo2048.
Just forget server.send(200, "text/html", BIGSTRING); and use the code below instead, it works like a chram. Remember to use also the headers, if you use the code below without headers it will not work and still result in a white blank page. I hope this will help someone in the future.

  server.sendHeader("Cache-Control", "no-cache, no-store, must-revalidate");
  server.sendHeader("Pragma", "no-cache");
  server.sendHeader("Expires", "-1");
  server.setContentLength(CONTENT_LENGTH_UNKNOWN);
  // here begin chunked transfer
   server.send(200, "text/html", "");
  server.sendContent(WEBPAGE_BIG_0); 
  server.sendContent(WEBPAGE_BIG_1); 
  server.sendContent(WEBPAGE_BIG_2); 
  server.sendContent(WEBPAGE_BIG_3);
  server.sendContent(WEBPAGE_BIG_4);
  server.sendContent(WEBPAGE_BIG_5);
  server.client().stop();

I have chunked my page in 5 pieces and each is a string formed by lots of concatenations. No more issues this way.

If you are looking for a automatic way to chuck you html webpage, I have made python3 script that does that and returns the chunks in a .txt file.
See the python3 code below or find it here.

#decide the character number for every division    
divideEvery = 100

myHtmlString = "<!DOCTYPE html><html lang='en'><title>W3.CSS Template</title><meta charset='UTF-8'><meta name='viewport' content='width=device-width, initial-scale=1'><link rel='stylesheet' href='https://www.w3schools.com/w3css/4/w3.css'><link rel='stylesheet' href='https://fonts.googleapis.com/css?family=Lato'><link rel='stylesheet' href='https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css'><style>body {font-family: 'Lato', sans-serif}.mySlides {display: none}</style><body></body></html>"

myLength = len(myHtmlString)
division = myLength/divideEvery
print("number of divisions")
print(division)

carry = myLength%divideEvery
print("characters in the last piece of string")
print(carry)

f = open("result.txt","w+")
f.write("Below the string splitted \r\n")
f.close()

x=myHtmlString
n=divideEvery
myArray=[]
for i in range(0,len(x),n):
    myArray.append(x[i:i+n])
#print(myArray)

for item in myArray:
    f = open('result.txt', 'a')
    f.write(item+ '\n'+ '\n')

f.close()

I also want to add that it is better to not make a string but plug indirectly the Webpage in the function

Not like this

String  WEBPAGE_BIG_4 ="<html><head>put your code here</head><body>more code</body><script>your js</script></html>";
server.sendContent(WEBPAGE_BIG_4);

Like this is better because more efficient and it is also faster
server.sendContent("<html><head>put your code here</head><body>more code</body><script>your js</script></html>");

pedros89 changed the title ~~ESP Webserver sometimes returns a blank white page, loop does not always enter handleClient() but keeps looping~~ ESP Webserver sometimes returns a blank white page, loop does not always mange handleClient() completely Oct 25, 2021

pedros89 closed this as completed Nov 17, 2021

pedros89 changed the title ~~ESP Webserver sometimes returns a blank white page, loop does not always mange handleClient() completely~~ SOLVED - ESP Webserver sometimes returns a blank white page, loop does not always mange handleClient() completely Nov 17, 2021

pedros89 mentioned this issue Jun 20, 2022

Conflict between espMqttClient.h and HTTPClient.h and update.h which use WiFiClientSecure.h SORTED bertmelis/espMqttClient#12

Closed

realA10001986 mentioned this issue Apr 21, 2023

handleParam() - memory issue? (Blank Page in browser) tzapu/WiFiManager#1594

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SOLVED - ESP Webserver sometimes returns a blank white page, loop does not always mange handleClient() completely #5802

SOLVED - ESP Webserver sometimes returns a blank white page, loop does not always mange handleClient() completely #5802

pedros89 commented Oct 25, 2021 •

edited

pedros89 commented Nov 16, 2021 •

edited

SOLVED - ESP Webserver sometimes returns a blank white page, loop does not always mange handleClient() completely #5802

SOLVED - ESP Webserver sometimes returns a blank white page, loop does not always mange handleClient() completely #5802

Comments

pedros89 commented Oct 25, 2021 • edited

Hardware:

Debug Messages:

pedros89 commented Nov 16, 2021 • edited

pedros89 commented Oct 25, 2021 •

edited

pedros89 commented Nov 16, 2021 •

edited