ESP8266WebServer works with limited number of browsers

[tpillera]

Basic Infos

I have developed a web server that serves a webpage stored in flash (SPIFFS) to a browser. Overall this works great but only with two browsers, Windows Internet Explorer and iPhone Safari. On PC Firefox, Chrome, iPad Safari, and Mac Safari it is either slow and times out or does not work at all.

I thought about and started re-writing this using \arduino-1.8.0\hardware\esp8266com\esp8266\libraries\ESP8266WiFi but this does not easily support all of the GET and POST data transfer that ESP8266WebServer does.

Hardware

Hardware: NodeMCU/ESP-12 (version 0.9 and 1.0)
Core Version: 1.8.0 libraries

Description

Problem description

• ESP8266 is set up as access point
• All devices (PC, Mac, iPhone, iPad) connect with no apparent problem
• Only certain browsers work (see basic description above), other browsers do not work

Settings in IDE

Module: NodeMCU 1.0
Flash Size: 4M(3M SPIFFS)
CPU Frequency: 80Mhz
Flash Mode: SPIFFS
Flash Frequency: ?
Upload Using: Serial
Reset Method: power on

Sketch

#include <Arduino.h>

/*
********************************************************
* Name        : setup
* inputs      : nil
*             :
* returns     : nil
* Description : Set up main actions.
********************************************************
*/
void setup(void) 
{
  // LEDs on board.
  const short int BUILTIN_LED1 = 2;
  const short int BUILTIN_LED2 = 16; 
  
  // Wait a second.
  delay(1000);
	
  // Set up serial port.
  Serial.begin(115200);
	Serial.println (" ");
  
  // Start the file system.
  if (true != SPIFFS.begin ())
    {
    DBG_PRINTF2(DEBUG_FATAL, "PROBLEM OPENING SPIFFS AT: ", __LINE__);
    reportError(__FILE__, __LINE__);
    
    // Fatal error, stop.
    while (1) {delay(1000);}
    }

  // Initialize GPIO and LCD functionality.
  gpio_init ();
  initLCD ();
  
  // Initialize the BUILTIN_LED's pin as outputa.
  pinMode(BUILTIN_LED1, OUTPUT);  
  pinMode(BUILTIN_LED2, OUTPUT);  
  
  // If button down is pressed then wifi mode, otherwise run mode.
  if ((LOW == digitalRead(BUTTON_DOWN)) || (LOW == digitalRead(BUTTON_UP)))
    {
    DBG_PRINTF1(DEBUG_INFO, "CONFIGURING ACCESS POINT...");
    
    //Our ESP8266-12E is an AccessPoint.
    WiFi.mode (WIFI_AP); 
    WiFi.softAP(ssid, password);
    server.begin();
    delay(250);
    
    IPAddress myIP = WiFi.softAPIP();
    int32_t wifi_channel = WiFi.channel();
    DBG_PRINTF4(DEBUG_DIAGNOSTIC, "AP IP ADDRESS: ", myIP, ", CHANNEL: ", wifi_channel);
    
    // Display PROGRAM MODE on LCD.
    clearLCD ();
    displayOutput (ROW0, "PROGRAM MODE");
    displayOutput (ROW1, "192.168.4.1");
    
    // Server actions.
    server.on("/", readHTML);
    server.on("/save", saveTable);
    server.on("/load", loadTable);
    server.on("/delete", deleteTable);
    
    // Called when the url is not defined here
    // Use it to load content from SPIFFS
    server.onNotFound([](){
      if(!handleFileRead(server.uri()))
        {
        server.send(200, "text/plain", "File Not Found");
        DBG_PRINTF2(DEBUG_ERROR, "FILE NOT FOUND AT: ", __LINE__);
        }
    });
    
    DBG_PRINTF1(DEBUG_INFO, "HTTP server started");
    
    // Turn the LED ON by making the voltage LOW
    digitalWrite(BUILTIN_LED1, LOW);  
    digitalWrite(BUILTIN_LED2, HIGH);  
    }
  else
    {
    DBG_PRINTF1(DEBUG_INFO, "CONFIGURING RUN MODE...");
    
    // Indicate not wifi mode.
    wifi_startup = false;

    // Turn the LED ON by making the voltage LOW
    digitalWrite(BUILTIN_LED1, HIGH);  
    digitalWrite(BUILTIN_LED2, LOW);  
    }
}


/*
********************************************************
* Name        : loop
* inputs      : nil
*             :
* returns     : nil
* Description : Main loop.
********************************************************
*/
void loop() 
{
  // Depending on mode run wifi or runtime enviromnent.
  if (true == wifi_startup)
    {
    server.handleClient();
    }
  else
    {
    run_setlist ();
    }  
}

Debug Messages

GOOD PASS SHOWN BELOW (serial debug output)

handleFileRead: /html/index.html
streamFile done
FREE HEAP = : 28632
THIS SKETCH SIZE = : 318048
FREE SKETCH SPACE = : 729088
handleFileRead: /css/bootstrap.min.css
streamFile done
handleFileRead: /css/style.css
streamFile done
handleFileRead: /js/jquery-2.2.0.min.js
streamFile done
handleFileRead: /js/bootstrap.min.js
streamFile done
handleFileRead: /js/underscore-min.js
streamFile done
handleFileRead: /fonts/glyphicons.eot
streamFile done
handleFileRead: /images/blurredimage.jpg
streamFile done
handleFileRead: /js/index.js
streamFile done

BAD PASS SHOWN BELOW
handleFileRead: /html/index.html
streamFile done
FREE HEAP = : 28400
THIS SKETCH SIZE = : 318048
FREE SKETCH SPACE = : 729088
handleFileRead: /css/bootstrap.min.css


Times out...

[bebo-dot-dev]

What does the html in the page being served by the ESP device look like? If your html contains regular script and CSS link tags, this markup would cause the browser to make many requests to your device in rapid succession possibly overwhelming the device

[tpillera]

Setlist Creator.zip
Joe,
Yes it contains those links. I can see it loading them correctly for IE but for others it just times out. I tried adding a queue to stack the requests but it didn't solve the problem.

[bebo-dot-dev]

This is a problem that I've encountered. The page will appear to work / the firmware will appear to serve pages successfully in some browsers and not in others but it can also be intermittently flakey in those browsers you believe it to be stable with.

I believe this problem is worse with some browsers compared to others because some of the more popular modern browsers attempt to be more efficient by parallelizing multiple requests for additional resources from the client side which puts the device under a more extreme load which I've noticed can sometimes cause a reset. Of course this browser technique works great when making requests to full fat web servers..

My solution to this problem was to directly inline all js, css, image and font resources into the html page itself and gzip that html too. Use inlined base64 css data uris for any required images and fonts.

One of my gzipped pages is typically about 1/4 the size of the non gzipped html + the sum of all the non gzipped css and js resources and it's all delivered in a single request / response which hugely reduces the load on the little ESP device.

IMO this is the way to go but it's worth pointing out that I've suggested this as a solution to others in the past and it's been rejected because there's an opinion that there is a resident bug in the ESP8266WebServer that should be fixed rather than inlining resources which is considered to be just working around the problem. That said, if this is a bug it remains a bug today, I'm happy that my solution is "good enough" and my response to these people would be take it or leave it - YMMV.

If you do fancy following the inlining / gzipping approach to fix this, this excellent article about using gulp to help out is well worth a read: http://tinkerman.cat/optimizing-files-for-spiffs-with-gulp/

[tpillera]

Thanks Joe!! There's likely a bug in ESP8266WebServer but I'm for whatever makes this work :) and don't really care to wait for a fix as there are a number of issues probably ahead of this one.

I do some dynamic creation of the webpage based on stored SPIFFS data so I'll have to see how that plays out. The actual dynamically created html file turned out to be too large and took up all of RAM space so i had to write it line by line to a temporary file in SPIFFS and then stream that file. Possibly could include the js/css in the written file and stream it as a single file, will have to think about the best way to do this.

There is an example of this webpage (below) on my site that stores the data in MySQL instead if SPIFFS. Had to modify this to work on the ESP...has been a challenge.
http://www.pilleraengineering.com/project/html/

Thanks again, will give this a try as seems like the most reliable solution.

Best Regards,
Ted

[bebo-dot-dev]

No problem you're welcome.

Whilst on the subject of dynamic pages based on data stored in SPIFFS / flash / in-memory variables etc, I've seen a few different approaches to crack this. Some people seem to go for a solution design based on a html "template" of sorts where the html contains string placeholders in positions that get replaced in C/C++ code.

My own approach to this is an ajax pattern where html / css / js gets delivered as previously described above and the dynamic part of the page then gets plugged in via a secondary ajax request / response + some jquery after the page has been fully served / rendered. This method works reliably with the ESP8266WebServer.

This pattern does force you to think about implementing a design pattern around ESP8266WebServer handlers for all of the ajax operations you need in your pages along with a js pattern for making these requests / handling the responses appropriately but if you like the sound of this, I wrote it up in a bit more detail here: https://jjssoftware.github.io/dynamic-web-forms/

[vicnevicne]

FWIW, my solution was to store all static files on a "fat server" somewhere on the net and only keep the dynamic pages on the ESP. This of course only works if the ESP is in STA mode and you'll have to work around cross-site security settings.

[tpillera]

Hello Joe, Just an update…I copied all minimized .css files together into a single file and gzipped them and did the same with the .js files. This created 1-each .css.gz and .js.gz file in contrast to 6-individual much larger files that I started with. I tested it with Windows Firefox, IE, Windows Safari, and iPhone Safari and all browsers load the page correctly (see load data below). I’m an embedded design engineer so I know through years of battle hardened experience that this is not a conclusive fix and is going to require much more testing. It is however a positive step and possibly with some slight tweaking can be turned into a reliable solution. Also, I’m not as confident with this solution as I would be with your suggestion of creating a single file, however since my dynamic code is basically inserted in the middle of the html with a C++ program, so this seemed like the least painful way to go. Hopefully will test out on all platforms and can move on to the next issue. Best Regards, Ted FILESIZE = 2584 STRING SIZE = 2576 FREE HEAP = : 21280 handleFileRead: /html/index.html File size = 34189, Context = text/html streamFile done FREE HEAP = : 27592 THIS SKETCH SIZE = : 318240 FREE SKETCH SPACE = : 729088 handleFileRead: /css/default.css.gz File size = 20324, Context = text/css streamFile done handleFileRead: /js/default.js.gz File size = 47376, Context = application/javascript streamFile done handleFileRead: /images/blurredimage.jpg File size = 6140, Context = image/jpeg streamFile done handleFileRead: /fonts/glyphicons.woff2 File size = 18028, Context = text/plain streamFile done From: Joe [mailto:notifications@github.com] Sent: Wednesday, April 05, 2017 5:09 PM To: esp8266/Arduino <Arduino@noreply.github.com> Cc: tpillera <info@pilleraengineering.com>; Author <author@noreply.github.com> Subject: Re: [esp8266/Arduino] ESP8266WebServer works with limited number of browsers (#3119) No problem you're welcome. Whilst on the subject of dynamic pages based on data stored in SPIFFS / flash / in-memory variables etc, I've seen a few different approaches to crack this. Some people seem to go for a solution design based on a html "template" of sorts where the html contains string placeholders in positions that get replaced in C/C++ code. My own approach to this is an ajax pattern where html / css / js gets delivered as previously described above and the dynamic part of the page then gets plugged in via a secondary ajax request / response + some jquery after the page has been fully served / rendered. This method works reliably with the ESP8266WebServer. This pattern does force you to think about implementing a design pattern around ESP8266WebServer handlers for all of the ajax operations you need in your pages along with a js pattern for making these requests / handling the responses appropriately but if you like the sound of this, I wrote it up in a bit more detail here: https://jjssoftware.github.io/dynamic-web-forms/ — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#3119 (comment)> , or mute the thread <https://github.com/notifications/unsubscribe-auth/AM2LqFb_a-53eqCnZAWAUAzx_mtBmT_Jks5rtBDfgaJpZM4M0i6M> . <https://github.com/notifications/beacon/AM2LqKUGMK6cHsfcfr2FEBJFqS4N-y3_ks5rtBDfgaJpZM4M0i6M.gif>

[bebo-dot-dev]

Great sounds like a move in the right direction and some progress. If nothing else we did at least appear to correctly anaylse that the volume of separate http requests was the cause of your original reported problem.

[devyte]

@tpillera please retry with latest git. I believe that the ESP8266Webserver recently received several fixes.
If the issue is no longer valid, or no longer applies to you, please close it.

[earlephilhower]

Over 2 years and a very vague problem report. Closing. If there is an issue with a specific browser and current release/GIT, please open a new issue and fill out the template.