Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Portmapping prohibits loading index.html #16

Closed
Marc-Vreeburg opened this issue Oct 15, 2018 · 16 comments
Closed

Portmapping prohibits loading index.html #16

Marc-Vreeburg opened this issue Oct 15, 2018 · 16 comments

Comments

@Marc-Vreeburg
Copy link

I'm using AutoConnect with static ip address. First starting with captive portal to obtain a dynamic ip-address. Then setting static ip address. It's working great, except for remote access to the index.html page on my esp8266.
While within my lan index.html can be reached direcly, from wan/internet/outside my lan it seems portmapping forces startup with captive portal, i.e. /_ac. Can this be fixed?

@Hieromon
Copy link
Owner

Hieromon commented Oct 16, 2018

I think you are using port-forwarding via static IP at the router to access ESP8266 from WAN.
So...

While within my lan index.html can be reached direcly, from wan/internet/outside my lan it seems portmapping forces startup with captive portal,

Yes, it redirects to /ac if the IP contained in the host header of the http request is different from the IP of ESP8266. This code corresponds to AutoConnect::_captivePortal().

if (!_isIP(hostHeader) && (hostHeader != WiFi.localIP().toString())) {

But that logic is tied to the onNotFound of the base class ESP8266WebServer.
Therefore I think that the base ESP8266WebServer library had not found the index.html. Can you access to /index.html on ESP8266 from WAN without AutoConnect? There may be some way if that is possible.
Can you take a debug log HTTP_SERVER and AutoConnect? To enable debug log, activate Arduino-IDE option as Tools -> Debug Level -> HTTP_SERVER, Debug port -> Serial and AC_DEBUG in AutoConnect.h header, the debug log can be obtained by serial monitor. If you got a log in your environment I will consider a remedy.

@Marc-Vreeburg
Copy link
Author

Yes from wan i can access index.html without Autoconnect.

I did some logging. I think the problem is with:

Request: /index.html
Arguments:
request handler not found

Part of Logging (changed my external ip address with 'my_external_ip'):

bcn 0
del if1
pm open,type:2 0
add if1
pm close 7
dhcp server start:(ip:192.168.244.1,mask:255.255.255.0,gw:192.168.244.1)
bcn 100
bcn 0
del if1
pm open,type:2 0
mode : sta(60:01:94:73:ea:1a)
[AC] WiFi.config(IP=192.168.1.119, Gateway=192.168.1.1, Subnetmask=255.255.255.0, DNS1=192.168.1.5, DNS2=0.0.0.0) ip:192.168.1.119,mask:255.255.255.0,gw:192.168.1.1
[AC] DHCP client(STOPPED)
sl
[AC] WiFi.begin()
[AC] Connecting.scandone
usl
[AC] established IP:192.168.1.119
[AC] http server started
Webserver started
Started, IP:192.168.1.119
Autoconnect portal started

New client
method: GET url: / search:
[AC] /
[AC] Page[] allocated
headerName: Host
headerValue: my_external_ip:1036
headerName: Connection
headerValue: keep-alive
headerName: Upgrade-Insecure-Requests
headerValue: 1
headerName: User-Agent
headerValue: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36
headerName: Accept
headerValue: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
headerName: Referer
headerValue: http://my_external_ip/sensordata/menu13.html
headerName: Accept-Encoding
headerValue: gzip, deflate
headerName: Accept-Language
headerValue: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
args:
Request: /
Arguments:
New client
method: GET url: /index.html search:
[AC] my_external_ip:1036/index.html
[AC] Page[] allocated
headerName: Host
headerValue: my_external_ip:1036
headerName: Connection
headerValue: keep-alive
headerName: Upgrade-Insecure-Requests
headerValue: 1
headerName: User-Agent
headerValue: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36
headerName: Accept
headerValue: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
headerName: Referer
headerValue: http://my_external_ip/sensordata/menu13.html
headerName: Accept-Encoding
headerValue: gzip, deflate
headerName: Accept-Language
headerValue: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
args:
Request: /index.html
Arguments:
request handler not found
New client
method: GET url: /_ac search:
[AC] my_external_ip:1036/_ac
[AC] Page[/_ac] allocated
headerName: Host
headerValue: 192.168.1.119
headerName: Connection
headerValue: keep-alive
headerName: Upgrade-Insecure-Requests
headerValue: 1
headerName: User-Agent
headerValue: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36
headerName: Accept
headerValue: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
headerName: Referer
headerValue: http://my_external_ip/sensordata/menu13.html
headerName: Accept-Encoding
headerValue: gzip, deflate
headerName: Accept-Language
headerValue: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
args:
Request: /_ac
Arguments:
[AC] 192.168.1.119/_ac
New client
method: GET url: /favicon.ico search:
[AC] 192.168.1.119/favicon.ico
[AC] Page[] allocated
headerName: Host
headerValue: 192.168.1.119
headerName: Connection
headerValue: keep-alive
headerName: User-Agent
headerValue: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36
headerName: Accept
headerValue: image/webp,image/apng,image/,/*;q=0.8
headerName: Referer
headerValue: http://192.168.1.119/_ac
headerName: Accept-Encoding
headerValue: gzip, deflate
headerName: Accept-Language
headerValue: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
args:
Request: /favicon.ico
Arguments:
request handler not found
loadFromSpiffs: /favicon.ico

New client
method: GET url: / search:
[AC] 192.168.1.119/
[AC] Page[] allocated
headerName: Host
headerValue: my_external_ip:1036
headerName: Connection
headerValue: keep-alive
headerName: Upgrade-Insecure-Requests
headerValue: 1
headerName: User-Agent
headerValue: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36
headerName: Accept
headerValue: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
headerName: Referer
headerValue: http://my_external_ip/sensordata/menu13.html
headerName: Accept-Encoding
headerValue: gzip, deflate
headerName: Accept-Language
headerValue: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
args:
Request: /
Arguments:
New client
method: GET url: /index.html search:
[AC] my_external_ip:1036/index.html
[AC] Page[] allocated
headerName: Host
headerValue: my_external_ip:1036
headerName: Connection
headerValue: keep-alive
headerName: Upgrade-Insecure-Requests
headerValue: 1
headerName: User-Agent
headerValue: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36
headerName: Accept
headerValue: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
headerName: Referer
headerValue: http://my_external_ip/sensordata/menu13.html
headerName: Accept-Encoding
headerValue: gzip, deflate
headerName: Accept-Language
headerValue: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
args:
Request: /index.html
Arguments:
request handler not found
New client
method: GET url: /_ac search:
[AC] my_external_ip:1036/_ac
[AC] Page[/_ac] allocated
headerName: Host
headerValue: 192.168.1.119
headerName: Connection
headerValue: keep-alive
headerName: Upgrade-Insecure-Requests
headerValue: 1
headerName: User-Agent
headerValue: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36
headerName: Accept
headerValue: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8
headerName: Referer
headerValue: http://my_external_ip/sensordata/menu13.html
headerName: Accept-Encoding
headerValue: gzip, deflate
headerName: Accept-Language
headerValue: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
args:
Request: /_ac
Arguments:
[AC] 192.168.1.119/_ac
New client
method: GET url: /favicon.ico search:
[AC] 192.168.1.119/favicon.ico
[AC] Page[] allocated
headerName: Host
headerValue: 192.168.1.119
headerName: Connection
headerValue: keep-alive
headerName: User-Agent
headerValue: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/69.0.3497.100 Safari/537.36
headerName: Accept
headerValue: image/webp,image/apng,image/,/*;q=0.8
headerName: Referer
headerValue: http://192.168.1.119/_ac
headerName: Accept-Encoding
headerValue: gzip, deflate
headerName: Accept-Language
headerValue: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7
args:
Request: /favicon.ico
Arguments:
request handler not found
loadFromSpiffs: /favicon.ico

@Hieromon
Copy link
Owner

Hieromon commented Oct 16, 2018

It seems that /index.html was not found by the ESP8266WebServer class, and AutoConnect is redirecting to /ac of the local-IP on ESP8266. "request handler not found" is the behavior in ESP8266WebServer class. The request is my_external_ip:1036/index.html, but since this is not an AutoConnect page, ESP8266WebServer class will try to process it as it is.
Is it accessible except for /index.html with port #1036?

@Hieromon
Copy link
Owner

Hieromon commented Oct 17, 2018

I suggest an additional idea. If you can access except for /index.html from WAN without AutoConnect, it might be able to use a way of giving static IP immediately after WiFi.begin with before WiFi.status check. Implement it in the AutoConnect::begin function. (or entrust it to the user sketch with a callback function.)
However, in this case, the decision to launch a captive portal will be complicated. Because it starts up with the local IP.
I will study what kind of implementation is appropriate. If you have some ideas, tell me, please.

@Marc-Vreeburg
Copy link
Author

Is it accessible except for /index.html with port #1036?

Yes, without captive portal/AutoConnect my_external_ip:1036 redirects to my_internal_ip/index.html because i'm using server.on("/", handleRoot);, where function handleRoot redirects to /index.html

I thought that Config.autoRise = false; would suppress captive portal showing, but it's not.

@Marc-Vreeburg
Copy link
Author

Maybe before checking
if (!_isIP(hostHeader) && (hostHeader != WiFi.localIP().toString()))
we should verify that we're on lan. This means

  • 10.0.0.0 <= ip adddress hostHeader <= 10.255.255.255 or
  • 172.16.0.0 <= ip adddress hostHeader <= 172.31.255.255 or
  • 192.168.0.0 <= ip adddress hostHeader <= 192.168.255.255

If we're on lan, and only then, check wether we're on another domain.
If we're on wan do not perform check wether we're on another domain and don't redirect to AUTOCONNECT_URI.

@Hieromon
Copy link
Owner

I understood your idea. So, please tell me handleRoot() part of the current code that embeds AutoConnect. Is server.on("/", handleRoot) enabled with AutoConnect embedded code currently?
If it is enabled, does it send 302 response from handleRoot itself for redirecting to /index.html?
It's my view that it is a code like this:

ESP8266WebServer server(1036);
AutoConnect portal(server);

void handleRoot() {
  String location = String("http://") + server.client().localIP().toString() + String("/index.html");  // Port considerations omitted.
  server.sendHeader("Location", location, true);
  server.send(302, "text/plain", "");
  server.client().stop();
}

void handleIndex() {
  server.send(200, "text/plain", "hello from /index.html");
}

server.on("/", handleRoot);
server.on("/index.html", handleIndex");
portal.begin();

In such a case, did handleRoot() function adopt server.hostHeader() for the IP address included in Location?

@Marc-Vreeburg
Copy link
Author

In my code i have:

AutoConnect portal(server);
server.on("/", handleRoot);

My old code:

void handleRoot() { server.sendHeader("Cache-Control", "no-cache"); server.sendHeader("Location", "/index.html", true); server.send(302, "text/plane", ""); }

Tried your solution:

void handleRoot() { server.sendHeader("Cache-Control", "no-cache"); String location = String("http://") + server.client().localIP().toString() + String("/index.html"); server.sendHeader("Location", location, true); server.send(302, "text/plain", ""); //server.client().stop(); }

In such a case, did handleRoot() function adopt server.hostHeader() for the IP address included in Location?

Yes, but this is not working.

With portmapping i try to map my_external_ipaddress: cliënt on port 1036 to my_internal_ipaddress: server on port 80.

When i submit my_external_ipaddress: port 1036 into a webbrowser while i'm on wan i notice the external_ipaddress: port 1036 changes into my_internal_ipaddress\index.html. I think it's because i'm on wan and my_internal_ipaddress is a private ipaddress this address can never be reached from wan.

When portmapping is working correctly my index.html page can be reached at my_external_ipaddress:1036\index.html or my_external_ipaddress:1036

@Hieromon
Copy link
Owner

After the connection has been established, AutoConnect::handleClient() simply calls ESP8266WebServer::handleClient(). The sequence of control flow is as follows.

AutoConnect::handleClient() -> ESP8266WebServer::handleClient() -> ESP8266WebServer::_parseRequest() -> FunctionRequestHandler::canHandle() -> FunctionRequestHandler::handle()/index.html

The timing when control is passed to AutoConnect is after _uri mismatch is judged by canHandle function of FunctionRequestHandler class in ESP8266WebServer library.
https://github.com/esp8266/Arduino/blob/42c977bd4d14281e46ff5d01f8f860dd0c195a08/libraries/ESP8266WebServer/src/detail/RequestHandlersImpl.h#L24
Therefore local_ipaddress:80/index.html or external_ipaddress:1036/index.html must be included in the Location header in 302 response according to the situation.

in handleRoot()

"Location: http://local_ip/index.html"

String location = String("http://") + server.client().localIP().toString() + String("/index.html");

"Location: http://external_ip:1036/index.html"

String location = String("http://") + server.hostHeader() + String("/index.html");

Can the sketch make that judgment with handleRoot() which returns 302 response?

@Hieromon
Copy link
Owner

Hieromon commented Oct 27, 2018

I tried the above solution in my WAN environment, and it worked fine. I configured in my ISP router for port forwarding by static IP masquerade as like this:
WAN --- (External IP:1036) > router > (Static Local IP:80) --- ESP8266
Originally I thought that this issue is the necessity of port forwarding by ESP8266WebServer itself, but in fact, it was different. It's only returning correctly 302 response with handleRoot().

void handleRoot() {
  String location = String("http://") + server.hostHeader() + String("/index.html");
  server.sendHeader("Location", location, true);
  server.send(302, "text/plain", "");
  server.client().stop();
}

@Marc-Vreeburg
Copy link
Author

I agree. Looking at my serial port i can see the following lines connecting from wan to lan:

[AC] external_ip:1036/index.html [AC] Page[] allocated

One would expect index.html to be loaded. However this seems not to be the case.
My index,html is large. Could it be that there's some kind of time-out after which Autoconnect defaults to internal_ip/_ac?

@Hieromon
Copy link
Owner

Hieromon commented Nov 5, 2018

It means the value of the host header which the ESP8266WebServer class could not find the request handler before detecting AutoConnect handler. Looking at the log you provided, It has not found "/" as well.
Perhaps running server.on(...) after portal.begin() may cause the same logging.

@Hieromon
Copy link
Owner

My index,html is large. Could it be that there's some kind of time-out after which Autoconnect defaults to internal_ip/_ac?

Perhaps, the cause is insufficient RAM for TCP stack. If your sketch is handing a large String directly to server.send for sending /index.html you need to chunked-encoding it or implement a class that inherits from the Stream interface and gives it to server.streamFile. The pre-release branch for the next version of PageBuilder has that code.

@Marc-Vreeburg
Copy link
Author

Tried an almost empty root page (/index.html). Same problem. So it's not size causing the problem.
Looking at my log it seems that after a local, static ip address has been assigned AutoConnect refuses to allocate my root page (/index.html) from wan.

[AC] WiFi.config(IP=192.168.1.119, Gateway=192.168.1.1, Subnetmask=255.255.255.0, DNS1=192.168.1.5, DNS2=0.0.0.0)
[AC] DHCP client(STOPPED)
[AC] WiFi.begin()
[AC] Connecting.
[AC] established IP:192.168.1.119
[AC] http server started
Started, IP:192.168.1.119
Autoconnect portal started
[AC] /
[AC] Page[] allocated
http://external_ip:1036/index.html
[AC] external_ip:1036/index.html
[AC] Page[] allocated
[AC] external_ip:1036/_ac
[AC] Page[/_ac] allocated
[AC] 192.168.1.119/_ac
[AC] 192.168.1.119/favicon.ico
[AC] Page[] allocated
loadFromSpiffs: /favicon.ico

Starting the esp8266 without a wifi-connection my sequence is as follows:

  1. start esp8266 as AP
  2. set SSID and password via AutoConnect portal through Wifi on phone
  3. SSID and password are set: connecting esp8266 with ip address from dhcp
  4. read staip, staGateway, staNetmask and dns1 from webform and store to eeprom
  5. restart esp8266
  6. esp8266 starts, reads staip, staGateway, staNetmask and dns1 from eeprom and uses static ip address (192.168.1.119)
  7. trying to connect esp8266 from wan (http://external_ip:1036/index.html) isn't possible because AutoConnect returns 192.168.1.119/_ac

@Hieromon
Copy link
Owner

Hieromon commented Dec 8, 2018

AutoConnect returns 192.168.1.119/_ac

Of course, AutoConnect handles the unhandled request of ESP8266Webserver.
You need to redirect to localIP/index.html before the ESP8266WebServer determines that the request handler does not exist. It is not the facility of the AutoConnect library.

@Hieromon
Copy link
Owner

Hieromon commented Mar 2, 2019

Closing due to lack of response.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants