Unable to use HTTP client inside callback (ESP8266 only, no problem with ESP32)

[CosmicMac]

Hello,

With ESP8266, using HTTP client from inside a ESPAsyncWebServer callback always returns a "connection refused" error.
Same function called from anywhere outside the callback works as expected.
No problem at all with ESP32, both calls return the expected content.

My test code for ESP8266:

#include <Arduino.h>
#include <ESP8266HTTPClient.h>
#include <ESP8266mDNS.h>
#include <ESPAsyncWebServer.h>

const char *ssid = "********";
const char *password = "********";
const char *hostname = "esptest";
const char *url = "http://httpbin.org/ip";

HTTPClient http;
AsyncWebServer server(80);

void cltTest(const char *context) {

    Serial.printf("\n\n>>>> cltTest called from %s\nHeap: %d\n", context, ESP.getFreeHeap());

    http.begin(url);
    if (http.GET() > 0) {
        http.writeToStream(&Serial);
    }
    http.end();
}

void setup() {

  Serial.begin(115200);
  Serial.setDebugOutput(true);

  Serial.printf("Attempting to connect to SSID: %s", ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    Serial.print(".");
    delay(1000);
  }
  Serial.println("\nConnected!");

  MDNS.begin(hostname);
  MDNS.addService("http", "tcp", 80);

  server.on("/", HTTP_GET, [](AsyncWebServerRequest *request) {
    cltTest("Server callback");
    request->send(200, "text/plain", "Done");
  });
  server.begin();

  cltTest("Setup");
}

void loop() {
  MDNS.update();
}

And some traces, after pointing my browser at http://esptest.local :

SDK:2.2.1(cfd48f3)/Core:2.4.1/lwIP:1.4.0rc2

[...]

>>>> cltTest called from Setup
Heap: 38776
[HTTP-Client][begin] url: http://httpbin.org/ip
[HTTP-Client][begin] host: httpbin.org port: 80 url: /ip
[HTTP-Client] connected to httpbin.org:80
[HTTP-Client] sending request header
-----
GET /ip HTTP/1.1
Host: httpbin.org
User-Agent: ESP8266HTTPClient
Connection: close
Accept-Encoding: identity;q=1,chunked;q=0.1,*;q=0

-----
[HTTP-Client][handleHeaderResponse] RX: 'HTTP/1.1 200 OK'
[HTTP-Client][handleHeaderResponse] RX: 'Connection: close'
[HTTP-Client][handleHeaderResponse] RX: 'Server: gunicorn/19.8.1'
[HTTP-Client][handleHeaderResponse] RX: 'Date: Sun, 13 May 2018 15:26:56 GMT'
[HTTP-Client][handleHeaderResponse] RX: 'Content-Type: application/json'
[HTTP-Client][handleHeaderResponse] RX: 'Content-Length: 26'
[HTTP-Client][handleHeaderResponse] RX: 'Access-Control-Allow-Origin: *'
[HTTP-Client][handleHeaderResponse] RX: 'Access-Control-Allow-Credentials: true'
[HTTP-Client][handleHeaderResponse] RX: 'Via: 1.1 vegur'
[HTTP-Client][handleHeaderResponse] RX: ''
[HTTP-Client][handleHeaderResponse] code: 200
[HTTP-Client][handleHeaderResponse] size: 26
{"origin":"123.45.67.89"}
[HTTP-Client][writeToStreamDataBlock] connection closed or file end (written: 26).
[HTTP-Client][end] tcp stop
[HTTP-Client][end] tcp is closed
pm open,type:2 0


>>>> cltTest called from Server callback
Heap: 35448
[HTTP-Client][begin] url: http://httpbin.org/ip
[HTTP-Client][begin] host: httpbin.org port: 80 url: /ip
[HTTP-Client] failed connect to httpbin.org:80
[HTTP-Client][returnError] error(-1): connection refused
[HTTP-Client][end] tcp is closed

Did I miss something?

Cheers

[CosmicMac]

Quoting our chat session on Gitter:

Me No Dev
@CosmicMac I am not sure that hat can really be done (in a simple way)
it work on ESP32 because ESP32 is running FreeRTOS and network is handled inside a thread
that allows you to delay the response a bit and run that client inside
though... it's not a good approach either because you are in effect halting all other connections
I am thinking on making AsyncTCP multithreaded on ESP32 to overcome this shortcoming, but obviosly also not something that can be done on esp8266
the way I see this working in eneral is to detach the client from the server and manage it's data and callbacks separately to be entangled with the http client

Cosmic Mac
Thanks a lot @me-no-dev, that's crystal clear.
My plan B is to use the callback only to set a flag (to trigger the HTTP Client) and send a 204 response.
Then I'll send the real payload (or any error, depending on the client response) to the browser via a server-sent event.