HTTPClient.h not compatible with TinyGsm?

[bgondell]

Hardware:

Board: ESP32 Dev Module v1
Core Installation version: 1.0.4
IDE name: Arduino IDE
Flash Frequency: 80Mhz
PSRAM enabled: no
Upload Speed: 921600
Computer OS: Windows 10

Description:

Summary: Code works fine on WiFi, but on GPRS it crash. Given examples works fine also.
Modem: SIM800L
TinyGSM version: 0.10.5 (latest on Library Manager)

Note: First versions of code used ArduinoHTTPClient.h, which actually worked on Wifi/GPRS, but entire code was very different (not only the part on using Wifi or GPRS). Then I've changed to HTTPClient.h because code had others problems related to data, not communication itself.

Code: < related code because full code has 900 lines >

...
#define TINY_GSM_MODEM_SIM800
#define TINY_GSM_RX_BUFFER 1024
#define SerialAT Serial2
#define SerialMon Serial
#include <HTTPClient.h>

#define USE_GPRS true
#define USE_WIFI false

#ifdef USE_GPRS
  #include <TinyGsmClient.h>
#else
  #include <WiFi.h>
#endif


#ifdef USE_GPRS
    #ifdef DUMP_AT_COMMANDS
      #include <StreamDebugger.h>
      StreamDebugger debugger(SerialAT, SerialMon);
      TinyGsm modem(debugger);
    #else
      TinyGsm modem(SerialAT);
    #endif
  TinyGsmClient client(modem);
#else
  WiFiClient wifi;
#endif

const char apn[]  = "virtueyes.vivo.com.br";
const char gprsUser[] = "vivo";
const char gprsPass[] = "vivo";
const char* url = "http://XXX.XXX.XXX.XXX/ws/saveData";


void setup() {
...

  if ( USE_GPRS ) {
    SerialAT.begin(38400); // Set GSM module baud rate.
    ConnectGPRS(1); // 0 = modem init. 1 = modem restart. Pode levar algunms minutos.
  };

  if ( USE_WIFI ) {
    Serial.println("Connecting to ");
    Serial.println(ssid);
    WiFi.begin(ssid, password);
    
    while (WiFi.status() != WL_CONNECTED) {
      delay(1000);
      Serial.print(".");
    }
    Serial.println("");
    Serial.println("WiFi connected..!");
    Serial.print("Got IP: ");  Serial.println(WiFi.localIP());
    Serial.print("Gateway IP: "); Serial.println(WiFi.gatewayIP());
  }
 ...
}

// Conecta a rede GSM é activa comunicaçao GPRS
void ConnectGPRS(int mode) { 


  bool conectado = false;
  int i = 0;
  while ( !conectado && (i < 5) ){
    conectado = conectado || modem.isGprsConnected();
    i++;
    delay(100);
  }

  //Condiçao para reiniciar GPRS no caso do reinicio da ESP.
  if (millis() < 2000) {
    conectado = false;
    mode = 1;
  }
  
  if (!conectado) { 
    Serial.println("Initializando modem...");
    if (mode == 0) modem.init();
    else if (mode == 1 ) modem.restart();
    else Serial.println("ConnectGPRS Mode incorreto");
    
    Serial.print("Conectando ao APN: ");
    Serial.println(apn);

    while (!conectado) {
      conectado = modem.gprsConnect(apn, gprsUser, gprsPass);
      if (!conectado) Serial.println(" Falhou. Repetindo.");
      delay(10000);
    }
    if (conectado) {
      Serial.println(" Conectado!");
    }
     if (modem.isNetworkConnected()) {
      Serial.println("Conectado a rede GPRS.");
      lastcheck=millis();
    }  
  }
}

void loop() {
.....
 if ( USE_GPRS ) {
    //Verificar conexçao GPRS
      ConnectGPRS(0);
    }
    else ConnectWiFi();

    if (Dados!= "") { //Existem dados para enviar / There is data to send
      Serial.print("Enviando ");
      Serial.print(cant);
      Serial.print(" dados: ");
      Serial.println(Dados);
      
      Serial.println("Executando HTTP POST... ");

      HTTPClient http;
      Serial.println("http instance created");
      http.begin(url);
      Serial.println("http instance initiated at specified url");
      http.setTimeout(120000); // 120 seg
      Serial.println("http Timeout Setted");
      http.addHeader("Content-Type", "application/json");
      Serial.println("http Headers added");  
      int status = http.POST(Dados);
....
}

The last line (line 234) is when ESP32 Crash.

Expected result - This is what happens if use WIFI : Working Fine

Iniciando
Prgm value: 1
CICLOS DA PORTA: 0
UUID: 199
Connecting to
<-WiFi->
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 - WIFI_READY
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 2 - STA_START
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 4 - STA_CONNECTED
[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 7 - STA_GOT_IP
[D][WiFiGeneric.cpp:381] _eventCallback(): STA IP: 192.168.1.159, MASK: 255.255.255.0, GW: 192.168.1.254
.
WiFi connected..!
Got IP: 192.168.1.159
Gateway IP: 192.168.1.254
Inicializando dados da Porta
Dados Novos!
Enviando 1 dados: [{"ID":"2500.1","t":1595346122,"uuid":199,"c":0,"I1":0,"I2":1,"I3":1,"I4":1,"I5":0,"I6":0,"I7":0,"I8":0,"T1":0,"T2":0,"T3":0,"C1":0,"C2":0,"C3":0,"S1":8.5,"S2":10.2,"S3":6.9,"S4":6.2,"S5":0}]
Executando HTTP POST...
http instance created
[V][HTTPClient.cpp:235] beginInternal(): url: http://XXX.XXX.XXX.XXX/ws/saveData
[D][HTTPClient.cpp:276] beginInternal(): host: XXX.XXX.XXX.XXX port: 80 url: /ws/saveData
http Timeout Setted
http Headers added
[D][HTTPClient.cpp:1025] connect(): connected to XXX.XXX.XXX.XXX:80
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'HTTP/1.1 200 OK'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Date: Tue, 21 Jul 2020 18:42:03 GMT'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Server: Apache/2.4.29 (Ubuntu)'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Set-Cookie: super_mydev_session=dfasdfavdcertregsdfadcds; path=/; HttpOnly'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Expires: Thu, 19 Nov 1981 08:52:00 GMT'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Cache-Control: no-store, no-cache, must-revalidate'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Pragma: no-cache'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Content-Length: 0'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Keep-Alive: timeout=5, max=100'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Connection: Keep-Alive'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Content-Type: application/json; charset=utf-8'
[V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: ''
[D][HTTPClient.cpp:1158] handleHeaderResponse(): code: 200
[D][HTTPClient.cpp:368] disconnect(): tcp keep open for reuse

Actual result : This is what happens if use GPRS: (xQueueGenericReceive)- assert failed!

Iniciando
Prgm value: 1
CICLOS DA PORTA: 0
UUID: 199
Initializando modem...
Conectando ao APN: virtueyes.vivo.com.br
Conectado!
Conectado a rede GPRS.
Setting RTC time
Time set on internal RTC: 1595345958
Inicializando dados da Porta
Dados Novos!
Enviando 1 dados: [{"ID":"2500.1","t":1595345823,"uuid":199,"c":0,"I1":0,"I2":1,"I3":1,"I4":1,"I5":0,"I6":0,"I7":0,"I8":0,"T1":0,"T2":0,"T3":0,"C1":0,"C2":0,"C3":0,"S1":19.7,"S2":7.5,"S3":12.5,"S4":16.8,"S5":0}]
Executando HTTP POST...
http instance created
[V][HTTPClient.cpp:235] beginInternal(): url: http://XXX.XXX.XXX.XXX/ws/saveData
[D][HTTPClient.cpp:276] beginInternal(): host: XXX.XXX.XXX.XXX port: 80 url: /ws/saveData
http Headers added
/home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c:1442 (xQueueGenericReceive)- assert failed!
abort() was called at PC 0x4008890d on core 1

Backtrace: 0x4008c50c:0x3ffb3c50 0x4008c73d:0x3ffb3c70 0x4008890d:0x3ffb3c90 0x401217a2:0x3ffb3cd0 0x401219a2:0x3ffb3cf0 0x40113e04:0x3ffb3d10 0x40113e6d:0x3ffb3d30 0x4011100a:0x3ffb3d50 0x4010fe94:0x3ffb3d70 0x40113688:0x3ffb3db0 0x400d759e:0x3ffb3dd0 0x400d6e7a:0x3ffb3e30 0x400d683e:0x3ffb3e60 0x400d6921:0x3ffb3e90 0x400d69f3:0x3ffb3ed0 0x400d6a13:0x3ffb3ef0 0x400d491d:0x3ffb3f10 0x40088c21:0x3ffb3fe0

Rebooting...

Debug

Exception Decoder Tool:

Decoding stack results
0x4008c50c: invoke_abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c line 155
0x4008c73d: abort at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/esp32/panic.c line 170
0x4008890d: xQueueGenericReceive at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/queue.c line 1442
0x401217a2: sys_mutex_lock at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/port/esp32/freertos/sys_arch.c line 78
0x401219a2: sys_arch_protect at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/port/esp32/freertos/sys_arch.c line 469
0x40113e04: do_memp_malloc_pool at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/memp.c line 302
0x40113e6d: memp_malloc at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/core/memp.c line 398
0x4011100a: netconn_alloc at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/api_msg.c line 742
0x4010fe94: netconn_new_with_proto_and_callback at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/api_lib.c line 133
0x40113688: lwip_socket at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/lwip/lwip/src/api/sockets.c line 1587
0x400d759e: WiFiClient::connect(IPAddress, unsigned short, int) at C:\Users\evelis\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4/tools/sdk/include/lwip/lwip/sockets.h line 593
0x400d6e7a: WiFiClient::connect(char const*, unsigned short, int) at C:\Users\evelis\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\WiFi\src\WiFiClient.cpp line 281
0x400d683e: HTTPClient::connect() at C:\Users\evelis\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\HTTPClient\src\HTTPClient.cpp line 1017
0x400d6921: HTTPClient::sendRequest(char const*, unsigned char*, unsigned int) at C:\Users\evelis\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\HTTPClient\src\HTTPClient.cpp line 552
0x400d69f3: HTTPClient::POST(unsigned char*, unsigned int) at C:\Users\evelis\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\HTTPClient\src\HTTPClient.cpp line 493
0x400d6a13: HTTPClient::POST(String) at C:\Users\evelis\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\HTTPClient\src\HTTPClient.cpp line 498
0x400d491d: TaskCommunication(void*) at C:\Users\evelis\Documents\Arduino\PortaV2.3/PortaV2.3.ino line 234
0x40088c21: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c line 143

[Jeroen88]

HTTPClient will only work with a WiFiClient. You might try my PR #3848, that handles the HTTP protocol by directly manipulating the Client passed into the constructor, however I never tried it with TinyGsm.

[bgondell]

Thanks @Jeroen88 , I'll give it a try!

[bgondell]

Hi @Jeroen88 , at first try isn't working. I'll debug a little more and share the results.

It's just a post so if someone know another solution to be share, will be welcome!

[Jeroen88]

Sim800 has a http client also using AT commands. If I recall correctly tinygsm itself may have a http client as well (that uses these AT commands). If I recall correctly Sim800 only supports http,no https, so the server you're connecting to should allow unsecured http connections

[bgondell]

@Jeroen88 Exactly, I'm using an HTTP server to manage POST.

First I made a code to test HTTP GET with your library. Success Rate is bad, changing setTimeout (up to 100000) appears not to affect on nothing, error -1 comes just a couple of seconds later.

23:23:01.336 -> TRYING HTTP GET FIRTS TO CHECK CONNECTION
23:23:03.529 -> Request failed with error code -1
23:23:03.562 -> Closing connection
23:23:03.562 -> Success: 14. Intents: 81. Success Rate: 17%

Now... HTTP POST has a more acceptable success rate around 50%. (Also with setTimeout at 100000, and getting -1 error a couple of seconds later).

00:07:01.598 -> NOW TESTING TO MAKE A POST
00:07:03.898 -> Request failed with error code -1
00:07:03.936 -> Closing connection
00:07:03.936 -> Success: 95. Intents: 194. Success Rate: 48%

Maybe a fuction to retry connection? or maybe wait a bit more?

Thanks!!

[Jeroen88]

I have no access to my computer now, so I can not check anything before next week.
Could you add debug statements in the library itself to drill down the exact point of failure?

I understand correctly that sometimes it works? That means that in principle it is possible to make http requests over a Sim800 connection.

[stale]

[STALE_SET] This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions.

[stale]

[STALE_DEL] This stale issue has been automatically closed. Thank you for your contributions.

[roysG]

How do you added external library?

[bgondell]

I used <ArduinoHttpClient.h> instead <HTTPClient.h>

[roysG]

Do you know how to add external library?

[OptimumAssociatesUK]

Hi bgondell! Did you resolve the issue and got success to send data via GSM?