Garbage characters randomly received by HardwareSerial when communicating with SIM800L modem

[toomasz]

Hardware:

Board: TTGO Bluetooth&Battery
Core Installation/update date: 15 dec 2018
IDE name: Visual Micro
Flash Frequency: 80Mhz
PSRAM enabled: no
Upload Speed: 115200
Computer OS: Windows 10

Description:

Hi, I'm designing a library for Simcom gsm modems on ESP32. Since the beginning im fighting with strange serial behavior - which is that HardwareSerial sometimes receives garbage characters from GSM modem, especially when it's transmitting/receiving.

Example:

[GSM] => AT+CSQ
[GSM]  <= +CSQ: 11,0
[GSM]  <= OK
[GSM] --- 
[GSM] => AT+CBC
[GSM]  <= +CBC: 0,91,4132
[GSM]  <= OK
[GSM] --- 
[GSM] => AT+CPIN?
[GSM]  <= +CPIN: READY
[GSM]  <= OK
[GSM]  <= ����������������n��������W�����������������o{_��o���������s���o����������V��߿{����������������~������w��������������������߿w�o���������o��������~~����7�
[GSM]  <= RING
[GSM]  <= +CLIP: "+48XXXXXXXXX",145,"",0,"",0
[GSM] --- 
[GSM] => AT+CREG?
[GSM]  <= +CREG: 0,5
[GSM]  <= OK
[GSM] --- 
battery: 91%, voltage = 4.13 V
Signal quality: 11, operator = 
[GSM] => AT
[GSM]  <= }�����������?���������������}�������������~��������������_������������������������-���v��l��{��w?���׾���������;�����w������O_���=���������w���j��u����?������
[GSM]  <= OK
[GSM] --- 

Some facts:

• Problem occurs mainly when antenna is attached to SIM800L

• I'm using ESP32 16 Pin for RX from gsm, 17 Pin for TX to modem
• I tried many different ESP32 boards and Sim800l modes and problem always accured.
• Problem happens mainly when I connect external antenna to SIM800L, without it most of the time there is no garbage on serial.
• I have proper power source that is capable of giving 2A for SIM800L and also external 1000uF capacitor attached to GND and VCC of module.
• I didn't have this problem when I played with modem on regular Arduino UNO
• Garbage characters occur when modem is transmitting/received packets so while making call, while registering to network, while sending/receiving data
• Issue tracking the problem on my library: Garbage on serial toomasz/SimcomGsmLib#1
• Serial initialization code: https://github.com/toomasz/SimcomGsmLib/blob/master/src/SimcomAtCommandsEsp32.h

Questions:

• Does ESP32 have some kind of filtering for UART that can be activated?
• Why might ESP32 serial be more susectible to interference than Arduino one?

[copercini]

What baud rate are you using to communicate to Sim800l ?

[toomasz]

@copercini 115200, problem occurs on all baud rates though. Even on 9600.

[euquiq]

@toomasz I have a similar configuration ... I am using over 100 esp32 and SIM900L (although I used also SIM800L earlier on) and I have NOT experienced this behavior.

Frankly I would recommend you to check either your electrical connections / design. And even check board quality. Not less important, how are you powering up the whole thing. Which SIM800L board are you using (also, are you aware that SIM800L is an ANCIENT chip?) (Actually SIM900L is also quite old :) )

Also maybe you can post some code ... as in HOW do you begin your serial port ? And last and not least, which version of ESP32 ARDUINO libs are you using ? latest ?

Hope you can fix it !

[copercini]

I was testing this module with TinyGSM library yesterday, the problems were:

• some bugged characters when baud rate is 115200 or higher, then worked fine at 9600 or 4800,
• signal problems when power supply is weak

But nothing like your issue.... maybe you can try test with TinyGSM and see if the problem continues

[euquiq]

@copercini Tiny GSM library did not work for me. If I correctly recall, I posted the issue on their github page, and it turned out the library has problems inside esp32 architecture / tasks and memory handling (Again if I correctly recall, due to using Strings inside the lib).

A quick googleing throws:

vshymanskyy/TinyGSM#133

[copercini]

@euquiq I didn't test it extensively, but was working here with this code https://gist.github.com/copercini/4a47ba2375a96320918c47ef6b7d3bbd

[euquiq]

@copercini In my case, I needed to use tinyGSM inside a task ... I think it was inside there that things went berzerk. It was a long time ago, so I cannot provide details. I really wanted it to work, tho, but I ended up doing everything my "myself" and I got it working.

[toomasz]

I managed to make it more stable by adding TTL lever converter between ESP32 and SIM800L since ESP32 is 2.2 to 3.3V and Sim800l is 3.4 to 4.4 V

Nevertheless sometimes i still get garbage characters, for example when GPRS connection is open and I try to call module number.

@euquiq Im using last ESP32 revision from github, actually I'm developing my library for simcom modules(it aims not to use Arduino strings and be non blocking) and you can see my initialization code here: https://github.com/toomasz/SimcomGsmLib/blob/master/src/SimcomAtCommandsEsp32.h
My library has built in detection for garbage on serial so that why im able to reproduce it I think. Sometimes you might not even notice that with other libraries, you just get command timeout and garbage gets unnoticed.

What pins are you using for TX/RX on ESP32 side?

Yes, i do realize that its ancient and buggy module :) But I didn't have this problem with arduino so that why im reporting issue here.

Anyway I don't think problem is related to any library code, i was also getting strange characters when using basic serial proxy program.

[euquiq]

@toomasz, I am using hardwareserial 2 (GPIO17 and GPIO16):

#include "HardwareSerial.h" //CONNECTING INTO GSM MODEM

and on setup():

Serial2.begin(115200, SERIAL_8N1); //GSM MODEM CONNECTIVITY

[euquiq]

@toomasz I got a possible clue:

Do you have a SIM installed on that SIM800L ? Could it be that your telco provider is sending you PROVISIONING PACKETS ?? It did happen down here to me with ONE telco: They detect that you are not contacting the APN for -say- MMS pictures and they PUSH into your SIM800L a PACKET "asking" for autoconfiguration.

I recall that SUDDENLY, I had "GARBAGE" along with my stuff, and it was the incoming provisioning GSM PACKET or whatever it was called.

You will need to google it a bit, but it was kind of unavoidable, sudden, and binary. It would help if you could create a better "printout" of incoming data, not only showing ascii chars, but also de binary values, so you can see if the garbage has the correct "pattern" (and even google it)

[toomasz]

@euquiq I don't think SIM800L would send provisioning packets as random garbage characters to SIM800L. These random characters are usually interrupting normal AT commands response so my library cannot parse it. Also, i have garbage especially when I attach external antenna to SIM800l so i'm almost sure that this problem is related to ESP32 uart sensitivity. Anyway, for now problem is almost gone when SIM800L is attached through logic level converter.

[me-no-dev]

Is it possible that a buffer termination is not added in some case and some garbage (super secret data) is printed out until 0 is reached? I did not look at the library code...

[euquiq]

@toomasz I will not insist, I will only say the following: the provisioning packets should be received by your SIM800L after being powered on, within a minute or two. and be periodic.

They will have priority, and in my case it was EXACTLY like that: They would go over AT commands or anything. My SIM800L (and after that same thing with the SIM900L just inserted those bytes whenever they arrived, into the serial stream. In all cases, if you would decode in bytes, that "garbage" you could eventually detect if there is a pattern, and even google it, or just dismiss it as garbage.

[lrmoreno007]

I have the same garbage with the A6 (and with SIM900) but I use SoftwareSerial and what I do is wait for response and retry 3 times:

Global:

#include <SoftwareSerial.h>   // https://github.com/plerup/espsoftwareserial
boolean response_SIM900 = false; 
char PIN[] = "0787";
SoftwareSerial SIM900(D5, D6, false, 1024); // Pin de RXD, Pin de TXD

Wait response function:

void waitFor(const char *resp1, const char *resp2) {
  unsigned long entry = millis();
  String reply = "";
  int timeout = 2000;
  do {
    reply += SIM900.readString();
    yield();
  }
  while (((reply.indexOf(resp1) + reply.indexOf(resp2)) == -2) && ((millis() - entry) < timeout));

  if ((millis() - entry) >= timeout) {
    Serial_Telnet_ln("ERROR: Tiempo de espera superado");
    response_SIM900 = false;
  }
  else if (reply.indexOf(resp1) + reply.indexOf(resp2) > -2) {
    if (reply.indexOf(resp1) != -1) {
      Serial_Telnet_ln("OK");
      response_SIM900 = true;
    }
    if (reply.indexOf(resp2) != -1) {
      Serial_Telnet("ERROR: ");
      Serial_Telnet_ln(reply.c_str());
      response_SIM900 = false;
    }
  }
  else {
    Serial_Telnet("ERROR: Desconocido");
    response_SIM900 = false;
  }
}

Setup:

void setup() {
  Serial.begin(115200);
  SIM900.begin(115200);
}

In loop (or other function called from loop):

    response_SIM900 = false;
    int retry = 3;
    while (!response_SIM900 && retry > 0) {
      SIM900.print("AT+CPIN="); //Introducimos el PIN (variable PIN)
      SIM900.println(PIN);
      SIM900.flush();
      waitFor("OK", "+CME");
      delay(3000);
      retry--;
    }

Excuse me: Serial_Telnet_ln and Serial_Telnet are my functions that send message to Telnet and Serial at same time.