Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESP32 LilyGo GSM module OTA possibilities #132

Open
krupis opened this issue Apr 8, 2021 · 66 comments
Open

ESP32 LilyGo GSM module OTA possibilities #132

krupis opened this issue Apr 8, 2021 · 66 comments

Comments

@krupis
Copy link

krupis commented Apr 8, 2021

Hello. I am using this module to learn how to use GSM modules. I have been experimenting with the example codes and I have learnt the basic working principles. The big project that I am going to be working on next will involve the GSM and the GPS module.

I am very interested in how to perform a OTA over the GSM since the there will be very limited WIFI availability where my device will be used. At some point, I may want to update the firmware, fix bugs, so I the probably the easiest way to be to use the GSM and do the OTA over the GSM.

I have noticed the update_firmware under the example codes but there is nothing there. Perhaps anyone has made any progress and could share their knowledge and ideas with me? I appreciate any help! Thanks in advance

@thomsenle
Copy link

see tinygsm examples on how to open a tcp connection and use it, and see esp32 https ota examples
https://github.com/espressif/arduino-esp32/tree/master/libraries/Update/examples/HTTPS_OTA_Update

@thomsenle
Copy link

see also here vshymanskyy/TinyGSM#152

@gunter72
Copy link

gunter72 commented May 9, 2021

Hi, I have the same problem.
How did you solve?

@emigbur
Copy link

emigbur commented May 15, 2021

I did a skectch and work fine, is any are interested I can share ti, but I dont know how to make the server to store the bin file.
I did the sketch using the TTGO CALL SIM800L github GSM tiny OTA example.
If someone has and server example I will apreciate it, my idea is to store the bin file on my web server.

@krupis
Copy link
Author

krupis commented May 15, 2021

Hello. Here is the code to perform OTA over the GSM:


#include <Update.h>
#define TINY_GSM_MODEM_SIM800
// Increase RX buffer
#define TINY_GSM_RX_BUFFER 1030

const char apn[] = "omnitel";
const char user[] = "omni";
const char pass[] = "omni";



#define MODEM_RST             5
#define MODEM_PWRKEY          4
#define MODEM_POWER_ON       23
#define MODEM_TX             27
#define MODEM_RX             26
#define LED_GPIO             13
#define LED_ON               HIGH
#define LED_OFF              LOW


#define SerialAT  Serial1

#include <TinyGsmClient.h>
#include <CRC32.h>

#include "FS.h"
#include "SPIFFS.h"

#ifdef DUMP_AT_COMMANDS
#include <StreamDebugger.h>
StreamDebugger debugger(SerialAT, Serial);
TinyGsm modem(debugger);
#else
TinyGsm modem(SerialAT);
#endif


TinyGsmClient client(modem);

const char server[] = "myesptestserver.ddns.net";
const int port = 80;
const char resource[] = "/esp.bin"; //here de bin file

uint32_t knownCRC32 = 0x6f50d767;
uint32_t knownFileSize = 1024; // In case server does not send it

void setup()
{
    SerialAT.begin(115200, SERIAL_8N1, MODEM_RX, MODEM_TX);
    // Set console baud rate
    Serial.begin(115200);

    setupModem();
    delay(10);

    if (!SPIFFS.begin(true))
    {
        Serial.println("SPIFFS Mount Failed");
        return;
    }
    SPIFFS.format();
    listDir(SPIFFS, "/", 0);

    // Set GSM module baud rate
    
    delay(3000);

    // Restart takes quite some time
    // To skip it, call init() instead of restart()
    Serial.println("Initializing modem...");
    modem.restart();

    String modemInfo = modem.getModemInfo();
    Serial.print("Modem: ");
    Serial.println(modemInfo);

    // Unlock your SIM card with a PIN
    //modem.simUnlock("1234");
}



void loop()
{
    Serial.print("Waiting for network...");
    if (!modem.waitForNetwork())
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(apn);
    if (!modem.gprsConnect(apn, user, pass))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(server);

    // if you get a connection, report back via serial:
    if (!client.connect(server, port))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");
    
    // Make a HTTP request:
    client.print(String("GET ") + resource + " HTTP/1.0\r\n");
    client.print(String("Host: ") + server + "\r\n");
    client.print("Connection: close\r\n\r\n");

    long timeout = millis();
    while (client.available() == 0)
    {
        if (millis() - timeout > 5000L)
        {
            Serial.println(">>> Client Timeout !");
            client.stop();
            delay(10000L);
            return;
        }
    }

    Serial.println("Reading header");
    uint32_t contentLength = knownFileSize;

    File file = SPIFFS.open("/update.bin", FILE_APPEND);


    while (client.available())
    {
        String line = client.readStringUntil('\n');
        line.trim();
        //Serial.println(line);    // Uncomment this to show response header
        line.toLowerCase();
        if (line.startsWith("content-length:"))
        {
            contentLength = line.substring(line.lastIndexOf(':') + 1).toInt();
        }
        else if (line.length() == 0)
        {
            break;
        }
    }


    timeout = millis();
    uint32_t readLength = 0;
    CRC32 crc;

    unsigned long timeElapsed = millis();
    printPercent(readLength, contentLength);
    

    while (readLength < contentLength && client.connected() && millis() - timeout < 10000L)
    {
        int i = 0;
        while (client.available())
        {
                // read file data to spiffs
            if (!file.print(char(client.read())))
            {
                Serial.println("Appending file");
            }
            //Serial.print((char)c);       // Uncomment this to show data
            //crc.update(c);
            readLength++;

            if (readLength % (contentLength / 13) == 0)
            {
                printPercent(readLength, contentLength);
            }
            timeout = millis();
        }
    }

    file.close();

    printPercent(readLength, contentLength);
    timeElapsed = millis() - timeElapsed;
    Serial.println();

    client.stop();
    Serial.println("stop client");

    modem.gprsDisconnect();
    Serial.println("gprs disconnect");
    Serial.println();

    float duration = float(timeElapsed) / 1000;
  /*
    Serial.print("Tamaño de Archivo: ");
    Serial.println(contentLength);
    Serial.print("Leido:  ");
    Serial.println(readLength);
    Serial.print("Calculado. CRC32:    0x");
    Serial.println(crc.finalize(), HEX);
    Serial.print("Conocido CRC32:    0x");
    Serial.println(knownCRC32, HEX);
    Serial.print("Bajado en:       ");
    Serial.print(duration);
    Serial.println("s");

    Serial.println("Se genera una espera de 3 segundos");
    for (int i = 0; i < 3; i++)
    {
        Serial.print(String(i) + "...");
        delay(1000);
    }
  */
    //readFile(SPIFFS, "/update.bin");

    updateFromFS();

    // Do nothing forevermore
    while (true)
    {
        delay(1000);
    }
}







void appendFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Appending to file: %s\n", path);

    File file = fs.open(path, FILE_APPEND);
    if (!file)
    {
        Serial.println("Failed to open file for appending");
        return;
    }
    if (file.print(message))
    {
        Serial.println("APOK");
    }
    else
    {
        Serial.println("APX");
    }
}

void readFile(fs::FS &fs, const char *path)
{
    Serial.printf("Reading file: %s\n", path);

    File file = fs.open(path);
    if (!file || file.isDirectory())
    {
        Serial.println("Failed to open file for reading");
        return;
    }

    Serial.print("Read from file: ");
    while (file.available())
    {
        Serial.write(file.read());
        delayMicroseconds(100);
    }
}

void writeFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Writing file: %s\n", path);

    File file = fs.open(path, FILE_WRITE);
    if (!file)
    {
        Serial.println("Failed to open file for writing");
        return;
    }
    if (file.print(message))
    {
        Serial.println("File written");
    }
    else
    {
        Serial.println("Write failed");
    }
}

void listDir(fs::FS &fs, const char *dirname, uint8_t levels)
{
    Serial.printf("Listing directory: %s\n", dirname);

    File root = fs.open(dirname);
    if (!root)
    {
        Serial.println("Failed to open directory");
        return;
    }
    if (!root.isDirectory())
    {
        Serial.println("Not a directory");
        return;
    }

    File file = root.openNextFile();
    while (file)
    {
        if (file.isDirectory())
        {
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if (levels)
            {
                listDir(fs, file.name(), levels - 1);
            }
        }
        else
        {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("  SIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void deleteFile(fs::FS &fs, const char *path)
{
    Serial.printf("Deleting file: %s\n", path);
    if (fs.remove(path))
    {
        Serial.println("File deleted");
    }
    else
    {
        Serial.println("Delete failed");
    }
}

void updateFromFS()
{
    File updateBin = SPIFFS.open("/update.bin");
    if (updateBin)
    {
        if (updateBin.isDirectory())
        {
            Serial.println("Directory error");
            updateBin.close();
            return;
        }

        size_t updateSize = updateBin.size();

        if (updateSize > 0)
        {
            Serial.println("Starting update");
            performUpdate(updateBin, updateSize);
        }
        else
        {
            Serial.println("Error, archivo vacío");
        }

        updateBin.close();

        // whe finished remove the binary from sd card to indicate end of the process
        //fs.remove("/update.bin");
    }
    else
    {
        Serial.println("no such binary");
    }
}

void performUpdate(Stream &updateSource, size_t updateSize)
{
    if (Update.begin(updateSize))
    {
        size_t written = Update.writeStream(updateSource);
        if (written == updateSize)
        {
            Serial.println("Writes : " + String(written) + " successfully");
        }
        else
        {
            Serial.println("Written only : " + String(written) + "/" + String(updateSize) + ". Retry?");
        }
        if (Update.end())
        {
            Serial.println("OTA finished!");
            if (Update.isFinished())
            {
                Serial.println("Restart ESP device!");
                ESP.restart();
            }
            else
            {
                Serial.println("OTA not fiished");
            }
        }
        else
        {
            Serial.println("Error occured #: " + String(Update.getError()));
        }
    }
    else
    {
        Serial.println("Cannot beggin update");
    }
}





void printPercent(uint32_t readLength, uint32_t contentLength)
{
    // If we know the total length
    if (contentLength != -1)
    {
        Serial.print("\r ");
        Serial.print((100.0 * readLength) / contentLength);
        Serial.print('%');
    }
    else
    {
        Serial.println(readLength);
    }
}



void setupModem()
{
#ifdef MODEM_RST
    // Keep reset high
    pinMode(MODEM_RST, OUTPUT);
    digitalWrite(MODEM_RST, HIGH);
#endif

    pinMode(MODEM_PWRKEY, OUTPUT);
    pinMode(MODEM_POWER_ON, OUTPUT);

    // Turn on the Modem power first
    digitalWrite(MODEM_POWER_ON, HIGH);

    // Pull down PWRKEY for more than 1 second according to manual requirements
    digitalWrite(MODEM_PWRKEY, HIGH);
    delay(100);
    digitalWrite(MODEM_PWRKEY, LOW);
    delay(1000);
    digitalWrite(MODEM_PWRKEY, HIGH);

    // Initialize the indicator as an output
    pinMode(LED_GPIO, OUTPUT);
    digitalWrite(LED_GPIO, LED_OFF);
}

Make sure to change accordingle the following lines of code:

const char apn[] = "omnitel";
const char user[] = "omni";
const char pass[] = "omni";

and

const char server[] = "myesptestserver.ddns.net";
const int port = 80;
const char resource[] = "/esp.bin"; //here de bin file

For storing the bin files, you have multiple choises.

  1. Use Raspberry pi as a server ( I have created a simple apache server where I uploaded my binary file, and then I can access that server through the following sketch.

However, for testing it maybe be easier to setup Xampp on windows machine just to see whether you can get it to work.
2. Create an apache or simmilar server on your windows machine ( probably the simplest method would be to use Xampp as it does everything for you. In the xampp folder you will have htdocs folder where you can put your binary. and you can access that binary by using my sketch or your own sketch

const char server[] = "myesptestserver.ddns.net"; // ip of server or web address
const int port = 80;
const char resource[] = "/esp.bin"; // put your binary file name

Let me know if you have any more questions

@eueduardo3
Copy link

Hello. Here is the code to perform OTA over the GSM:


#include <Update.h>
#define TINY_GSM_MODEM_SIM800
// Increase RX buffer
#define TINY_GSM_RX_BUFFER 1030

const char apn[] = "omnitel";
const char user[] = "omni";
const char pass[] = "omni";



#define MODEM_RST             5
#define MODEM_PWRKEY          4
#define MODEM_POWER_ON       23
#define MODEM_TX             27
#define MODEM_RX             26
#define LED_GPIO             13
#define LED_ON               HIGH
#define LED_OFF              LOW


#define SerialAT  Serial1

#include <TinyGsmClient.h>
#include <CRC32.h>

#include "FS.h"
#include "SPIFFS.h"

#ifdef DUMP_AT_COMMANDS
#include <StreamDebugger.h>
StreamDebugger debugger(SerialAT, Serial);
TinyGsm modem(debugger);
#else
TinyGsm modem(SerialAT);
#endif


TinyGsmClient client(modem);

const char server[] = "myesptestserver.ddns.net";
const int port = 80;
const char resource[] = "/esp.bin"; //here de bin file

uint32_t knownCRC32 = 0x6f50d767;
uint32_t knownFileSize = 1024; // In case server does not send it

void setup()
{
    SerialAT.begin(115200, SERIAL_8N1, MODEM_RX, MODEM_TX);
    // Set console baud rate
    Serial.begin(115200);

    setupModem();
    delay(10);

    if (!SPIFFS.begin(true))
    {
        Serial.println("SPIFFS Mount Failed");
        return;
    }
    SPIFFS.format();
    listDir(SPIFFS, "/", 0);

    // Set GSM module baud rate
    
    delay(3000);

    // Restart takes quite some time
    // To skip it, call init() instead of restart()
    Serial.println("Initializing modem...");
    modem.restart();

    String modemInfo = modem.getModemInfo();
    Serial.print("Modem: ");
    Serial.println(modemInfo);

    // Unlock your SIM card with a PIN
    //modem.simUnlock("1234");
}



void loop()
{
    Serial.print("Waiting for network...");
    if (!modem.waitForNetwork())
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(apn);
    if (!modem.gprsConnect(apn, user, pass))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(server);

    // if you get a connection, report back via serial:
    if (!client.connect(server, port))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");
    
    // Make a HTTP request:
    client.print(String("GET ") + resource + " HTTP/1.0\r\n");
    client.print(String("Host: ") + server + "\r\n");
    client.print("Connection: close\r\n\r\n");

    long timeout = millis();
    while (client.available() == 0)
    {
        if (millis() - timeout > 5000L)
        {
            Serial.println(">>> Client Timeout !");
            client.stop();
            delay(10000L);
            return;
        }
    }

    Serial.println("Reading header");
    uint32_t contentLength = knownFileSize;

    File file = SPIFFS.open("/update.bin", FILE_APPEND);


    while (client.available())
    {
        String line = client.readStringUntil('\n');
        line.trim();
        //Serial.println(line);    // Uncomment this to show response header
        line.toLowerCase();
        if (line.startsWith("content-length:"))
        {
            contentLength = line.substring(line.lastIndexOf(':') + 1).toInt();
        }
        else if (line.length() == 0)
        {
            break;
        }
    }


    timeout = millis();
    uint32_t readLength = 0;
    CRC32 crc;

    unsigned long timeElapsed = millis();
    printPercent(readLength, contentLength);
    

    while (readLength < contentLength && client.connected() && millis() - timeout < 10000L)
    {
        int i = 0;
        while (client.available())
        {
                // read file data to spiffs
            if (!file.print(char(client.read())))
            {
                Serial.println("Appending file");
            }
            //Serial.print((char)c);       // Uncomment this to show data
            //crc.update(c);
            readLength++;

            if (readLength % (contentLength / 13) == 0)
            {
                printPercent(readLength, contentLength);
            }
            timeout = millis();
        }
    }

    file.close();

    printPercent(readLength, contentLength);
    timeElapsed = millis() - timeElapsed;
    Serial.println();

    client.stop();
    Serial.println("stop client");

    modem.gprsDisconnect();
    Serial.println("gprs disconnect");
    Serial.println();

    float duration = float(timeElapsed) / 1000;
  /*
    Serial.print("Tamaño de Archivo: ");
    Serial.println(contentLength);
    Serial.print("Leido:  ");
    Serial.println(readLength);
    Serial.print("Calculado. CRC32:    0x");
    Serial.println(crc.finalize(), HEX);
    Serial.print("Conocido CRC32:    0x");
    Serial.println(knownCRC32, HEX);
    Serial.print("Bajado en:       ");
    Serial.print(duration);
    Serial.println("s");

    Serial.println("Se genera una espera de 3 segundos");
    for (int i = 0; i < 3; i++)
    {
        Serial.print(String(i) + "...");
        delay(1000);
    }
  */
    //readFile(SPIFFS, "/update.bin");

    updateFromFS();

    // Do nothing forevermore
    while (true)
    {
        delay(1000);
    }
}







void appendFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Appending to file: %s\n", path);

    File file = fs.open(path, FILE_APPEND);
    if (!file)
    {
        Serial.println("Failed to open file for appending");
        return;
    }
    if (file.print(message))
    {
        Serial.println("APOK");
    }
    else
    {
        Serial.println("APX");
    }
}

void readFile(fs::FS &fs, const char *path)
{
    Serial.printf("Reading file: %s\n", path);

    File file = fs.open(path);
    if (!file || file.isDirectory())
    {
        Serial.println("Failed to open file for reading");
        return;
    }

    Serial.print("Read from file: ");
    while (file.available())
    {
        Serial.write(file.read());
        delayMicroseconds(100);
    }
}

void writeFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Writing file: %s\n", path);

    File file = fs.open(path, FILE_WRITE);
    if (!file)
    {
        Serial.println("Failed to open file for writing");
        return;
    }
    if (file.print(message))
    {
        Serial.println("File written");
    }
    else
    {
        Serial.println("Write failed");
    }
}

void listDir(fs::FS &fs, const char *dirname, uint8_t levels)
{
    Serial.printf("Listing directory: %s\n", dirname);

    File root = fs.open(dirname);
    if (!root)
    {
        Serial.println("Failed to open directory");
        return;
    }
    if (!root.isDirectory())
    {
        Serial.println("Not a directory");
        return;
    }

    File file = root.openNextFile();
    while (file)
    {
        if (file.isDirectory())
        {
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if (levels)
            {
                listDir(fs, file.name(), levels - 1);
            }
        }
        else
        {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("  SIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void deleteFile(fs::FS &fs, const char *path)
{
    Serial.printf("Deleting file: %s\n", path);
    if (fs.remove(path))
    {
        Serial.println("File deleted");
    }
    else
    {
        Serial.println("Delete failed");
    }
}

void updateFromFS()
{
    File updateBin = SPIFFS.open("/update.bin");
    if (updateBin)
    {
        if (updateBin.isDirectory())
        {
            Serial.println("Directory error");
            updateBin.close();
            return;
        }

        size_t updateSize = updateBin.size();

        if (updateSize > 0)
        {
            Serial.println("Starting update");
            performUpdate(updateBin, updateSize);
        }
        else
        {
            Serial.println("Error, archivo vacío");
        }

        updateBin.close();

        // whe finished remove the binary from sd card to indicate end of the process
        //fs.remove("/update.bin");
    }
    else
    {
        Serial.println("no such binary");
    }
}

void performUpdate(Stream &updateSource, size_t updateSize)
{
    if (Update.begin(updateSize))
    {
        size_t written = Update.writeStream(updateSource);
        if (written == updateSize)
        {
            Serial.println("Writes : " + String(written) + " successfully");
        }
        else
        {
            Serial.println("Written only : " + String(written) + "/" + String(updateSize) + ". Retry?");
        }
        if (Update.end())
        {
            Serial.println("OTA finished!");
            if (Update.isFinished())
            {
                Serial.println("Restart ESP device!");
                ESP.restart();
            }
            else
            {
                Serial.println("OTA not fiished");
            }
        }
        else
        {
            Serial.println("Error occured #: " + String(Update.getError()));
        }
    }
    else
    {
        Serial.println("Cannot beggin update");
    }
}





void printPercent(uint32_t readLength, uint32_t contentLength)
{
    // If we know the total length
    if (contentLength != -1)
    {
        Serial.print("\r ");
        Serial.print((100.0 * readLength) / contentLength);
        Serial.print('%');
    }
    else
    {
        Serial.println(readLength);
    }
}



void setupModem()
{
#ifdef MODEM_RST
    // Keep reset high
    pinMode(MODEM_RST, OUTPUT);
    digitalWrite(MODEM_RST, HIGH);
#endif

    pinMode(MODEM_PWRKEY, OUTPUT);
    pinMode(MODEM_POWER_ON, OUTPUT);

    // Turn on the Modem power first
    digitalWrite(MODEM_POWER_ON, HIGH);

    // Pull down PWRKEY for more than 1 second according to manual requirements
    digitalWrite(MODEM_PWRKEY, HIGH);
    delay(100);
    digitalWrite(MODEM_PWRKEY, LOW);
    delay(1000);
    digitalWrite(MODEM_PWRKEY, HIGH);

    // Initialize the indicator as an output
    pinMode(LED_GPIO, OUTPUT);
    digitalWrite(LED_GPIO, LED_OFF);
}

Make sure to change accordingle the following lines of code:

const char apn[] = "omnitel";
const char user[] = "omni";
const char pass[] = "omni";

and

const char server[] = "myesptestserver.ddns.net";
const int port = 80;
const char resource[] = "/esp.bin"; //here de bin file

For storing the bin files, you have multiple choises.

  1. Use Raspberry pi as a server ( I have created a simple apache server where I uploaded my binary file, and then I can access that server through the following sketch.

However, for testing it maybe be easier to setup Xampp on windows machine just to see whether you can get it to work.
2. Create an apache or simmilar server on your windows machine ( probably the simplest method would be to use Xampp as it does everything for you. In the xampp folder you will have htdocs folder where you can put your binary. and you can access that binary by using my sketch or your own sketch

const char server[] = "myesptestserver.ddns.net"; // ip of server or web address
const int port = 80;
const char resource[] = "/esp.bin"; // put your binary file name

Let me know if you have any more questions

This firmware below make ota from Github. I try mix your code with this, but I yet cant:

#include <WiFi.h>
#include <HTTPClient.h>
#include <HTTPUpdate.h>
#include <WiFiClientSecure.h>
#include "cert.h"

const char * ssid = "home_wifi";
const char * password = "helloworld";

String FirmwareVer = {
"2.2"
};
#define URL_fw_Version "https://raw.githubusercontent.com/programmer131/ESP8266_ESP32_SelfUpdate/master/esp32_ota/bin_version.txt"
#define URL_fw_Bin "https://raw.githubusercontent.com/programmer131/ESP8266_ESP32_SelfUpdate/master/esp32_ota/fw.bin"

//#define URL_fw_Version "http://cade-make.000webhostapp.com/version.txt"
//#define URL_fw_Bin "http://cade-make.000webhostapp.com/firmware.bin"

void connect_wifi();
void firmwareUpdate();
int FirmwareVersionCheck();

unsigned long previousMillis = 0; // will store last time LED was updated
unsigned long previousMillis_2 = 0;
const long interval = 60000;
const long mini_interval = 1000;
void repeatedCall() {
static int num=0;
unsigned long currentMillis = millis();
if ((currentMillis - previousMillis) >= interval) {
// save the last time you blinked the LED
previousMillis = currentMillis;
if (FirmwareVersionCheck()) {
firmwareUpdate();
}
}
if ((currentMillis - previousMillis_2) >= mini_interval) {
previousMillis_2 = currentMillis;
Serial.print("idle loop...");
Serial.print(num++);
Serial.print(" Active fw version:");
Serial.println(FirmwareVer);
if(WiFi.status() == WL_CONNECTED)
{
Serial.println("wifi connected");
}
else
{
connect_wifi();
}
}
}

struct Button {
const uint8_t PIN;
uint32_t numberKeyPresses;
bool pressed;
};

Button button_boot = {
0,
0,
false
};
/void IRAM_ATTR isr(void arg) {
Button* s = static_cast<Button*>(arg);
s->numberKeyPresses += 1;
s->pressed = true;
}*/

void IRAM_ATTR isr() {
button_boot.numberKeyPresses += 1;
button_boot.pressed = true;
}

void setup() {
pinMode(button_boot.PIN, INPUT);
attachInterrupt(button_boot.PIN, isr, RISING);
Serial.begin(115200);
Serial.print("Active firmware version:");
Serial.println(FirmwareVer);
pinMode(LED_BUILTIN, OUTPUT);
connect_wifi();
}
void loop() {
if (button_boot.pressed) { //to connect wifi via Android esp touch app
Serial.println("Firmware update Starting..");
firmwareUpdate();
button_boot.pressed = false;
}
repeatedCall();
}

void connect_wifi() {
Serial.println("Waiting for WiFi");
WiFi.begin(ssid, password);
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}

Serial.println("");
Serial.println("WiFi connected");
Serial.println("IP address: ");
Serial.println(WiFi.localIP());
}

void firmwareUpdate(void) {
WiFiClientSecure client;
client.setCACert(rootCACertificate);
httpUpdate.setLedPin(LED_BUILTIN, LOW);
t_httpUpdate_return ret = httpUpdate.update(client, URL_fw_Bin);

switch (ret) {
case HTTP_UPDATE_FAILED:
Serial.printf("HTTP_UPDATE_FAILD Error (%d): %s\n", httpUpdate.getLastError(), httpUpdate.getLastErrorString().c_str());
break;

case HTTP_UPDATE_NO_UPDATES:
Serial.println("HTTP_UPDATE_NO_UPDATES");
break;

case HTTP_UPDATE_OK:
Serial.println("HTTP_UPDATE_OK");
break;
}
}
int FirmwareVersionCheck(void) {
String payload;
int httpCode;
String fwurl = "";
fwurl += URL_fw_Version;
fwurl += "?";
fwurl += String(rand());
Serial.println(fwurl);
WiFiClientSecure * client = new WiFiClientSecure;

if (client)
{
client -> setCACert(rootCACertificate);

// Add a scoping block for HTTPClient https to make sure it is destroyed before WiFiClientSecure *client is 
HTTPClient https;

if (https.begin( * client, fwurl)) 
{ // HTTPS      
  Serial.print("[HTTPS] GET...\n");
  // start connection and send HTTP header
  delay(100);
  httpCode = https.GET();
  delay(100);
  if (httpCode == HTTP_CODE_OK) // if version received
  {
    payload = https.getString(); // save received version
  } else {
    Serial.print("error in downloading version file:");
    Serial.println(httpCode);
  }
  https.end();
}
delete client;

}

if (httpCode == HTTP_CODE_OK) // if version received
{
payload.trim();
if (payload.equals(FirmwareVer)) {
Serial.printf("\nDevice already on latest firmware version:%s\n", FirmwareVer);
return 0;
}
else
{
Serial.println(payload);
Serial.println("New firmware detected");
return 1;
}
}
return 0;
}

@krupis
Copy link
Author

krupis commented May 16, 2021 via email

@eueduardo3
Copy link

I tried to use your script to try to OTA a file hosted on GitHub, but I couldn't. In this example that I posted, it is okay to OTA a file through github through Wifi, but I was unable to do that using a gsm modem through the TinyGsmClient library

@krupis
Copy link
Author

krupis commented May 16, 2021 via email

@eueduardo3
Copy link

That is really weird. I have created an apache server using xampp and can do an OTA from there. What error are you getting? Are you using esp32 with spiffs fs partition? Do you know where your code gets stuck

On Sun, 16 May 2021, 23:38 eueduardo3, @.***> wrote: I tried to use your script to try to OTA a file hosted on GitHub, but I couldn't. In this example that I posted, it is okay to OTA a file through github through Wifi, but I was unable to do that using a gsm modem through the TinyGsmClient library — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#132 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFJU62U5PEISOCCM36IP2OTTOAUNPANCNFSM42SG5BMA .

Now it worked!
Thank you very much for making your code available and helping me. I changed the server and it worked. I was using 000webhost. And it was certainly my mistake, I was not sure how to correctly make the file available for download. Switching from server to another site / server with free hosting worked. Thank you very much for your attention in trying to help me. I wish success for your projects.

@gunter72
Copy link

I did a skectch and work fine, is any are interested I can share ti, but I dont know how to make the server to store the bin file.
I did the sketch using the TTGO CALL SIM800L github GSM tiny OTA example.
If someone has and server example I will apreciate it, my idea is to store the bin file on my web server.

Hi, could you share your code? I'm having lots of issues trying any other code

Thank you

@krupis
Copy link
Author

krupis commented Jun 24, 2021

I did a skectch and work fine, is any are interested I can share ti, but I dont know how to make the server to store the bin file.
I did the sketch using the TTGO CALL SIM800L github GSM tiny OTA example.
If someone has and server example I will apreciate it, my idea is to store the bin file on my web server.

Hi, could you share your code? I'm having lots of issues trying any other code

Thank you

Hey. Have you tried the OTA over GSM code that I have posted above? It would be a good start if you can get that to work, then you can modify it as you wish. Make sure you have some sort of webserver available where you going to store your binary file for the OTA. Myself, I have downloaded XAMPP on windows and store my binary files there, but there are many options available.

@gunter72
Copy link

Hi, I tried your code, but it always reaches 90% and then it gives me an error. I have a private webserver, but I also tried altervista, I always have the same problem.

@krupis
Copy link
Author

krupis commented Jun 24, 2021 via email

@gunter72
Copy link

Yes, I tried with small 100k files but I have the same error. My bin is 660k

@krupis
Copy link
Author

krupis commented Jun 24, 2021

Yes, I tried with small 100k files but I have the same error. My bin is 660k

I see. Thats unfortunate.. What GSM are you using? Have you tried to just read the whole binary file and not do the OTA and see if you can read the whole binary?
Instead of updateFromFS try to read your binary

    readFile(SPIFFS, "/update.bin");

    //updateFromFS(); // comment this out

@gunter72
Copy link

Hi krupis, thank you very much for your support. I found the problem. It was a mistake in my code.
The update took place while the mqtt communication was still active.
I have stopped all active clients first, then I start the update process and everything works.
Thanks so much!

@krupis
Copy link
Author

krupis commented Jun 28, 2021

Glad you found the issue.

@Mohamed-ali1998
Copy link

Mohamed-ali1998 commented Sep 4, 2021

Hello. Here is the code to perform OTA over the GSM:


#include <Update.h>
#define TINY_GSM_MODEM_SIM800
// Increase RX buffer
#define TINY_GSM_RX_BUFFER 1030

const char apn[] = "omnitel";
const char user[] = "omni";
const char pass[] = "omni";



#define MODEM_RST             5
#define MODEM_PWRKEY          4
#define MODEM_POWER_ON       23
#define MODEM_TX             27
#define MODEM_RX             26
#define LED_GPIO             13
#define LED_ON               HIGH
#define LED_OFF              LOW


#define SerialAT  Serial1

#include <TinyGsmClient.h>
#include <CRC32.h>

#include "FS.h"
#include "SPIFFS.h"

#ifdef DUMP_AT_COMMANDS
#include <StreamDebugger.h>
StreamDebugger debugger(SerialAT, Serial);
TinyGsm modem(debugger);
#else
TinyGsm modem(SerialAT);
#endif


TinyGsmClient client(modem);

const char server[] = "myesptestserver.ddns.net";
const int port = 80;
const char resource[] = "/esp.bin"; //here de bin file

uint32_t knownCRC32 = 0x6f50d767;
uint32_t knownFileSize = 1024; // In case server does not send it

void setup()
{
    SerialAT.begin(115200, SERIAL_8N1, MODEM_RX, MODEM_TX);
    // Set console baud rate
    Serial.begin(115200);

    setupModem();
    delay(10);

    if (!SPIFFS.begin(true))
    {
        Serial.println("SPIFFS Mount Failed");
        return;
    }
    SPIFFS.format();
    listDir(SPIFFS, "/", 0);

    // Set GSM module baud rate
    
    delay(3000);

    // Restart takes quite some time
    // To skip it, call init() instead of restart()
    Serial.println("Initializing modem...");
    modem.restart();

    String modemInfo = modem.getModemInfo();
    Serial.print("Modem: ");
    Serial.println(modemInfo);

    // Unlock your SIM card with a PIN
    //modem.simUnlock("1234");
}



void loop()
{
    Serial.print("Waiting for network...");
    if (!modem.waitForNetwork())
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(apn);
    if (!modem.gprsConnect(apn, user, pass))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");

    Serial.print("Connecting to ");
    Serial.print(server);

    // if you get a connection, report back via serial:
    if (!client.connect(server, port))
    {
        Serial.println(" fail");
        delay(10000);
        return;
    }
    Serial.println(" OK");
    
    // Make a HTTP request:
    client.print(String("GET ") + resource + " HTTP/1.0\r\n");
    client.print(String("Host: ") + server + "\r\n");
    client.print("Connection: close\r\n\r\n");

    long timeout = millis();
    while (client.available() == 0)
    {
        if (millis() - timeout > 5000L)
        {
            Serial.println(">>> Client Timeout !");
            client.stop();
            delay(10000L);
            return;
        }
    }

    Serial.println("Reading header");
    uint32_t contentLength = knownFileSize;

    File file = SPIFFS.open("/update.bin", FILE_APPEND);


    while (client.available())
    {
        String line = client.readStringUntil('\n');
        line.trim();
        //Serial.println(line);    // Uncomment this to show response header
        line.toLowerCase();
        if (line.startsWith("content-length:"))
        {
            contentLength = line.substring(line.lastIndexOf(':') + 1).toInt();
        }
        else if (line.length() == 0)
        {
            break;
        }
    }


    timeout = millis();
    uint32_t readLength = 0;
    CRC32 crc;

    unsigned long timeElapsed = millis();
    printPercent(readLength, contentLength);
    

    while (readLength < contentLength && client.connected() && millis() - timeout < 10000L)
    {
        int i = 0;
        while (client.available())
        {
                // read file data to spiffs
            if (!file.print(char(client.read())))
            {
                Serial.println("Appending file");
            }
            //Serial.print((char)c);       // Uncomment this to show data
            //crc.update(c);
            readLength++;

            if (readLength % (contentLength / 13) == 0)
            {
                printPercent(readLength, contentLength);
            }
            timeout = millis();
        }
    }

    file.close();

    printPercent(readLength, contentLength);
    timeElapsed = millis() - timeElapsed;
    Serial.println();

    client.stop();
    Serial.println("stop client");

    modem.gprsDisconnect();
    Serial.println("gprs disconnect");
    Serial.println();

    float duration = float(timeElapsed) / 1000;
  /*
    Serial.print("Tamaño de Archivo: ");
    Serial.println(contentLength);
    Serial.print("Leido:  ");
    Serial.println(readLength);
    Serial.print("Calculado. CRC32:    0x");
    Serial.println(crc.finalize(), HEX);
    Serial.print("Conocido CRC32:    0x");
    Serial.println(knownCRC32, HEX);
    Serial.print("Bajado en:       ");
    Serial.print(duration);
    Serial.println("s");

    Serial.println("Se genera una espera de 3 segundos");
    for (int i = 0; i < 3; i++)
    {
        Serial.print(String(i) + "...");
        delay(1000);
    }
  */
    //readFile(SPIFFS, "/update.bin");

    updateFromFS();

    // Do nothing forevermore
    while (true)
    {
        delay(1000);
    }
}







void appendFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Appending to file: %s\n", path);

    File file = fs.open(path, FILE_APPEND);
    if (!file)
    {
        Serial.println("Failed to open file for appending");
        return;
    }
    if (file.print(message))
    {
        Serial.println("APOK");
    }
    else
    {
        Serial.println("APX");
    }
}

void readFile(fs::FS &fs, const char *path)
{
    Serial.printf("Reading file: %s\n", path);

    File file = fs.open(path);
    if (!file || file.isDirectory())
    {
        Serial.println("Failed to open file for reading");
        return;
    }

    Serial.print("Read from file: ");
    while (file.available())
    {
        Serial.write(file.read());
        delayMicroseconds(100);
    }
}

void writeFile(fs::FS &fs, const char *path, const char *message)
{
    Serial.printf("Writing file: %s\n", path);

    File file = fs.open(path, FILE_WRITE);
    if (!file)
    {
        Serial.println("Failed to open file for writing");
        return;
    }
    if (file.print(message))
    {
        Serial.println("File written");
    }
    else
    {
        Serial.println("Write failed");
    }
}

void listDir(fs::FS &fs, const char *dirname, uint8_t levels)
{
    Serial.printf("Listing directory: %s\n", dirname);

    File root = fs.open(dirname);
    if (!root)
    {
        Serial.println("Failed to open directory");
        return;
    }
    if (!root.isDirectory())
    {
        Serial.println("Not a directory");
        return;
    }

    File file = root.openNextFile();
    while (file)
    {
        if (file.isDirectory())
        {
            Serial.print("  DIR : ");
            Serial.println(file.name());
            if (levels)
            {
                listDir(fs, file.name(), levels - 1);
            }
        }
        else
        {
            Serial.print("  FILE: ");
            Serial.print(file.name());
            Serial.print("  SIZE: ");
            Serial.println(file.size());
        }
        file = root.openNextFile();
    }
}

void deleteFile(fs::FS &fs, const char *path)
{
    Serial.printf("Deleting file: %s\n", path);
    if (fs.remove(path))
    {
        Serial.println("File deleted");
    }
    else
    {
        Serial.println("Delete failed");
    }
}

void updateFromFS()
{
    File updateBin = SPIFFS.open("/update.bin");
    if (updateBin)
    {
        if (updateBin.isDirectory())
        {
            Serial.println("Directory error");
            updateBin.close();
            return;
        }

        size_t updateSize = updateBin.size();

        if (updateSize > 0)
        {
            Serial.println("Starting update");
            performUpdate(updateBin, updateSize);
        }
        else
        {
            Serial.println("Error, archivo vacío");
        }

        updateBin.close();

        // whe finished remove the binary from sd card to indicate end of the process
        //fs.remove("/update.bin");
    }
    else
    {
        Serial.println("no such binary");
    }
}

void performUpdate(Stream &updateSource, size_t updateSize)
{
    if (Update.begin(updateSize))
    {
        size_t written = Update.writeStream(updateSource);
        if (written == updateSize)
        {
            Serial.println("Writes : " + String(written) + " successfully");
        }
        else
        {
            Serial.println("Written only : " + String(written) + "/" + String(updateSize) + ". Retry?");
        }
        if (Update.end())
        {
            Serial.println("OTA finished!");
            if (Update.isFinished())
            {
                Serial.println("Restart ESP device!");
                ESP.restart();
            }
            else
            {
                Serial.println("OTA not fiished");
            }
        }
        else
        {
            Serial.println("Error occured #: " + String(Update.getError()));
        }
    }
    else
    {
        Serial.println("Cannot beggin update");
    }
}





void printPercent(uint32_t readLength, uint32_t contentLength)
{
    // If we know the total length
    if (contentLength != -1)
    {
        Serial.print("\r ");
        Serial.print((100.0 * readLength) / contentLength);
        Serial.print('%');
    }
    else
    {
        Serial.println(readLength);
    }
}



void setupModem()
{
#ifdef MODEM_RST
    // Keep reset high
    pinMode(MODEM_RST, OUTPUT);
    digitalWrite(MODEM_RST, HIGH);
#endif

    pinMode(MODEM_PWRKEY, OUTPUT);
    pinMode(MODEM_POWER_ON, OUTPUT);

    // Turn on the Modem power first
    digitalWrite(MODEM_POWER_ON, HIGH);

    // Pull down PWRKEY for more than 1 second according to manual requirements
    digitalWrite(MODEM_PWRKEY, HIGH);
    delay(100);
    digitalWrite(MODEM_PWRKEY, LOW);
    delay(1000);
    digitalWrite(MODEM_PWRKEY, HIGH);

    // Initialize the indicator as an output
    pinMode(LED_GPIO, OUTPUT);
    digitalWrite(LED_GPIO, LED_OFF);
}

Make sure to change accordingle the following lines of code:

const char apn[] = "omnitel";
const char user[] = "omni";
const char pass[] = "omni";

and

const char server[] = "myesptestserver.ddns.net";
const int port = 80;
const char resource[] = "/esp.bin"; //here de bin file

For storing the bin files, you have multiple choises.

  1. Use Raspberry pi as a server ( I have created a simple apache server where I uploaded my binary file, and then I can access that server through the following sketch.

However, for testing it maybe be easier to setup Xampp on windows machine just to see whether you can get it to work.
2. Create an apache or simmilar server on your windows machine ( probably the simplest method would be to use Xampp as it does everything for you. In the xampp folder you will have htdocs folder where you can put your binary. and you can access that binary by using my sketch or your own sketch

const char server[] = "myesptestserver.ddns.net"; // ip of server or web address
const int port = 80;
const char resource[] = "/esp.bin"; // put your binary file name

Let me know if you have any more questions

@krupis Hi krupis, I'm ok with esp32 coding side. But I have no idea to configure xamp server and where to store the .bin file in the server. How can I do this? Can you share any reference for this?
Thanks in advance.

@krupis
Copy link
Author

krupis commented Sep 4, 2021

@Mohamed-ali1998 Hey mate. The binary should be stored inside htdocs folder where you have installed XAMP. There should be no additionall setup required apart from starting the XAMPP services. Also, keep in mind that in order to perfrom GSM OTA, you will need to port forward the IP address. I have used some free domain service. The reason is because the GSM is not connected to your home network and will not be able to reach your local computer IP address.

@Mohamed-ali1998
Copy link

@krupis Hi krupis. I used our own server. That's working fine. Thank you so much.

@TawalMc
Copy link

TawalMc commented Oct 15, 2021

@krupis Hi everyone. I tried @krupis solution but have some issues (it seems to work but...):

Setup

  1. Zorin OS 15.3
  2. Arduino IDE 1.8.15
  3. Board: DOIT ESP32 DEVKIT V1
  4. File size: 198944 byte

I use the secure version ( TinyGSMClientSecure) of TinyGSM due to our server configs..

The .bin file download stop at 99.34% (~197630) and start the update. So it failed and return Error 9 (UPDATE_ERROR_ACTIVATE). Below is the print on serial monitor

17:32:39.384 -> Reading header
17:32:40.382 -> 
 0.00%
 7.69%
 15.38%
 23.08%
 30.77%
 38.46%
 46.15%
 53.84%
 61.54%
 69.23%
 76.92%
 84.61%
 92.31%
 99.33%
17:36:54.214 -> stop client
17:36:54.580 -> gprs disconnect
17:36:54.580 -> 
17:36:54.580 -> Starting update
17:36:57.475 -> Writes : 197621 successfully
17:36:57.608 -> E (311078) esp_image: Checksum failed. Calculated 0x54 read 0xff
17:36:57.675 -> Error occured #: 9
17:36:57.675 -> END

So a help or an idea about ?
Thanks and great code!

@krupis
Copy link
Author

krupis commented Oct 15, 2021

@TawalMc Hello. I dont think I can help you with this issue but @gunter72 had a simillar problem.

Before you do anything else, I would try 2 things

  1. Try with small binary
  2. Isolate the rest of your code just to confirm if GSM OTA part is working. If it does, then start adding your other code bit by bit and see which part cause the issues.

@TawalMc
Copy link

TawalMc commented Oct 15, 2021 via email

@TawalMc
Copy link

TawalMc commented Oct 15, 2021

@TawalMc Hello. I dont think I can help you with this issue but @gunter72 had a simillar problem.

Before you do anything else, I would try 2 things

  1. Try with small binary
  2. Isolate the rest of your code just to confirm if GSM OTA part is working. If it does, then start adding your other code bit by bit and see which part cause the issues.

Thanks. I'll try them. Good

@KASSIMSAMJI
Copy link

@TawalMc Hello. I dont think I can help you with this issue but @gunter72 had a simillar problem.
Before you do anything else, I would try 2 things

  1. Try with small binary
  2. Isolate the rest of your code just to confirm if GSM OTA part is working. If it does, then start adding your other code bit by bit and see which part cause the issues.

Thanks. I'll try them. Good

Hello, i am getting same error#9, any luck ?

@ilker07
Copy link

ilker07 commented Dec 28, 2021

f

@TawalMc Hello. I dont think I can help you with this issue but @gunter72 had a simillar problem.
Before you do anything else, I would try 2 things

  1. Try with small binary
  2. Isolate the rest of your code just to confirm if GSM OTA part is working. If it does, then start adding your other code bit by bit and see which part cause the issues.

Thanks. I'll try them. Good

Hello, i am getting same error#9, any luck ?

I get same error sometimes.Sometime it works.Could you find any solution?

@KASSIMSAMJI
Copy link

f

@TawalMc Hello. I dont think I can help you with this issue but @gunter72 had a simillar problem.
Before you do anything else, I would try 2 things

  1. Try with small binary
  2. Isolate the rest of your code just to confirm if GSM OTA part is working. If it does, then start adding your other code bit by bit and see which part cause the issues.

Thanks. I'll try them. Good

Hello, i am getting same error#9, any luck ?

I get same error sometimes.Sometime it works.Could you find any solution?

in my case the success rate was so high (2 fails out of 20) few hours after midnight, so I deduced It is due to network being hogged during the day

@ilker07
Copy link

ilker07 commented Dec 28, 2021

in my case the success rate was so high (2 fails out of 20) few hours after midnight, so I deduced It is due to network being hogged during the day

this morning I upload the code and it updated but after that I created new bin file and it gave me error.The error is E (342241) esp_image: invalid segment length 0x2d832b6.

@ilker07
Copy link

ilker07 commented Dec 28, 2021

f

@TawalMc Hello. I dont think I can help you with this issue but @gunter72 had a simillar problem.
Before you do anything else, I would try 2 things

  1. Try with small binary
  2. Isolate the rest of your code just to confirm if GSM OTA part is working. If it does, then start adding your other code bit by bit and see which part cause the issues.

Thanks. I'll try them. Good

Hello, i am getting same error#9, any luck ?

I get same error sometimes.Sometime it works.Could you find any solution?

in my case the success rate was so high (2 fails out of 20) few hours after midnight, so I deduced It is due to network being hogged during the day

2021-12-28

Should I change these settings?

@zark-zeugan
Copy link

Most problems can be solved by by providing a good voltage for the modem. Even the 3.6V is considered to be the absolute minimum for the modem. The recommended is around 4.0V.

@krupis Okay. Just one last question do we have to write something in the code to let the device know which power source to use? Like use the USB one or the battery one that is attached?

@krupis
Copy link
Author

krupis commented Feb 15, 2022

You definately dont need to do that in the code. I am not much of a hardware guy so cant give you correct answer. I assume this depends on how the hardware is designed... I assume if you have USB plugged in but you also supply an external voltage higher than 3.3V than it should be fine.

@zark-zeugan
Copy link

Hello, so I finally had the SIM800L and the ESP32 SoC on the neat PCB, with an LM2596S-adj giving out 3.6V

I never had issues anymore with the OTA, out of 20 trials, no failure,

looks like It was a power issue to the SIM800L

@KASSIMSAMJI
Where are you hosting the binary files?

@KASSIMSAMJI
Copy link

Hello, so I finally had the SIM800L and the ESP32 SoC on the neat PCB, with an LM2596S-adj giving out 3.6V
I never had issues anymore with the OTA, out of 20 trials, no failure,
looks like It was a power issue to the SIM800L

@KASSIMSAMJI Where are you hosting the binary files?

I have a shared hosting server for that

@DP1000
Copy link

DP1000 commented Mar 2, 2022

Hello everyone, I'm using @krupis first code and i'm having 2 problems:

  1. It's very slow, for downloading/saving a sketch of 340KB it takes over an hour.
  2. It doesn't stop, it goes over 100%, i stopped it after 2 hours of downloading at about 175%.

I tried adding if(readLength == contentLength) break; in the downloading while loop, it took about 1:30 hours but it ended with an error.

Is it normal that it takes so much?

Did I missed something?

Thanks in advance.

@krupis
Copy link
Author

krupis commented Mar 2, 2022

Hello everyone, I'm using @krupis first code and i'm having 2 problems:

  1. It's very slow, for downloading/saving a sketch of 340KB it takes over an hour.
  2. It doesn't stop, it goes over 100%, i stopped it after 2 hours of downloading at about 175%.

I tried adding if(readLength == contentLength) break; in the downloading while loop, it took about 1:30 hours but it ended with an error.

Is it normal that it takes so much?

Did I missed something?

Thanks in advance.

That is quite strange. I have tested with simillar size of file and it took me no longer than 1 minute. Are you getting good signal power?

@DP1000
Copy link

DP1000 commented Mar 2, 2022

Hello everyone, I'm using @krupis first code and i'm having 2 problems:

  1. It's very slow, for downloading/saving a sketch of 340KB it takes over an hour.
  2. It doesn't stop, it goes over 100%, i stopped it after 2 hours of downloading at about 175%.

I tried adding if(readLength == contentLength) break; in the downloading while loop, it took about 1:30 hours but it ended with an error.
Is it normal that it takes so much?
Did I missed something?
Thanks in advance.

That is quite strange. I have tested with simillar size of file and it took me no longer than 1 minute. Are you getting good signal power?

I'm printing modem.getSignalQuality() at every printPercent() and it's always at 31, that it should be the maximum signal strenght.

@DP1000
Copy link

DP1000 commented Mar 2, 2022

Ok, I've found out that something that i've added was interfering.
I'm still searching what I've done wrong, when i'll discover it i'll add to this comment, in case anyone will encounter the same problem.

Edit:
The problem was #define TINY_GSM_RX_BUFFER 2048 instead of #define TINY_GSM_RX_BUFFER 1030, if someone knows why it's a problem let me know.

Sorry for bothering.

@ilker07
Copy link

ilker07 commented Mar 3, 2022

Ok, I've found out that something that i've added was interfering. I'm still searching what I've done wrong, when i'll discover it i'll add to this comment, in case anyone will encounter the same problem.

Edit: The problem was #define TINY_GSM_RX_BUFFER 2048 instead of #define TINY_GSM_RX_BUFFER 1030, if someone knows why it's a problem let me know.

Sorry for bothering.

I never tried to increase rx buffer.

@Sivapraveen-Entuple
Copy link

Sivapraveen-Entuple commented Mar 24, 2022

That is really weird. I have created an apache server using xampp and can do an OTA from there. What error are you getting? Are you using esp32 with spiffs fs partition? Do you know where your code gets stuck

On Sun, 16 May 2021, 23:38 eueduardo3, @.***> wrote: I tried to use your script to try to OTA a file hosted on GitHub, but I couldn't. In this example that I posted, it is okay to OTA a file through github through Wifi, but I was unable to do that using a gsm modem through the TinyGsmClient library — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#132 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFJU62U5PEISOCCM36IP2OTTOAUNPANCNFSM42SG5BMA .

Now it worked! Thank you very much for making your code available and helping me. I changed the server and it worked. I was using 000webhost. And it was certainly my mistake, I was not sure how to correctly make the file available for download. Switching from server to another site / server with free hosting worked. Thank you very much for your attention in trying to help me. I wish success for your projects.

eueduardo3 I'm also trying to update firmware via OTA using SIM868. Could you please share the firmware which is working fine. it will be more helpful for me to further

thanks in advance

@ilker07
Copy link

ilker07 commented Mar 28, 2022

f

@TawalMc Hello. I dont think I can help you with this issue but @gunter72 had a simillar problem.
Before you do anything else, I would try 2 things

  1. Try with small binary
  2. Isolate the rest of your code just to confirm if GSM OTA part is working. If it does, then start adding your other code bit by bit and see which part cause the issues.

Thanks. I'll try them. Good

Hello, i am getting same error#9, any luck ?

I get same error sometimes.Sometime it works.Could you find any solution?

in my case the success rate was so high (2 fails out of 20) few hours after midnight, so I deduced It is due to network being hogged during the day

2021-12-28

Should I change these settings?

How should these settings be(partition scheme)? or What does it change?

Why this is like this? Why 1030? When will we change it? #define TINY_GSM_RX_BUFFER 1030

@krupis
Copy link
Author

krupis commented Mar 28, 2022

Default settings are good. You can change various things here but I suggest you keep them as they are untill you get more familiar with each one and then you know exactly what you are doing.

Its been a while but TINY_GSM_RX_BUFFER 1030 does not have to be 1030. You can play with it and will have an effect on how many bytes can modem receive at once.

Actually, it should be set to 1024 because it would make more sense as this would mean you have 1kB chunks.

You can try for yourself and set the TINY_GSM_RX_BUFFER to values such as ,128, 256, 512 ,1024, 2048 and etc and see what difference you see.
Either way, it should work :) are you having any problems?

@ilker07
Copy link

ilker07 commented Mar 28, 2022

Default settings are good. You can change various things here but I suggest you keep them as they are untill you get more familiar with each one and then you know exactly what you are doing.

Its been a while but TINY_GSM_RX_BUFFER 1030 does not have to be 1030. You can play with it and will have an effect on how many bytes can modem receive at once.

Actually, it should be set to 1024 because it would make more sense as this would mean you have 1kB chunks.

You can try for yourself and set the TINY_GSM_RX_BUFFER to values such as ,128, 256, 512 ,1024, 2048 and etc and see what difference you see. Either way, it should work :) are you having any problems?

Actually right now ,i have no trouble at all .I just wondered why this numbers.I use buffer 1030.I just wanna know when we will change.Right now with my code it takes 10-15 minutes to update code.I never tried to change buffer value or partition scheme settings.Could it be related these 2 things that it takes 10-15 minutes?

For example my bin size 426 kb.how can we relate between this file size and buffer size?

@krupis
Copy link
Author

krupis commented Mar 28, 2022

Default settings are good. You can change various things here but I suggest you keep them as they are untill you get more familiar with each one and then you know exactly what you are doing.
Its been a while but TINY_GSM_RX_BUFFER 1030 does not have to be 1030. You can play with it and will have an effect on how many bytes can modem receive at once.
Actually, it should be set to 1024 because it would make more sense as this would mean you have 1kB chunks.
You can try for yourself and set the TINY_GSM_RX_BUFFER to values such as ,128, 256, 512 ,1024, 2048 and etc and see what difference you see. Either way, it should work :) are you having any problems?

Actually right now ,i have no trouble at all .I just wondered why this numbers.I use buffer 1030.I just wanna know when we will change.Right now with my code it takes 10-15 minutes to update code.I never tried to change buffer value or partition scheme settings.Could it be related these 2 things that it takes 10-15 minutes?

For example my bin size 426 kb.how can we relate between this file size and buffer size?

10-15 minutes seems like quite a long time. The buffer size may have tiny affect on this but you definately have some other issues. When I did my testing, with good signal strength I managed to perform OTA in about 30 seconds.

You should try different server to confirm if the issue is not from the server side and also ensure that you get good signal.

The buffer size simply describes what is the maximum receive buffer length. Since you now defined it to be 1030, that means the modem can receive 1030 bytes before it needs to split the chunk and receive another chunk of 1030 bytes. In theory, if you had infinite amount of RAM, you could set the RX buffer to something ridiculous lets say 1MB. That would allow you the receive a whole firmware in a single data chunk thus saving some time in theory.. In practise things vary..

@ilker07
Copy link

ilker07 commented Mar 29, 2022

Default settings are good. You can change various things here but I suggest you keep them as they are untill you get more familiar with each one and then you know exactly what you are doing.
Its been a while but TINY_GSM_RX_BUFFER 1030 does not have to be 1030. You can play with it and will have an effect on how many bytes can modem receive at once.
Actually, it should be set to 1024 because it would make more sense as this would mean you have 1kB chunks.
You can try for yourself and set the TINY_GSM_RX_BUFFER to values such as ,128, 256, 512 ,1024, 2048 and etc and see what difference you see. Either way, it should work :) are you having any problems?

Actually right now ,i have no trouble at all .I just wondered why this numbers.I use buffer 1030.I just wanna know when we will change.Right now with my code it takes 10-15 minutes to update code.I never tried to change buffer value or partition scheme settings.Could it be related these 2 things that it takes 10-15 minutes?
For example my bin size 426 kb.how can we relate between this file size and buffer size?

10-15 minutes seems like quite a long time. The buffer size may have tiny affect on this but you definately have some other issues. When I did my testing, with good signal strength I managed to perform OTA in about 30 seconds.

You should try different server to confirm if the issue is not from the server side and also ensure that you get good signal.

The buffer size simply describes what is the maximum receive buffer length. Since you now defined it to be 1030, that means the modem can receive 1030 bytes before it needs to split the chunk and receive another chunk of 1030 bytes. In theory, if you had infinite amount of RAM, you could set the RX buffer to something ridiculous lets say 1MB. That would allow you the receive a whole firmware in a single data chunk thus saving some time in theory.. In practise things vary..

actually you may be right .it takes always 10-15 minutes whatever the size of the file.Could it be realated power issues or gprs?

@krupis
Copy link
Author

krupis commented Mar 29, 2022 via email

@ilker07
Copy link

ilker07 commented Mar 29, 2022

The buffer size simply describes what is the maximum receive buffer length. Since you now defined it to be 1030, that means the modem can receive 1030 bytes before it needs to split the chunk and receive another chunk of 1030 bytes. In theory, if you had infinite amount of RAM, you could set the RX buffer to something ridiculous lets say 1MB. That would allow you the receive a whole firmware in a single data chunk thus saving some time in theory.. In practise things vary.

Yes. Power issues or the signal can be related to the length of the OTA. Ensure that the MODEM must have stable and powerful enough power supply. If you want to test that, You can provide an external power to the modem if you have a lab power supply unit

How do you supply the circuit?Can you show me?

@krupis
Copy link
Author

krupis commented Mar 29, 2022

Could you show me what board are you using and explain how are you currently powering it?

@ilker07
Copy link

ilker07 commented Mar 29, 2022

Could you show me what board are you using and explain how are you currently powering it?

3.7 v lithium battery and 5v through USB. if I don t use the USB ,it does not work.

@krupis
Copy link
Author

krupis commented Mar 29, 2022

That is a first indicator that you have some power issues if it does not work when USB is disconnected.
Can you measure the voltage across the battery terminals?
Have you tried with 100% charged battery?
Have you got a spare battery just to test it?

@krupis
Copy link
Author

krupis commented Mar 29, 2022

Also, are you using battery connector that is located on the bottom of the board to connect your 3.7V battery or you are connecting it to 3.3V?

The correct way is to connect it to battery connector

@ilker07
Copy link

ilker07 commented Mar 29, 2022

That is a first indicator that you have some power issues if it does not work when USB is disconnected.
Can you measure the voltage across the battery terminals?
Have you tried with 100% charged battery?
Have you got a spare battery just to test it?

Unfortunaletly i dont have spare battery.Right now i dont have the circuit but the voltage across the battery terminals around 4v as far as i remember.

@ilker07
Copy link

ilker07 commented Mar 29, 2022

Also, are you using battery connector that is located on the bottom of the board to connect your 3.7V battery or you are connecting it to 3.3V?

The correct way is to connect it to battery connector

yeah,i use that way

@krupis
Copy link
Author

krupis commented Mar 29, 2022

Seems to be a common power issue for these boards.. Have a read at:
#29

Xinyuan-LilyGO/TTGO-T-Display#27

If you have a lab power supply you can experiment with the power but if you dont know what you are doing you might destroy the device.

You could probably wire the lab power supply terminal wires to the usb +5V and GND pins. You may want to disconnect the battery when you do this. That way you would have a strong power supply ( much stronger than you PC USB ) . Keep in mind that the whole system probably needs up to 3A to be stable. So you need a power supply that can deliver 5V 3A.

@ilker07
Copy link

ilker07 commented Mar 29, 2022

Seems to be a common power issue for these boards.. Have a read at: #29

Xinyuan-LilyGO/TTGO-T-Display#27

If you have a lab power supply you can experiment with the power but if you dont know what you are doing you might destroy the device.

You could probably wire the lab power supply terminal wires to the usb +5V and GND pins. You may want to disconnect the battery when you do this. That way you would have a strong power supply ( much stronger than you PC USB ) . Keep in mind that the whole system probably needs up to 3A to be stable. So you need a power supply that can deliver 5V 3A.

i have power supply 5v 3a . i will try what you said.i will let you know ,Thanks.

@krupis
Copy link
Author

krupis commented Mar 29, 2022

Yes please be sure to let everyone know if you managed to confirm whether this is related to power issue after all :)

@mostafahk
Copy link

I recommend you use OTAdrive for OTA through GSM. see here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests