Supporting latest Arduino Core v3 - issues with fixed Network IP address, settings lost after upgrade etc

[tp1de]

Being in my holiday home with netfit thermostat moduline 2050 (ems 2.0) I recognized that the thermostat_hc1_mode does not work anymore (neither dashboard nor mqtt). No changes possible - no error reported.

EMS-ESP version 3.7.0-dev.10 works ok, but EMS-ESP version 3.7.0-dev.12 and 13 are not working.
On my EMS+ system with RC310 versions EMS-ESP version 3.7.0-dev.12 and 13 are working fine.

What has been changed?

[tp1de]

.... and while installing 3.7.0-dev.13 both systems do not restart anymore. My home system with E32 needed a hard reboot and was working again.
My S3 system within my holiday home lost all config data and wifi started with access point. This is not good.

[MichaelDvP]

For mode see #1800, should be fixed.

The new dev is compiled with arduino 3.0.1. Maybe the newer littleFS version reformats on an old system. I've seen this a long time ago when frst trying the arduino 3.xx. But since that i can change between 2.xx and 3.xx versions without issue.

[proddy]

This is indeed not good. We'll need to test the migration and see what is causing the FS issue. If it's a problem for others we should perhaps better remove the firmware bin files from the dev repo

[MichaelDvP]

I don't think it'a a general issue, Harvay in #1800 and others don't have FS cleared on update.
I think there was one specific version in the past and systems USB-flashed with this version are not recognized y the arduino-3-LittleFS. I have seen it a very few time, but can not reproduce and find the specific version.

[tp1de]

For mode see #1800, should be fixed.

The fix is still not within latest dev13 .... or?

[mattreim]

Version 3.7.0-dev.13(3) from yesterday has been running for me for 24 hours without any problems.

[tp1de]

I tested again: dev.13 was not working and after uploading bin-file all settings where lost on my S3 gateway again.
So I flashed the gateway from my laptop , loaded the settings and afterwards it was working again.
The thermostat.hc1.mode was changeable. So far so good.

But I recognized that the network settings in respect to fixed ip address and hostname where loaded but not used.
(dhcp instead). Fixed ip was only used after some retries. The defined hostname is not populated on the network.

[mattfro]

I have esp32 s3, after going from dev12 to dev14..I lost wifi, esp-ems went into AP mode.
In AP mode I tried to use wifi scan to search for networks, it did not find anything. Going back to dev12 it worked again.
In the end I even tried to manually install dev14 with ESP flasher and in that point I think all my settings was lost, but still wifi scan did not work. Stupid me, did not even think of setting wifi manually before this :) Because that seems to work in the end. Now running dev14 after setting back all settings like mqtt and wifi. It seems to work as it should

[proddy]

Thanks @mattfro - there are definitely something compatibility issues with the LittleFS file system that stores the EMS-ESP settings between Espressif Core (v2 on dev 12 and before) and the newer builds on Core v3.0.1 (dev 13 + 14). When I'm back home I'll need to run some tests, flashing with dev12 on an S3 and trying the upgrade. It may also be something related to the partitions or how the BBQKees gateways are originally flashed - we'll see. For now I would stick to dev12.

[MichaelDvP]

I don't know why s3 compiles with arduino core 3.xx. There seems to be a bug in platformio.
We use platform = espressif32 which should use the latest stable from https://registry.platformio.org/platforms/platformio/espressif32, which is 6.7.0 with arduino core 2.0.16. Also https://github.com/platformio/platform-espressif32/releases says latest release is 6.7.0.
Using the development branch platform=https://github.com/platformio/platform-espressif32.git compiles with arduino 2.0.17.

In https://github.com/MichaelDvP/EMS-ESP32/releases i have the actual dev14 compiled with arduino core 2 (s3: espessif32@6.7.0, esp32: tasmota@2024-05-00)

With the arduino 3.0.1 i don't see issues with littleFS and fixed ip, but i can confirm that network scanner is not working with arduino 3.0.1 and 3.0.0 builds.

[proddy]

Maybe just explicitly add the 6.7.0 to the platform as we did last time.

For WiFi maybe it's related to the event names https://randomnerdtutorials.com/esp32-migrating-version-2-to-3-arduino/#wifi-event-name-changes

I'm not home so can't test at the moment

[MichaelDvP]

Maybe just explicitly add the 6.7.0 to the platform as we did last time.

That's exactly what i have done for the test: see b175026

BTW: the S3 arduino3.0.1builds are also very small, like the tasmota-arduino without TSL, only 1.6M bin file
But with:

platform_packages=
  platformio/framework-arduinoespressif32 @ https://github.com/espressif/arduino-esp32.git#3.0.1
  platformio/framework-arduinoespressif32-libs @ https://github.com/espressif/esp32-arduino-libs.git#idf-release/v5.1

i get 2.1M bin file. (But also network scan with empty result)

The event names are changed since arduino 2.0.12, in 3.xx the old ones are removed and throw error when using.

[proddy]

I made a dev14 that uses the previous 2.0.17 Arduino espressif32 library

[tp1de]

It is dev 15 and seems to work as before. WiFi scan is ok.

[mattfro]

Same here... everything seems to work now.

[proddy]

thanks @tp1de and @mattfro for quickly testing. I think we'll keep to the old Ardunio library until we see the benefits of moving to v3 of the core and have fully tested the Network and Filesystem changes.

Interesting article: https://www.cnx-software.com/2024/06/01/espressif-releases-arduino-esp32-core-3-0-0-but-platformio-support-in-doubt/