ESP32 Support...

[aaronse]

Hello!

Been using my ESP32 fork #44 for couple of projects for ~yr. However, my forked branch has fallen behind, the pull request included a bunch GUI edits, unrelated to ESP32 port, and I didn't get around to splitting the pull request into more digestible edits. Open to me trying again? This time I'd submit pull request with bare minimum to get OTA and Config working, refraining from GUI and other edits.

Asking because... Am currently working on some more ESP32 projects, trying to figure out whether to create a basic ESP32 port that can merge to main. Doubt I'll be able to get everything working, but hopefully enough that others like me will benefit.

Let me know if it's worth making another attempt, or, if you prefer the main branch to not support optional ESP32 compilation? Noticed </README.md> "many functionalities of this framework are supported by ESP-IDF for the ESP32". I've not looked at ESP-IDF much before, checking it out in the meantime...

Thanks again for creating and extending this framework! Cheers!

aaron

[aaronse]

Can see progress at https://github.com/aaronse/esp8266-iot-framework/tree/esp32.

Lots TODO, but builds, deploys. Limited set of web portal features working.

[aaronse]

Pushed some ESP32 updates implementing/fixing file upload/delete/download and firmware flashing.

Futures: Out of scope for these ESP32 edits. But consider detecting flash progress and completion faster by implementing more aggressive client request timeout (retry with capped backoff) for /api/update-status

[aaronse]

Anyone out there successfully doing ESP32-S2 via PlatformIO within VSCode?

As well as ESP32, am working on a ESP32-S2 project (Adafruit QT Py ESP32-S2). Naturally, although ESP32 [esp32dev] deployment works. Am currently unable to get ESP32-S2 framework=arduino to work within VSCode-PlatformIO. Am able to use Arduino IDE 2.0 and MicroPython.

Slowly learning about ESP-IDF ( framework = espidf ), ideally, would like to just get this platform working for ESP32-S2 and just get on with coding rest of my project code. Curious if folks have tutorial recommendations? Am digging through :

• https://docs.espressif.com/projects/esp-idf
• https://www.youtube.com/watch?v=fVcru1pq9TY

[aaronse]

Anyone out there successfully doing ESP32-S2 via PlatformIO within VSCode? Or, anyone doing ESP32-S2 using something other than CircuitPython?

[Eisbaeeer]

I tried, but don´t get any framework generated in .pio\libdeps\esp32dev
Using a ESP32-C3 board.
WS2812FX example running fine.
Greets

[aaronse]

Hello Lars! Am in Seattle area (~1am), will try again later today and share my setup info. Cheers!

[Eisbaeeer]

Hey, this will be fine. Im located in germany. +2h GMT.
I will try your version and give feedback. Greets

[erikm30]

Question: Do you expect the framework to work with M5Stack Boards with ESP32 ? They look quite neat and would be useful as they integrate a display and the processor in a box.

[aaronse]

Hey, this will be fine. Im located in germany. +2h GMT. I will try your version and give feedback. Greets

@Eisbaeeer created repo with an example helloWorld project that builds and deployed to my heltec ESP32 and M5Stack ESP32.
Example project @ https://github.com/aaronse/esp-iot-framework-examples/tree/main/helloWorld

Note WifiManager.begin() must be called before GUI.begin, see readme.md for details on this and other quirks related to using this framework on ESP32.

[aaronse]

Question: Do you expect the framework to work with M5Stack Boards with ESP32 ? They look quite neat and would be useful as they integrate a display and the processor in a box.

Hello @erikm30, yes, is possible to build a subset of this framework's functionality to work on a ESP32 based M5Stack. Just created an example hello world that uses the edits in this ESP32 Support pull request at helloWorld_M5Stack

Note this pull request as-is doesn't fully implement time sync, secure client or cert store. Grep "TODO:" for details.

That said, I wouldn't blame anyone for using UI Flow, see https://flow.m5stack.com or Micropython instead depending on their requirements and available time.

[aaronse]

Added clone and build instructions to aaronse/esp-iot-framework-examples#1 (comment)

[marcolino7]

Hi @aaronse,
starting make some test with ESP32 versione, I can compile e portal work. I have issue with Time Sync.
Calling timeSync.isSynced() will return false and do not sync time against ntp.

Do you have any suggestion?

Marco

[maakbaas]

Hi aaronse, great work to continue this. In the coming period I think I will have some time to support the effort.

Linked to the discussion in #140 I would propose the following:

• If you create a PR with only ESP32 changes and nothing else, meaning the only thing that is allowed are basically #ifdef ESP32 blocks, I will push this to an ESP32 branch and see where we get 😊.
• Other functionalities will be considered on a change by change basis as PR's to the master repo.

I will try to work with existing issues and PR's this week to try to get to version 1.11.0. We can then use this as a baseline for the ESP32 branch, and I will aim then to move from 1.11.0 to 2.0.0 which will support the ESP32.

[Eisbaeeer]

Hey maakbaas
Can I support you by adding the esp32 to framework?
This will be so great and I have some projects in my tasks.
Greets Lars

[Eisbaeeer]

Hi maakbaas
Can you tell us the ETA of version 2.00? Feel free to contact me to test alpha! ;-)
Greets Lars

[maakbaas]

@Eisbaeeer @aaronse Apparently it took me a little bit more than one week to get 1.11.0 out of the door 😉. To get back to my post from April:

• I have now created an ESP32 branch based on the latest release
• I will accept pull requests into this branch with only ESP32 changes and nothing else, meaning the only thing that is allowed are basically #ifdef ESP32 blocks.

The branch will only move to the master if 100% of the functionality is ported, but until then of course the branch can be used for partial functionality

[aaronse]

Hello @maakbaas! Please confirm you're not planning to make any major edits in the next 'week' ? More churn potentially means more time required to put a pull request together. Providing main branch will remain relatively untouched, I'll have a go at porting the ESP32 changes from my ESP32 fork https://github.com/aaronse/esp8266-iot-framework/tree/esp32 sound good?

Recently wrapped up making a CNC ( https://www.youtube.com/watch?v=FJ7nYio8oOE ), but have an ESP32 project I'd like to wrap up. Helping broaden your framework for ESP32 usage would be nice to wrap up too.

[maakbaas]

@aaronse: The ESP32 branch will remain frozen, but most likely the master as well for some time.

If some bugfixes need to happen in master I will deal with this only when ESP32 branch will merge back to master.

Appreciate your support on this!

[JoergTiedemann]

Hi aaronse and maakbas
are there some news about time shedule when ESP32 support is available and pull into main branch is done ?

[aaronse]

Hello @JoergTiedemann, hoping to make progress this weekend.

[aaronse]

Quick update... Got COVID shortly after sending previous message. My brain is finally executing reasonably enough to make some edits to https://github.com/aaronse/esp8266-iot-framework/tree/esp32 and prepare to send a pull request. Cleaning up a few things first...

[aaronse]

Submitted #156 pull request.

See pull request description/comments to understand which subfeatures are implemented, or still pending.

[JoergTiedemann]

Hi @maakbaas
What do you think, when will es esp32 branch merged into the master branch so a new releas for Esp32 will be available?

[maakbaas]

@JoergTiedemann I can't give a timeline for this. The work from aaronse is already merged into the ESP32 branch, so you can start using that one in the meantime. Feel free to also take on part of the missing functionality!

[aaronse]

Submitted info for how to build, and couple of example projects for different boards to https://github.com/aaronse/esp-iot-framework-examples

Thanks to relatively recent Arduino and PlatformIO updates, am finally able to run ESP32 builds of this framework on the tiny QT PY ESP32-S2 boards, very cool!

[aaronse]

Hi @aaronse, starting make some test with ESP32 versione, I can compile e portal work. I have issue with Time Sync. Calling timeSync.isSynced() will return false and do not sync time against ntp.

Do you have any suggestion?

Marco

Hello @marcolino7 I recently updated code in the ESP32 pull request with edits to support setting/querying time (I didn't do much, am just calling into library method that does the real work). Also updated my hello world example showing usage https://github.com/aaronse/esp-iot-framework-examples/blob/main/helloWorld/src/helloWorld.cpp hope that helps!

Note, for some reason I don't understand yet, I had to edit my example code to explicitly set NTP server1 fqdn to pool.ntp.org. This framework's code has default of 0.pool.ntp.org fqdn which does not work from my location (Seattle, USA). At some point I'll setup an NTP service on my LAN and reference that.

[maakbaas]

@aaronse: how stable does this run for you? I am trying on different variants of ESP32 boards I have available here, but resetting basically all the time on startup or after WiFi connection. I did change the order such that the program starts with the WiFi manager, per your suggestion.

[aaronse]

Hello @maakbaas, what board(s) are you using? Vaguely recall observing watchdog resets if incorrect sequence used to setup/initialize WiFi. Have you already seen https://github.com/aaronse/esp-iot-framework-examples#firmware-crashes-on-startup ?

Haven't touched this project in a while, and haven't been closely watching 'uptime' for my devices. Would be nice to Web UI expose uptime, and rolling trace buffer even...

Some ESP32 devices I've been using...

8MB Flash, HelTec ESP32 OLED 0.96 Inch
https://www.amazon.com/gp/product/B076KJZ5QM

ESP32 (not sure how much mem)
https://www.amazon.com/gp/product/B0718T232Z

[maakbaas]

I have tried so far on a generic nodemcu-32s module, and the official dev boards for the esp32-s2 and esp32-c3. I did see your comment on that link, that is what i referred to above.

I will give it another shot later this week.

[aaronse]

I haven't tried on esp32-s3.

Also works on Adafruit QT PY esp32-s2. Blows my mind to see something so small and powerful, that's also accessible to Makers.

Will update sample with config for esp32-s2 later this week and update this thread. Was PITA to get config just right to build and deploy.

[maakbaas]

After spending some further time on this, I keep coming back to my original conclusion that I will not pursue porting this framework with all features to ESP32.

• There is too much divergence between the ESP8266 and ESP32 Arduino code. The WiFi manager would probably also benefit from a rewrite when targetting the ESP32, same as the file operations. I actually thought that the promise of Arduino was to provide an abstraction layer to write platform independent code, but this seems not to be the case. Personally this would probably push me to use esp-idf instead of Arduino for ESP32 projects.

• HTTPS fetch requests are basically 'hacked' into the ESP8266 arduino code, and can't be ported to ESP32, for which a different strategy needs to be used, for which there are probably also more elegant approaches within the espressif ecosystem.

• The ESP Async Web Server used causes some stability issues in my current testing. This library seems to be receiveing almost no updates anymore, and pull request that fix issues with new ESP32 Arduino library versions have not been released. I don't want to spend a significant effort on making this work if there is a good chance it will break again in the near future. If I would work on an ESP32 implementation I would probably look at something like Mongoose webserver. However, this one is not compatible with the ESP8266 Arduino code, since this also relies on a pretty outdated SDK from Espressif

Summarizing, I think by now the ESP8266 can be considered somewhat of a legacy device, and development seems slowing down. To do the ESP32 justice I think this framework would need a complete rewrite based on the current state of software development for that target. Maybe I will do this at one point in the future in a separate repo, and maybe not. But I feel that spending effort on making the existing codebase compatible with the ESP32 is not the way to go.

I think your work is awesome, and clearly it is working very well if you have been using it for a year already. Therefore, for now I will close this topic here, and would recommend anybody that wants to try this on the ESP32 to check it out here:
https://github.com/aaronse/esp8266-iot-framework/tree/esp32

[aaronse]

LOL, so I submitted this pull request after struggling to quickly grasp and get going with ESP-IDF.

Appreciate you sharing detailed rationale behind your thinking. Will take another look at ESP-IDF again, but expect I'll end up continuing to tweak my fork. Until if/when someone else is available to contribute, my fork will likely only support the subset of framework features that my home projects are using on my ESP8266 and ESP32 devices.

Will watch out for your future repos. Cheers!