Aligning with ESP-IDF: Preparing for Our Future

[edwardtfn]

Hello Community,

As we approach the March release and our transition to ESP-IDF, understanding both the reasons behind this change and how to prepare is crucial. This message outlines the benefits of ESP-IDF, the future of Arduino in our project, and essential steps for a smooth update.

Why Choose ESP-IDF?

1. Strategic Alignment: Adopting ESP-IDF aligns us with the core development direction of both Espressif and ESPHome, opening doors to innovation and advanced features.

2. Future-Ready Development: This shift positions us to rapidly integrate new technologies and improvements inherent to ESP-IDF.

3. Simplifying the Framework Stack:

   • Removing a Layer: Transitioning to ESP-IDF eliminates the intermediary Arduino layer, reducing complexity and potential issues from additional layers.
   • Direct Hardware Access and Efficiency: With one less layer, ESPHome can more directly interact with the ESP hardware, improving performance, memory usage, and hardware feature utilization.

Considering Arduino?

Continued Support with Practical Considerations:

1. Customization and Specific Use Cases: Some projects may still require the Arduino framework.
2. OTA Update Limitations: If serial flash isn't feasible, staying with Arduino is safer. Switching frameworks via OTA can lead to errors and potentially render the device unbootable until a flash via serial is performed.
3. New Feature Availability: Note that some future features may be exclusive to ESP-IDF.

Determining Your Current Framework:

The default framework is currently Arduino. However, it's worth noting that all our example/suggested code since the last few months have been indicating the esp-idf enforcement, so new users may already be on the new framework.
In short, unless you've explicitly set esp-idf in your code, you're using Arduino.

Explicitly Staying on Arduino:

Upcoming releases will default to ESP-IDF. To remain on Arduino, add this to your panel's YAML:

# Custom Arduino framework enforcement 
esp32:
  framework:
    type: arduino

Add this in the customizations area or at the file's end. You can make this change now; no need to wait until March.

Early Transition Encouraged

Those who can are encouraged to start transitioning to ESP-IDF now, to take full advantage of its capabilities.
In order to enforce the new framework, add this to your code:

# Custom ESP-IDF framework enforcement 
esp32:
  framework:
    type: esp-idf

Important

Remember, migrating between frameworks requires flashing your panel via USB to Serial/TTL adapter.

Community Support on GitHub

We encourage you to actively use this discussion thread for any support or questions you have during our transition to ESP-IDF.
This is a collaborative space where we can all assist each other, share insights, and navigate any challenges together.
Your contributions, experiences, and queries are vital in making this a smooth process for everyone.
Let's support each other through this exciting change!

In Summary

This transition underlines our commitment to delivering a robust and feature-rich ESPHome experience on your NSPanel. Your involvement is invaluable as we embark on this exciting phase.

Warm regards,

Edward

[edwardtfn]

Placeholder for a FAQ.

[plasterer41]

Hi Edward, is there a step by step guide to do this?

many thanks.
Irwin

[edwardtfn]

Well, basically you have to add the framework info to your yaml and then flash your device via serial as you have done the first time you installed a custom firmware in your panel.
We do have a guide for installing it for the first time, so it should be almost the same: https://github.com/Blackymas/NSPanel_HA_Blueprint/blob/main/docs/install.md#flashing-the-device-for-the-first-time

[plasterer41]

thank you

[DozoG]

Decided to spend a little extra time for this version change and make the "early" change to the ESP-IDF framework.
Worked right away on both my NS-Panels!

What maybe good to know is that:

• The first device you update takes a little extra time because ESP-Home will download files for the framework.
• There is nothing else you need to do. If you keep the same yaml file and just add the "framework enforcement" code as described, then download the bin file and flash it onto the Panel (as if it is the first time)...
  Then Home Assistant will remember the device and apply the blueprint as if nothing changed.
  After you power on the panel, you will see the confirmation on the startup screen (when the ESPHome version loads), that you are in fact on ESP-IDF

THANK YOU Edward.

[edwardtfn]

Thank you for sharing your experience. 😃

[edwardtfn]

I think it worth mentioning also that you don't have to fully remove your panel from the wall, as you can detach only the front panel and that contains all parts that need to be flashed. 😉

[DozoG]

True that! ... And now you have mentioned it :-)

[vshijiav]

After flash the firmware, the nspanel does not connect to the WIFI at all, Is there any location I can check log or debug? on ESP home wen it says flash complete. the panel now boot into ewelink frontend and does not have any connections at all.

[edwardtfn]

Let's take this one as a bug:

• Panel does not connect to the WIFI #1842

[smarthomefamilyverrips]

I not totally understand, if you switch to ESP-IDF you need to do every version update by serial controller or only the first version update (4.3.0) and after that the updates are done again by OTA as usual?

[edwardtfn]

Only when you change frameworks, the first time should be via serial. So if you've been using esp-idf before, you can continue with OTA as usual. If you have been using arduino, then you change to esp-idf using serial once, then after that you always use OTA.

[smarthomefamilyverrips]

I never did specify esp-idf in customization, so if I understand correctly that means i am still on arduino.

If I will add the lines of code for arduino framework to customization section of my YAML then I can just update to V4.3 using OTA .... I understand this correctly?

[edwardtfn]

I never did specify esp-idf in customization, so if I understand correctly that means i am still on arduino.

That's probably right. Most likely you have been using Arduino. New implementations from Nov/23 to now are likely to have the esp-idf explicitly set in the yaml, but if that is not your case, you are probably with arduino.

If I will add the lines of code for arduino framework to customization section of my YAML then I can just update to V4.3 using OTA .... I understand this correctly?

That's correct. If you explicitly set to use arduino you can use ota to update from a previous version, but when you have a chance please change to esp-idf via serial as we have now migrated almost all of our tests to esp-idf.
I still believe Arduino will be supported for a wile, but I don't wanna restrict new features when something is not supported by arduino (or in arduino doesn't have enough memory available).

[smarthomefamilyverrips]

I will try to switch to esp-idf asap but at this moment have to many things on my hands to take of my panel from the wall andndo manual update over serial 🫣

When I come to it I can just install same version again over serial, for example, I now update to 4.3 using arduino, if I find time to switch to esp-idf I can just do this with 4.3 again, there is no need to wait for next version update?

[edwardtfn]

That's right you can do that at any time. No need to wait for a new version.

[jeroenvanadrichem]

Bought a new usb for this and successfully flashed my panel. (The first time I flashed I borrowed one) I thought that the 2 light had something to do with my connections so I made sure the both were lightened. 🫣 Forgot that my pins were to long so that's was my obstacle because I got an error every time.
Thanks guys for all the work. Looking forward to all the upcoming updates for this platform.

Only tip I've got is to make a better manual how to update and flash. There's pages with info what is great. For someone who's native language is not English is better to have an short and clear step by step manual.

[edwardtfn]

Have you followed this guide?

[jeroenvanadrichem]

Yeah I did. BTW, there was an option somewhere at step 3 where I had to chose between a modern or I thought something like a native way to download the file. Couldn't find it in the manual. I've chosen the modern and that worked

[mtwebit]

Esphome running on 32-bit RPI4 is unable to compile the code with ESP-IDF. (I got "liblto_plugin.so: cannot open shared object file" error.) See #3076 and #5510.