emontx: firmware flashing for the emonTx6 / emonPi3 over UART (design question before a PR)

[Frédéric Metrich (FredM67)]

Component name

emontx

Link to component documentation on our website

https://esphome.io/components/sensor/emontx/

Describe the enhancement

I would like your opinion on the design before I open a pull request.

Background

The emonTx6 and emonPi3 from OpenEnergyMonitor are standalone energy monitors built around a
SAMD21 microcontroller, with their own firmware (emon32-fw).
An ESP32 add-on board (emonWiFi) talks to them over UART. The emontx component, already in
ESPHome, reads their data and sends them commands. It also supports older models (emonTx4,
emonTx5) that run other firmware.

Each emonWiFi is dedicated to exactly one emonTx or emonPi, and one Home Assistant installation can
have several of them.

Firmware updates of the SAMD21 normally need a USB cable. With the SAM-BA bootloader over UART
(installed once by USB), the ESP32 can flash it instead. This works today in external components,
and a Home Assistant integration (HACS, ha-emon-config)
gives users a panel to configure the device and update its firmware.

What exists today

Two components, kept outside ESPHome (about 920 lines). Both have been tested on hardware,
including a full flash.

emontx_updater (ESP32 only):

• Registers a flash_emontx6(url) service with CustomAPIDevice::register_service
• Downloads the image with esp_http_client, following GitHub's redirects by hand
• Stores it at the end of the spare OTA partition, not in RAM: a 120 KB contiguous buffer is
  often not available on a fragmented heap. The current app image in that partition stays
  intact, and the OTA update lock (esphome#20588) stops an ESPHome OTA from overwriting it
• Checks the image (size, ARM vector table) before touching the device
• Pauses emontx, sends the firmware's "enter bootloader" commands, then writes the flash with
  SAM-BA word writes and NVM commands (as BOSSA does), in a FreeRTOS task. A flash takes about
  45 s
• Reports progress by sending esphome.emontx_flash_status events with
  fire_homeassistant_event
• Options: bootloader_timeout, verify_ssl, dry_run

emontx_ha_bridge (all models):

• Registers a send_command(command) service
• Sends every line received from the device as an esphome.emontx_raw event, and every JSON
  frame as esphome.emontx_json

Setup for users today:

emontx_updater:
emontx_ha_bridge:

plus http_request:, and custom_services: true and homeassistant_services: true under api:.

Requirements

• Same features: flashing the latest release, an older release, any firmware URL (custom
  builds), and the switch back to the USB bootloader; progress and status messages; the checks
  before entering the bootloader; the OTA lock; sending commands and receiving every line for the
  configuration panel.
• Simple setup: one short block per feature in the device config, as today. Many users of these
  devices are not technical, so they should not have to copy YAML.
• Several devices in one Home Assistant, and older models without flashing.

The Home Assistant panel can be rewritten as needed.

What we think may not fit upstream

• CustomAPIDevice: register_service and fire_homeassistant_event seem meant for external
  components and lambdas. No in-tree component uses them, and they require custom_services and
  homeassistant_services under api:.
• Fixed names in C++: the service and event names are fixed, and every event builds a
  std::map<std::string, std::string> on the heap.
• The download bypasses http_request and uses esp_http_client directly. It also blocks the
  main loop for about 2.5 s.
• Tests: only build tests, no unit tests for the SAM-BA sequence.

Options for flashing

A. Keep the service and the events (CustomAPIDevice)

Setup stays exactly as today: emontx_updater:. The panel does not change. The code would be
cleaned up (no heap-built event data where avoidable, download in the flash task).

B. Standard entities

An update platform of emontx, structured like esp32_hosted/update:

update:
  - platform: emontx
    name: Firmware

• Update entity: installed version (read from the device with the v command, already in
  emontx), latest release (from GitHub), Install button and progress bar, all native in Home
  Assistant.
• Text entity "Firmware URL": when set, Install flashes this URL instead of the latest
  release. This covers older releases, custom builds and the USB bootloader switch.
• Text sensor "Flash status": the status and error messages.

The panel would follow these entities instead of the events. Ideally the platform creates both
extra entities by default, so the lines above stay the whole setup. We found no in-tree component
that creates entities the user did not list, hence question 2 below.

Options for sending commands and receiving lines

The panel sends commands (l, k1 ...) and needs every line the device sends back. Reading the
configuration is a burst of about 50 lines.

A. Keep the service and the events (CustomAPIDevice)

As today: emontx_ha_bridge:, or the same feature as an option of emontx.

B. Entities

A text entity for the command and a text sensor for the last line. We see two problems: the
bursts would fill Home Assistant's history, and Home Assistant ignores a new state that is the same
as the previous one, so a repeated line would be lost.

C. A YAML package

The same service and events, built from api: actions:, emontx.send_command, the on_data
trigger and homeassistant.event. It only uses existing building blocks, but users would have to
add a packages: entry that points to a file in another repository, which we would like to avoid.

Questions

1. Is CustomAPIDevice acceptable in an in-tree component for this use? If yes, option A for
   both parts keeps everything as it is today.
2. If not, is option B acceptable for flashing? In particular, may the update platform create
   its "Firmware URL" and "Flash status" entities by default?
3. What would you accept for the command and line bridge?
4. Is it acceptable to stage the image in the spare OTA partition, and to flash from a FreeRTOS
   task (about 45 s)?
5. Should dry_run stay? It is mostly useful while testing a new setup.

In any case

• Download through http_request if it handles the GitHub redirects with buffers small enough for
  the ESP32-C3 (8 KB or more crashed its Wi-Fi driver during the download); otherwise keep
  esp_http_client and explain why. Run the download in the flash task.
• Add gtests for the SAM-BA command sequence, with a mocked UART.

Next steps

The working version stays on its current branch. Once the design is agreed, the work is done on a
new branch based on dev, tested on hardware with the updated Home Assistant panel, and submitted
as one pull request per component.

Use cases

updating without a USB cable when the device is in a meter cabinet; installing a new release (for example v1.2.0 with automatic calibration) or going back to an older one; custom builds; switching back to the USB bootloader; configuring the device from the Home Assistant panel

Anything else?

OpenEnergyMonitor docs,
ha-emon-config repo

[github-actions[bot]]

🏷️ I've automatically added the component: emontx label to help categorize this feature request.