ESPectre v3 on NSpanel-Easy - early experiences

[HACS-bank]

I am trialling 3.0.0-rc1 on NSpanel-Easy with Home Assistant ... here are my first notes based on deploying onto 10 NSpanel units, with climate control, iBeacon, bluetooth_proxy and BLE scanning also enabled (yes, I am deliberately pushing the envelope)

1. a few hours after I added 2.8.0 to all 10 panels, my router DHCP started to fail causing the home Wi-Fi to become unusable.

I disabled router DHCP and moved the service to the AdGuard app on my Home Assistant and all was well for a week or so

2. since upgrading to 3.0.0-rc1 on all panels, my router stops working after a day or two and has to be power-cycled to restore service

I theorize that the additional load caused by out-of-the-box ESPectre is causing trouble! I will try changing from ping to dns (which is handled by AdGuard) to see if it helps. it would be helpful if ESPectre gave a log entry to confirm what it is hammering - maybe it already does, I didn't search the logs yet

3. when I upgraded to 3.0.0-rc1 there was also an upgrade for NSpanel-Easy and so both upgraded by the ESPhome Device Manager in HA. When NSpanel-Easy is upgraded, it often needs to download a new TFT file for its display .... it does a version check, and if a new TFT is required, it schedules the download for 60s after reboot. It then shuts down any possible processes such as BLE scanning.

However the ESPectre process starts very early in the boot sequence and is not shut for the upload, which means that the TFT upload has barely enough RAM and runs very slowly, sometimes failing. it would be helpful if ESPectre waited for a bit before starting to give everything else some time, and preferably it should somehow delay start if the panel needs to upgrade itself (or unload - if the panel upgrades, it will reboot, to there is no need to worry about reloading)

4. on the current releases, the firmware for the NSpanel uses 98.2% of the partition ... soon I will need to decide what to remove :)

Here is my typical panel definition

substitutions:
  device_name: "bathroom-panel"
  friendly_name: "NSpanel Bathroom"
  wifi_ssid: !secret wifi_ssid
  wifi_password: !secret wifi_password
  ota_password: "*******"
  boot_sound: false
  upload_tft_automatically: true
  include_action_wake_up: true
  language: en
##### My customization - Start #####
#  # Enable fallback hotspot (captive portal) in case wifi connection fails
#  ap:
#    ssid: "Bathroom-Panel Fallback Hotspot"
#    password: "******"
# Removes captive portal - Pre-built and advanced only
captive_portal: !remove
# Removes embedded web server - A#dvanced only
web_server: !remove
# Bluetooth Proxy with smart scanning (only when HA is connected)
bluetooth_proxy:
esp32_ble_beacon:
  type: iBeacon
  uuid: fde3b150-2f64-43ba-aee9-867f75ee4a6f
  major: 1
  minor: 19
  min_interval: 500ms
  max_interval: 1000ms
esp32_ble_tracker:
  scan_parameters:
    continuous: false
api:
  encryption:
    key: !secret api_encryption
  # Reboot if HA is not connected for 15 minutes
  reboot_timeout: 15min
  on_client_connected:
    then:
      - esp32_ble_tracker.start_scan:
          continuous: true
  on_client_disconnected:
    then:
      - esp32_ble_tracker.stop_scan:
# ESPectre - WiFi CSI Motion Detection
external_components:
  - source:
      type: git
      url: https://github.com/francescopace/espectre
      ref: 3.0.0-rc1 # or main, or develop
      path: src/cpp/frontend/esphome/components
    components: [ espectre ]
espectre:
  id: espectre_csi
  detection_algorithm: lightweight # or high_accuracy
  direct_api: false
  motion_sensor:
    id: espectre_motion
    name: "ESPectre Motion"
    on_press:
      - script.execute:
          id: wakeup
          reset_timer: true
      - script.execute:
          id: refresh_relays
          relay_mask: 2
    on_release:
      then:
        - script.execute:
            id: refresh_relays
            relay_mask: 2
script:
  - id: !extend refresh_relays
    then:
      - lambda: |-
          // chip_relay2 is repurposed to mirror the ESPectre motion sensor
          // instead of relay 2's own state. This runs after the built-in
          // lambda, so it overrides whatever that one just wrote.
          if (!(relay_mask & 2)) return;
          const bool motion = espectre_motion->state;
          if (chip_states[CHIP_IDX_RELAY2].visible == motion) return;
          chip_states[CHIP_IDX_RELAY2].visible = motion;
          chip_render(CHIP_IDX_RELAY2);

##### My customization - End #####
packages:
  remote_package:
    url: https://github.com/edwardtfn/NSPanel-Easy
    ref: latest
    refresh: 300s
    files:
      - nspanel_esphome.yaml # Base package
      - esphome/nspanel_esphome_addon_climate_heat.yaml
      - esphome/nspanel_esphome_addon_display_light.yaml

[HACS-bank]

Example log entry which could add a bit more detail

[10:44:17.403][W][TrafficGen:535][traffic_gen]: ping send failed (errno=12, consecutive=1) [10:44:17.403][I][espectre.runtime:064]: [--------------------] | mvmt:0.000000 thr:0.662185 | CALIBRATING | csi:0/0 tx:89 occ:2% miss:0 excess:0 stale:0 ooo:0 | ch:4 rssi:-58 [10:44:18.226][W][TrafficGen:535][traffic_gen]: ping send failed (errno=12, consecutive=4) [10:44:18.226][I][espectre.runtime:064]: [--------------------] | mvmt:0.000000 thr:0.662185 | CALIBRATING | csi:0/0 tx:77 occ:2% miss:0 excess:0 stale:0 ooo:0 | ch:4 rssi:-58

[francescopace]

Hi @HACS-bank, I'm very sorry for your DHCP.

ESPectre’s internal traffic generator targets always the device’s default gateway, whatever its IP address is on your network.

destination.sin_addr.s_addr = manager->gateway_addr_;

This applies to both ping and DNS mode, so switching to DNS would still target your router unless AdGuard is also your default gateway. The startup log already reports the destination in the Traffic generator started (... gateway=...) message, although including it in error messages would help.

What ESPectre needs is incoming Wi-Fi traffic to extract CSI from received packets. Ping requests and DNS queries trigger replies, which provide that incoming traffic.

The bandwidth is fairly small: in ping mode, each request contains an 8-byte ICMP header with no extra payload. Including the IPv4 header, that is 28 bytes per request and reply. At 100 requests per second, this works out to approximately 5,600 bytes/second per device in both directions combined, excluding Wi-Fi overhead, acknowledgments, and retransmissions. Across 10 panels, however, that still means around 1,000 requests and 1,000 replies per second for the router to handle.

For your setup, I would try external traffic mode. It disables the internal generator and lets another machine supply the incoming traffic. Add this to your existing espectre: configuration:

  csi_traffic_mode: external
  csi_target_pps: 100

Then use tools/espectre_traffic_generator.py. Set TARGETS to your panels’ IP addresses, leave RATE = 100, and run:

python3 tools/espectre_traffic_generator.py start

The script sends UDP packets to port 5555, which the devices consume without sending an application reply. This removes the repeated ping/DNS exchanges with the gateway and reduces traffic on the devices’ Wi-Fi links. You can run it on a machine that can reach the panels, including a suitable Python environment on your Home Assistant host.

Multicast is also available through the default group 239.255.0.1, allowing one stream to serve multiple devices. The actual savings depend on how your access point handles multicast. The script uses a multicast TTL of 1, so start with the sender and panels on the same VLAN; ordinary routing between VLANs is insufficient.

This should reduce the router’s role in generating replies, and the traffic on your network.

[HACS-bank]

Suggestion: espectre should target the DNS server, not the default gateway, when DHCP mode is selected - this seems more intuitive

[francescopace]

Hi @HACS-bank, assuming you mean DNS mode: the DNS server advertised by DHCP is not necessarily local. It could be AdGuard, your router, or a public resolver such as Google’s 8.8.8.8. Automatically targeting it could send the generated traffic over the Internet.

The espectre version onmain already supports a custom destination for both ping and DNS through traffic_generator_target_ip. You can use it to point DNS traffic directly at your AdGuard server.

Change your external component ref to main:

external_components:
  - source:
      type: git
      url: https://github.com/francescopace/espectre
      ref: main
      path: src/cpp/frontend/esphome/components
    components: [espectre]
    refresh: 0s

Then add these settings to your existing espectre: block, keeping your other options and motion actions:

  csi_traffic_mode: internal
  traffic_generator_mode: dns
  traffic_generator_target_ip: "192.168.1.10" # Replace with your AdGuard server's IPv4 address

Rebuild and upload the firmware to apply the change. The same target setting also works with traffic_generator_mode: ping, provided the destination responds to ICMP echo requests.

If the target is omitted or empty, ESPectre still uses the default gateway; it does not automatically select the DNS server advertised by DHCP.

See the traffic configuration documentation.

[HACS-bank]

next experiment: removing all bluetooth, ibeacon and BLE takes the image down from 99% to 76%

[francescopace]

It helps also to increase sensing accuracy!

[francescopace]

Regarding the TFT upload, NSPanel-Easy already provides a useful integration point: its upload sequence calls and waits for the stop_all script before transferring the file. You can add the following entry alongside your existing refresh_relays customization under script::

  - id: !extend stop_all
    then:
      - lambda: |-
          if (system_flags.tft_upload_active) {
            id(espectre_csi).set_sensing_runtime(false);
          }

This uses ESPectre’s existing sensing control to stop the traffic generator and disable CSI capture before the upload.

The condition restricts it to TFT updates, since NSPanel-Easy uses stop_all elsewhere too.

That should be more reliable than a fixed delay: an ESPHome on_boot automation with delay only delays its subsequent actions, while other components continue initializing. Using the upload sequence keeps sensing paused for the transfer itself.

Pausing does not fully unload ESPectre or release all of its allocated memory, so this may reduce contention without completely resolving the RAM shortage. I verified the integration against the source, but have not tested it on an NSPanel yet.

[HACS-bank]

pausing during OTA firmware upload is also a good idea

[HACS-bank]

next experiment: reduce pps from 100 to 10 until a better router is available - family report needing to restart the router every 4 hours or so when working from home

[francescopace]

10 is too low, use an external generator instead, your router will thank you

[HACS-bank]

for widespread adoption on home assistant, an "app" (add-on) with the traffic generator would be a good thing; it could include a UI with multicast option, rate select with reasonable limits, etc

[francescopace]

Very cool! I completely agree. Feel free to open an issue and select “Feature request.”
I’ll add it to the roadmap, and someone from the community might even pick it up!

[HACS-bank]

  - id: !extend boot
    then:
      - lambda: |-
          id(espectre_csi).set_sensing_runtime(true);

  - id: !extend boot_early_routines
    then:
      - lambda: |-
          id(espectre_csi).set_sensing_runtime(false);```

[HACS-bank]

ota:
  platform: esphome
  encryption:

[HACS-bank]

First experiment with high_accuracy - complete example:

Total image size: 1385511 bytes (.bin may be padded larger)
RAM: [==== ] 39.7% (used 71744 bytes from 180736 bytes)
Flash: [======== ] 75.5% (used 1385511 bytes from 1835008 bytes)

substitutions:
  #nextion_update_url: http://192.168.1.5:8123/local/nspanel_eu.tft
  device_name: "bathroom-panel"
  friendly_name: "NSpanel Bathroom"
  wifi_ssid: !secret wifi_ssid
  wifi_password: !secret wifi_password
  boot_sound: false
  upload_tft_automatically: true
  include_action_wake_up: true
  language: en
##### My customization - Start #####
#  # Enable fallback hotspot (captive portal) in case wifi connection fails
#  ap:
#    ssid: "Bathroom-Panel Fallback Hotspot"
#    password: !secret wifi_password 
# Removes captive portal - Pre-built and advanced only
captive_portal: !remove
# Removes embedded web server - Advanced only
web_server: !remove
# Bluetooth Proxy with smart scanning (only when HA is connected)
#bluetooth_proxy:
#esp32_ble_beacon:
#  type: iBeacon
#  uuid: fde3b150-2f64-43ba-aee9-867f75ee4a6f
#  major: 1
#  minor: 19
#  min_interval: 500ms
#  max_interval: 1000ms
#esp32_ble_tracker:
#  scan_parameters:
#    continuous: false
ota:
  platform: esphome
  encryption:
api:
  encryption:
    key: !secret api_encryption
  # Reboot if HA is not connected for 15 minutes
  reboot_timeout: 15min
#  on_client_connected:
#    then:
#      - esp32_ble_tracker.start_scan:
#          continuous: true
#  on_client_disconnected:
#    then:
#     - esp32_ble_tracker.stop_scan:
# ESPectre - WiFi CSI Motion Detection
external_components:
  - source:
      type: git
      url: https://github.com/francescopace/espectre
      ref: 3.0.0-rc2 # or main, or develop
      path: src/cpp/frontend/esphome/components
    components: [ espectre ]
espectre:
  id: espectre_csi
  detection_algorithm: high_accuracy
  direct_api: false
  csi_target_pps: 100
  csi_traffic_mode: external
  traffic_generator_mode: ping
  traffic_generator_target_ip: 192.168.1.2 
  motion_sensor:
    id: espectre_motion
    name: "ESPectre Motion"
    on_press:
      then:
        - script.execute:
            id: wakeup
            reset_timer: true
        - script.execute:
            id: refresh_relays
            relay_mask: 2
    on_release:
      then:
        - script.execute:
            id: refresh_relays
            relay_mask: 2
script:
  - id: !extend boot
    then:
      - lambda: |-
          id(espectre_csi).set_sensing_runtime(true);
  - id: !extend boot_early_routines
    then:
      - lambda: |-
          id(espectre_csi).set_sensing_runtime(false);
  - id: !extend stop_all
    then:
      - lambda: |-
          if (system_flags.tft_upload_active) { id(espectre_csi).set_sensing_runtime(false); }
  - id: !extend refresh_relays
    then:
      - lambda: |-
          // chip_relay2 is repurposed to mirror the ESPectre motion sensor
          // instead of relay 2's own state. This runs after the built-in
          // lambda, so it overrides whatever that one just wrote.
          if (!(relay_mask & 2)) return;
          const bool motion = espectre_motion->state;
          if (chip_states[CHIP_IDX_RELAY2].visible == motion) return;
          chip_states[CHIP_IDX_RELAY2].visible = motion;
          chip_render(CHIP_IDX_RELAY2);

##### My customization - End #####
packages:
  remote_package:
    url: https://github.com/edwardtfn/NSPanel-Easy
    ref: latest
    refresh: 300s
    files:
      - nspanel_esphome.yaml # Base package
      - esphome/nspanel_esphome_addon_climate_heat.yaml
      - esphome/nspanel_esphome_addon_display_light.yaml

[HACS-bank]

[08:53:46.084][I][espectre.runtime:063]: [----------|---------] | mvmt:0.000000 thr:0.500000 | IDLE | tx:69.0 cb:73.0 accepted:73.0 hwerr:0.0 occ:38% | ch:7 rssi:-56
[08:53:46.084][I][espectre.runtime:063]: [----------|---------] | mvmt:0.000000 thr:0.500000 | IDLE | tx:31.0 cb:32.0 accepted:32.0 hwerr:0.0 occ:15% | ch:7 rssi:-59
[08:53:46.085][I][espectre.runtime:063]: [----------|---------] | mvmt:0.000000 thr:0.500000 | IDLE | tx:54.4 cb:58.4 accepted:57.4 hwerr:0.0 occ:22% | ch:7 rssi:-55

[HACS-bank]

@francescopace I think that this is a possible bug? - when booting or changing model, there may be a call to recalibrate missing

[francescopace]

Hi @HACS-bank ,high_accuracy uses a default threshold of 0.5. You can adjust it, and the chosen value is saved across reboots, so you do not need to set it again each time. Pressing Recalibrate simply restores this detector’s default threshold.

You may have changed the threshold manually at some point without noticing. Based on this, I do not see a bug.

However, I do see a separate issue with the Tx values (69.0, 31.0, and 54.4). They are too low; they should be closer to 100.
Since you are using an external traffic generator, what rate is configured in its script?

[HACS-bank]

Traffic generator is multicasting 100pps on a Linux workstation in my home office, which is connected via a fast ethernet (not gigabit) LAN with several hops, via a Gigabit LAN switch and router (where Home Assistant and the rest of the home is connected), and then finally a whole house 6 node mesh wifi network to the ESP devices. Some of the mesh is wired, some wireless, some areas still have poor signal 😢

I try to think of this as a good testing ground for real world challenges...

[francescopace]

I’m thinking of this not as a real-world scenario, but as a real-world war scenario 😂
Can I ask why you’re placing so much hope in traffic generated from devices?
As you know, UDP multicast is best-effort. I’d suggest running the script somewhere closer to the NS Panels. I'm working on creating an add-on of Home Assistant to facilitate this.

[HACS-bank]

All panels configured with

• external traffic
• no Bluetooth
• high accuracy model (I found that the lightweight model takes too long to calibrate - up to 12 hours sometimes)
• renamed entities in HA to have Motion in the name
• script extensions as above to pause ESPectre during NSpanel configuration, TFT upload, etc
• panel wakeup is handled locally

The experiment continues! I am no longer looking at v2 now that v3rc2 is available

[HACS-bank]

After changing some settings, the panel does not seem to respond any better ... but then I restarted the panel, and hey presto it works as desired.
It is possible that changing settings without a reboot of the panel has confused some earlier testing