Discrete / Unavailable sensor values in HA with deep sleep mode

[Lefuneste83]

Operating environment/Installation (Hass.io/Docker/pip/etc.):

Hassio on Docker

ESP (ESP32/ESP8266, Board/Sonoff):

ESP8266

Affected component:

https://esphome.io/components/mqtt.html#mqtt-last-will-birth

Description of problem:

Hi there. This may not be an issue apart from me being stupid...
I have been struggling to understand the bahaviour I have since switching from api mode to MQTT mode in order to accomodate for deep sleep mode.
I have this wireless weather station which has 10s uptime every 10min. It sends out various values for a bunch of sensors. While in api mode everything worked fine except that it would stop updating values after a while. The solution recommended by Otto was to switch to MQTT mode and indeed updates are much more reliable now.

The only remaining issue I have is that, whereas while in api mode I had my sensor values being displayed in HA lovelace as continuous values (in graphs) and last value received in entity cards, in MQTT mode, while everything updates fine every 10 min, I have discrete values in graphs (points displayed) and in entity cards, I have an "Unavailable" while in deep sleep (last value received is not displayed when in deep sleep).

This seems to be the purpose of the described procedure here as it is indeed the behaviour I am witnessing (ESPHome uses the last will testament and birth message feature of MQTT to achieve availability reporting for Home Assistant. If the node is not connected to MQTT, Home Assistant will show all its entities as unavailable (a feature 😉).) :
https://esphome.io/components/mqtt.html#mqtt-last-will-birth

I have spent several hours trying to understand how to get a continuous/last received value in HA lovelace, but have not suceeded to do so.

It seems to be the same issue as described here if I am not mistaking, but I am not sure.
#320

Sorry if this is a very obvious thing I am missing...

Problem-relevant YAML-configuration entries:

esphome:
  name: meteo
  platform: ESP8266
  board: d1_mini
  
  includes:
    - SI1145.h
  libraries:
    - "Adafruit SI1145 Library"

wifi:
  ssid: "myssid"
  password: "mypwd"
  fast_connect: on

  manual_ip:
    static_ip: 192.168.0.XX
    gateway: 192.168.0.XX
    subnet: 255.255.255.0
    dns1: 192.168.0.XX
    dns2: 8.8.8.8
    
# Enable logging
logger:

# Enable Home Assistant API
ota:
  password: 'mypwd'

mqtt:
  broker: 192.168.0.XX
  username: login
  password: "mypwd"

i2c:
  sda: D2
  scl: D1
  scan: True
  frequency: 200kHz

sensor:
  - platform: wifi_signal
    name: "Meteo WiFi Signal Sensor"
    update_interval: 1s

  - platform: bme280
    temperature:
      name: "BME280 Temperature"
      id: bme280_temperature
      oversampling: 16x
    pressure:
      name: "BME280 Pressure"
      id: bme280_pressure
      oversampling: 16x
    humidity:
      name: "BME280 Relative Humidity"
      id: bme280_humidity
    address: 0x76
    update_interval: 1s
  
  - platform: template
    name: "Absolute Humidity"
    lambda: |-
      const float mw = 18.01534;    // molar mass of water g/mol
      const float r = 8.31447215;   // Universal gas constant J/mol/K
      return (6.112 * powf(2.718281828, (17.67 * id(bme280_temperature).state) /
        (id(bme280_temperature).state + 243.5)) * id(bme280_humidity).state * mw) /
        ((273.15 + id(bme280_temperature).state) * r); // in grams/m^3
    accuracy_decimals: 2
    update_interval: 1s
  
  - platform: template
    name: "Equivalent sea level pressure"
    lambda: |-
      const float STANDARD_ALTITUDE = 260; // in meters, see note
      return id(bme280_pressure).state / powf(1 - ((0.0065 * STANDARD_ALTITUDE) /
        (id(bme280_temperature).state + (0.0065 * STANDARD_ALTITUDE) + 273.15)), 5.257); // in hPa
    update_interval: 1s

  - platform: custom
    lambda: |-
      auto UV_sensor = new SI1145_sensor();
      App.register_component(UV_sensor);
      return {UV_sensor->visible_sensor, UV_sensor->ir_sensor, UV_sensor->uvindex_sensor};

    sensors:
    - name: "Visible Light"
      unit_of_measurement: lux
    - name: "IR Light"
      unit_of_measurement: lux
    - name: "UV Index"
      unit_of_measurement: uvindex
      accuracy_decimals: 2
      
  - platform: tsl2561
    name: "TSL2561 Ambient Light"
    address: 0x39
    update_interval: 1s
      
deep_sleep:
  run_duration: 10s
  sleep_duration: 10min

Traceback (if applicable):

Additional information and things you've tried:

[OttoWinter]

Yeah, probably #320

Could you do two things:

• Check HA logs, if it's Cannot disable MQTT availability reporting - causes HASS schema validation error #320 then there will be errors in the HA logs
• Try with the latest beta version (1.13.0b3 I think); the patch for Cannot disable MQTT availability reporting - causes HASS schema validation error #320 is included in there.

[Lefuneste83]

Thanks again Otto for the great help. Yep indeed I see a similar thing happening.
I have done the following :

Add the following entry and recompiled my yaml
mqtt:
broker: 192.168.0.XX
username: login
password: "mypwd"
birth_message:
will_message:

See these logs in HA

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/homeassistant/components/mqtt/sensor.py", line 112, in discovery_update
    config = PLATFORM_SCHEMA(discovery_payload)
  File "/usr/local/lib/python3.7/site-packages/homeassistant/helpers/config_validation.py", line 677, in __call__
    validated = super().__call__(data)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 267, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 589, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 427, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: MQTT topic name/filter must not be empty. for dictionary value @ data['availability_topic']
2019-05-29 11:11:18 ERROR (MainThread) [homeassistant.core] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/homeassistant/helpers/config_validation.py", line 665, in __call__
    return super().__call__(data)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 267, in __call__
    return self._compiled([], data)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 589, in validate_dict
    return base_validate(path, iteritems(data), out)
  File "/usr/local/lib/python3.7/site-packages/voluptuous/schema_builder.py", line 427, in validate_mapping
    raise er.MultipleInvalid(errors)
voluptuous.error.MultipleInvalid: MQTT topic name/filter must not be empty. for dictionary value @ data['availability_topic']

I will try installing the patch now.

Where is located esphome/components/mqtt/mqtt_component.cpp in hassio ? Can't find it inside my docker instance. Is it inside the underlying OS ? I will install the whole Beta release for the moment.

[Lefuneste83]

Ok I have installed the beta.
Some good and some bad.

1/ For some strange reasons I have this weird regression

• platform: tsl2561

gets the validation error :

INFO Reading configuration...
Failed config

sensor.tsl2561: [source /config/esphome/meteo.yaml:48]
 

Don't know what '402' means as it has no time *unit*! Did you mean '402s'?.
 platform: tsl2561
 name: TSL2561 Ambient Light
 address: 57
 update_interval: 1s

The actual entry in yaml file is

  - platform: tsl2561
    name: "TSL2561 Ambient Light"
    address: 0x39
    update_interval: 1s

2/ The two template sensors don't get reported in HA anymore as sensors as before

  - platform: template
    name: "Absolute Humidity"
    lambda: |-
      const float mw = 18.01534;    // molar mass of water g/mol
      const float r = 8.31447215;   // Universal gas constant J/mol/K
      return (6.112 * powf(2.718281828, (17.67 * id(bme280_temperature).state) /
        (id(bme280_temperature).state + 243.5)) * id(bme280_humidity).state * mw) /
        ((273.15 + id(bme280_temperature).state) * r); // in grams/m^3
    accuracy_decimals: 2
    update_interval: 1s
  
  - platform: template
    name: "Equivalent sea level pressure"
    lambda: |-
      const float STANDARD_ALTITUDE = 260; // in meters, see note
      return id(bme280_pressure).state / powf(1 - ((0.0065 * STANDARD_ALTITUDE) /
        (id(bme280_temperature).state + (0.0065 * STANDARD_ALTITUDE) + 273.15)), 5.257); // in hPa
    update_interval: 1s

Should I declare a "sensors:" entry for these two now ?

3/ Apart from these two regressions, I can confirm that now I have a proper continuous reporting of values, once I add the birth and will empty entries in my mqtt tag in the yaml file, and no error logs in HA about it anymore.

[OttoWinter]

Thanks for testing! And for reporting issues in the beta!

1. Thanks! yes that's an issue. Will be fixed with esphome/esphome@3ca5e09

2. Haven't seen that before. I also tested with that config and MQTT now (once with default last will+birth, once with disabled availability) and it worked correctly. Could you provide some more information for that? Do you see anything in the HA logs? I'm not sure what you mean with Should I declare a "sensors:" entry for these two now ?

3. Perfect!

[Lefuneste83]

Yeah Ok I made myself a fool again... I deployed the Beta as I thought it was the version carrying the patch... I realized afterwards that the Dev branch did carry the patch as well. Once I woke up my brain and deployed the Dev Branch the second issue disappeared so do not worry about it... Only still have the point 1 with the TSL sensor being shout at...

[Lefuneste83]

Well this one was an easy one. I've just checked the content of the patch. I just forced the integration_time tag and "Voila" ! I will update my install as I've seen that you have already commited the change for the default value to 402ms. Everything is running smootly now. This is so great ! Thanks !