Device keeps disconnecting from Home Assistant if too many entities

[trapezz]

The problem

I don't know if there is an official limit of the entities in one ESPHome device, but it seems that after a certain number of entities the device struggles to connect to Home Assistant at all.
The same device works flawlessly if I remove devices and leaving only 20 of them. (didn't test the exact threshold)

After the device boots up, I keep getting connect/disconnect log:

[18:55:25][D][api.connection:158]: Home Assistant 2021.12.10 (192.168.100.99) requested disconnected
[18:55:25][D][api:098]: Accepted 192.168.100.99
[18:55:25][D][api.connection:779]: Home Assistant 2021.12.10 (192.168.100.99): Connected successfully
[18:55:30][D][api.connection:158]: Home Assistant 2021.12.10 (192.168.100.99) requested disconnected
[18:55:30][D][api:098]: Accepted 192.168.100.99
[18:55:30][D][api.connection:779]: Home Assistant 2021.12.10 (192.168.100.99): Connected successfully
[18:55:35][D][api.connection:158]: Home Assistant 2021.12.10 (192.168.100.99) requested disconnected
[18:55:35][D][api:098]: Accepted 192.168.100.99

The defice flash is not even half full:

Linking /data/shc-mcn2-t2/.pioenvs/shc-mcn2-t2/firmware.elf
RAM:   [=         ]  13.0% (used 42472 bytes from 327680 bytes)
Flash: [=====     ]  48.3% (used 886458 bytes from 1835008 bytes)
Building /data/shc-mcn2-t2/.pioenvs/shc-mcn2-t2/firmware.bin

Which version of ESPHome has the issue?

Tested on: 2022.1.2 and 2022.1.3

What type of installation are you using?

Home Assistant Add-on

Which version of Home Assistant has the issue?

Tested on: 2022.2.2 and 2021.12.10

What platform are you using?

ESP32

Board

ESP32 MH-ET LIVE DEVKIT

Component causing the issue

Not sure if it is connected to platform at all or only to number of entities. Using only Switch and Binary_Sensor

Example YAML snippet

Submitted the whole Yaml example in the Additional info section.

Anything in the logs that might be useful for us?

[18:55:25][D][api.connection:158]: Home Assistant 2021.12.10 (192.168.100.99) requested disconnected
[18:55:25][D][api:098]: Accepted 192.168.100.99
[18:55:25][D][api.connection:779]: Home Assistant 2021.12.10 (192.168.100.99): Connected successfully
[18:55:30][D][api.connection:158]: Home Assistant 2021.12.10 (192.168.100.99) requested disconnected
[18:55:30][D][api:098]: Accepted 192.168.100.99
[18:55:30][D][api.connection:779]: Home Assistant 2021.12.10 (192.168.100.99): Connected successfully
[18:55:35][D][api.connection:158]: Home Assistant 2021.12.10 (192.168.100.99) requested disconnected
[18:55:35][D][api:098]: Accepted 192.168.100.99

Additional information

The whole example configuration yaml file:
shc_mcn2_te2.txt

[trapezz]

I performed some tests to find out the limits.

Configuration:
I have one I2C bus with 6xPCF8575 port expanders (16 ports each) to add multiple inputs/outputs to the device. I've tryed different configurations of INPUTS (binary_sensor) or OUTPUTS (switch), but it seems that it does not matter. The results are allways the same.

• Up to 74 entities the system works fine, no errors.
• Between 75 and 77 it seems to work with some connect / disconnect errors in the log
• 78 and above it does not connect at all, the log is full of connect / disconnect errors as shown in the first post

Perhaps with so many devices the "loop" becomes too slow to handle the communications in parallel?

[ssieb]

Can you get serial logs? You might find out you're running out of memory or stack.

[trapezz]

I can not see much difference in the logs: Serial.log

[ssieb]

Ok, so it's not crashing.
[21:55:43][D][api.connection:158]: Home Assistant 2022.2.2 (192.168.100.99) requested disconnected
That means it's the HA side doing the disconnecting. Are there any logs from there?

[trapezz]

Yes there are:

2022-02-06 21:55:38 WARNING (MainThread) [homeassistant.components.esphome] Error getting initial data for 192.168.100.160: Timeout waiting for response for <class 'api_pb2.ListEntitiesRequest'>
2022-02-06 21:55:43 WARNING (MainThread) [homeassistant.components.esphome] Error getting initial data for 192.168.100.160: Timeout waiting for response for <class 'api_pb2.ListEntitiesRequest'>
2022-02-06 21:55:49 WARNING (MainThread) [homeassistant.components.esphome] Error getting initial data for 192.168.100.160: Timeout waiting for response for <class 'api_pb2.ListEntitiesRequest'>

2022-02-06 22:06:19 WARNING (MainThread) [aioesphomeapi.reconnect_logic] Can't connect to ESPHome API for shc-mcn2-t2 @ 192.168.100.160: Hello timed out

obviously the data is not arriving to HA...?

[ssieb]

I wonder if the timeout on the HA side is too short or if the ESP is getting stuck sending it.

[trapezz]

Is there a way to easily change the timeouts, so that I try and test?

[ssieb]

I don't know. You would have to look at the https://github.com/esphome/aioesphomeapi code. In the README there's example code you could test with.

[probot-esphome]

Hey there @OttoWinter, mind taking a look at this issue as it has been labeled with an integration (api) you are listed as a code owner for? Thanks!
_{^{(message by CodeOwnersMention)}}

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[chrismazanec]

I have exactly the same problem, although with much fewer devices

https://gist.github.com/chrismazanec/f296009e15223ef355656077deadb4f5

edit; found the issue, in my case this was caused by ssd1306 display, default slow i2c clock speed (50kHz), and fast (0.25s) refresh rate of the display

increasing i2c clock speed to 100kHz and decreasing display refresh rate time to 0.5s solved the issue entirely

#3325