Blacklist / Remove Entities #891
Comments
|
I've created a specific GH issue to look into optimizing the memory (#869) as it's not clear to me what the thresholds and lower limits are for Heap Memory and Max Alloc block before it causes EMS-ESP to run out of RAM and crash. I'm writing some code to simulate complex setups (like yours) which loads multiple device and entities to see where the memory goes. I have a hunch the problem is in the Max Alloc which is the largest amount of continuously aligned memory blocks which can be used to store a buffer. Often this becomes fragmented and gets smaller and smaller. We use large buffers for MQTT and the Web. If we solve that I think we should be fine. Having all the entities loaded is important to be able to enable/disable them from the customization screen. I don't think they take too much internal memory as the long string names are stored in Flash memory. If the results of the memory investigation gets us nowhere we can implement the black-list as you suggested. The are other alternatives. One is to switch to using the ESP32-S2 which has an extra 2MB PSRAM and costs 2 EUR. There is still some coding to be done to support using this extra RAM but we're close. |
|
I also don't hink the registering is to much memory, but i've added a poor mans implementation to test. |
Out of that I got the impression that there is a relationship between the number of entities and the free mem / max alloc. @MichaelDvP I'm happy to try that poor mans implementation AFTER the next time my device is restarted. For now I think we want to see if the function I disabled was enough to prevent the restarting. |
|
Summary:
Question: After installing EMS-ESP-3_5_0-dev_15-ESP32 (latest from emsesp) I'm having 99 / 39 (185 entities) Test results: Before installing EMS-ESP-3_5_0-dev_15-ESP32 from MichaelDvP After installing Marked all of RC300 as exclude from MQTT and hide from dashboard After restart Replaced "8 by "0 and loaded emsesp_customizations.json again ** Set all to visible again for RC300 ** Downloaded EMS-ESP-3_5_0-dev_15-ESP32.bin from After restart Double checked customization, removed 2 favorites |
|
Do you count the entities from dashboard (registered with value) or from customization (registered)?
I don't know, there are a lot of changes: library updates (arduinojson), new device-types (heatsource), a lot of new settings, etc.
These are entities where you have set custom min/max values. |
From dashboard.
I was able to reproduce this issue after a factory reset. Raised #895 for this. What are we going to do with this issue? Close it or keep it as an enhancement for reducing memory? |
|
Keep it until we know what to do with the memory. The heap can be a problem but still 50KB is a lot left. My bet is still on the max allocation size is the root cause. |
|
I've tested to read the removed entities from customization file and show with trash-can. |
|
That looks promising, I tried your latest build but that one doesn't contain this option yet. Note: in your screenshot, below the options, the "hide from dashboard" disappeared. |
|
It was a first attempt to show the manual removed shortnames. For setting in web and storing it needs some more code. |
|
Sure no problem. For now this change helps to free up some of the heap, but if #869 is working then maybe this whole change is not required anymore. |
|
@HansRemmerswaal I've found the way to edit the blacklist in web customizations. |
|
When I blacklist all the entities of one device then the device doesn't show up anymore in the customization. So you can't undo the blacklist (other than editing the file manually). Not sure how difficult it will be to solve this, quick fix could be that you are not allowed to blacklist all entities but that at least one entity should remain. See RC100H in below example. When you want to blacklist from multiple devices then you need to save and restart in between. That's ok to me, as this blacklisting is normally something you do once and most likely only when you have problems with a restarting EMS-ESP due to memory... Or to eliminate entities which are not relevant for your configuration. But for that we also have the hide option. For the rest is working very nice!
|
Good point, should work now. |
|
while you boys are playing in the Customization screen, ever get this problem? #911 |
|
Only a few times after the customization was introduced. First i was confused by the save-button in setting dialog. For me i renamed it to "update" and each time i see the update-button i remember to click save. (But this only works for me). |
|
Installed the lastest version and it's working nice! Thanks a lot. |
When it's merged into the dev branch we can close it, otherwise we'll loose track of the changes Or decide just to roll out v3.5 and stick this into v3.6 (my preference) |
|
tests have been added to 3.5.0-dev-16 |
|
@proddy, should we add the feature to set/reset in webui ti v3.5? It's ~1k flash, most for the extra webui-icon. |
|
I suggested this improvement to prevent the automatic restart each time of the EMS-ESP due to not enough free heap. Since I disabled all standard options, including the fixed ethernet connection, my device is not restarting anymore. The solutions works and frees up enough memory for users who can not disable all options. But once #869 has freed up enough heap then I think this #891 might become obsolete. |
sure, if it fits let's add it. |
You have done a great job in adding many entities to the EMS-ESP. In my case I see 125 for the Boiler and 53 for the Thermostat. But due to all these entities the memory of my BBQKEES module is getting full and my system started to restart due to this, see #859. I have disabled as much as possible all the functionality to free up memory and now it's running stable again for a few days. Soon you will add a few new entities and my module will start restarting again.
Part of these entities are useless for my configuration, I have full electric so all the ones related to gas are irrelevant.
Part of these entities will never change or are less relevant, e.g. I don't need the current date/time of my thermostat.
Luckily you have thought about that and added the option to hide these. Unfortunatly hiding these will not reduce the memory.
Proposed solution
In the ideal world we would have in the settings the option to indicate the kind of configuration you have (hybrid / full electric / ...) and automatically all the irrelevant entities are excluded and not part of the memory. But I doubt that is a feasible solution as someone needs to categorize all the entities of all the devices.
A more workable solution could be a kind of black list. From the customization tab you can blacklist an entity which will remove it from the list of entities. A new blacklist tab will show all the entities that are blacklisted, from there it can be deleted from the blacklist. During the receiving of the telegrams the blacklist is checked and blacklisted entities are ignored. I think it's acceptable that the user needs to restart the EMS-ESP after blacklisting entities to free up the already occupied memory.
My expectation is that I can eliminate a part of the entities and with that free up memory. That allows me to enable some disabled functionality again, and will make it possible for you to add new entities again without worrying about the available memory.
The text was updated successfully, but these errors were encountered: