Update Interval extremely Long

[rexprojects]

Hi,

I followed the steps by step install with docker and everything worked as far as I can see.

but I realized that the update Interval is extremely long. The CPU temp for example updates only once every few hours.

I have a DXP2800

I appreciate any help, thanks!

[Tom-Bom-badil]

tl;dr: No bug, just a misunderstanding what is displayed there. If it would update every 3 hours only, you wouldn't see the spikes between 8:00 and 10:00. Your cpu temperature is at 73°C since 3 hours now.

This is a common misunderstanding in HA, and has been discussed a number of times with the developers (see e.g. here):
If the value did not change, last_updated will not change - because it will always reflect the time the entity has last changed and been handled by the recorder. This is also documented in HA's official docs: Look up state.last_updated here.

The updates in this specific integration are as follows:

• normal sensors (aka configuration_endpoints) are updated every minute,
• quickly changing sensors (aka status_endpoints) are updated every 5 seconds.

That is how it's programmed - for ALL sensors.

How to test: If you start monitoring something that changes rather quickly (let's say overall_lan_download or overall_disk_read_rate), you will notice those change very often. There is no 'special treatment' for specific sensors like 'CPU temperature' - they are all read at the same time. So, if e.g. overall_lan_download is changing quickly, you can be sure that cpu_temperature is changing quickly as well, because they are both status_endpoints (you can look it up in the code, see link above).

/tom

[rexprojects]

Thank you for your fast response.

I could not see those rapid changes on the mentioned sensors either. They also had a strain line for several hours.

I realized now that it works as long as I have the UGREEN app on iOS open. As soon as I close it, the graphs do not update so frequently. You can see it at the straight line in between the frequent updates. That's where I closed the app.

[Tom-Bom-badil]

Thats weird - anything in the logs that gives a hint?

/tom

[rexprojects]

I found the following in the logs containing UGREEN at then end:

Logger: homeassistant
Quelle: components/sensor/init.py:668
Erstmals aufgetreten: 09:12:20 (8758 Vorkommnisse)
Zuletzt protokolliert: 21:22:12

Error doing job: Task exception was never retrieved (None)
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/sensor/init.py", line 666, in state
numerical_value = float(value) # type:ignore[arg-type]
ValueError: could not convert string to float: 'None'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 270, in _handle_refresh_interval
await self._async_refresh(log_failures=True, scheduled=True)
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 481, in _async_refresh
self.async_update_listeners()
~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 180, in async_update_listeners
update_callback()
~~~~~~~~~~~~~~~^^
File "/config/custom_components/ugreen/sensor.py", line 103, in _handle_coordinator_update
super()._handle_coordinator_update()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 561, in _handle_coordinator_update
self.async_write_ha_state()
~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1018, in async_write_ha_state
self._async_write_ha_state()
~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1140, in _async_write_ha_state
self.__async_calculate_state()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1080, in __async_calculate_state
state = self._stringify_state(available)
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1024, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/sensor/init.py", line 668, in state
raise ValueError(
...<5 lines>...
) from err
ValueError: Sensor sensor.ugreen_nas_cpu_fan_speed has device class 'None', state class 'None' unit 'rpm' and suggested precision 'None' thus indicating it has a numeric value; however, it has the non-numeric value: 'None' (<class 'str'>)

[rexprojects]

Another one:

Logger: homeassistant.components.sensor
Quelle: helpers/entity_platform.py:657
Integration: Sensor (Dokumentation, Probleme)
Erstmals aufgetreten: 09:12:15 (2 Vorkommnisse)
Zuletzt protokolliert: 15:05:22

Error adding entity sensor.ugreen_nas_cpu_fan_speed for domain sensor with platform ugreen
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/components/sensor/init.py", line 666, in state
numerical_value = float(value) # type:ignore[arg-type]
ValueError: could not convert string to float: 'None'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 657, in _async_add_entities
await self._async_add_entity(
entity, False, entity_registry, config_subentry_id
)
File "/usr/src/homeassistant/homeassistant/helpers/entity_platform.py", line 979, in _async_add_entity
await entity.add_to_platform_finish()
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1369, in add_to_platform_finish
self.async_write_ha_state()
~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1018, in async_write_ha_state
self._async_write_ha_state()
~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1140, in _async_write_ha_state
self.__async_calculate_state()
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1080, in __async_calculate_state
state = self._stringify_state(available)
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1024, in _stringify_state
if (state := self.state) is None:
^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/sensor/init.py", line 668, in state
raise ValueError(
...<5 lines>...
) from err
ValueError: Sensor sensor.ugreen_nas_cpu_fan_speed has device class 'None', state class 'None' unit 'rpm' and suggested precision 'None' thus indicating it has a numeric value; however, it has the non-numeric value: 'None' (<class 'str'>)

[Tom-Bom-badil]

I believe these two effects are not related to each other.

First the more obvious one:

The error indicating it has a numeric value; however, it has the non-numeric value: 'None' means that HA doesn't get a number here; just the string/text 'None'. It is surprising that exactly this one sensor is reported over and over again, as it's definition is the same like all other sensors.

To track down what happens, please do the following:

• at first: Restart HA and see if the error is still there,
• extract your api_token manually in the browser, as shown here,
• enter the following URL in your browser: http://NAS_IP:NAS_Port/ugreen/v1/taskmgr/stat/get_all?token=your_token,
• copy/paste the result to this web site (left window), click 'Make Pretty' button in the middle,
• check the tree: data --> overview --> cpu_fan and device_fan (this is what the integration gets 1:1 when it reads from the NAS - is there a number?)

Example call for my NAS: http://192.168.178.9:9999/ugreen/v1/taskmgr/stat/get_all?token=ABCDEF1234567890FEDCBA

Now regarding the slow updates:

I just have been able to verify this, but not yet sure what it means:

This graph 1:1 reflects the times when my PC was switched on in the last 24hrs. This PC is holding my second 'HA Develop' instance, accessing the NAS independently from my 'HA Production' VM, which is running 24/7 on the NAS itself. So it seems when a second system (another HA, or the mobile APP, or a browser logged in to UGOS - in this particular case my developer HA) is reaching out to the NAS, the production system is getting updates every 5s. Otherwise they are delayed; sometimes delayed a lot.

I need to further investigate, not yet sure what this means. Thanks for the good find, appreciated! :)

/tom

[Tom-Bom-badil]

Thanks - interesting, because that is an information I didn't have so far. Makes perfectly sense now. I was expecting every model to have at least a CPU fan and one chassis fan. So, it seems I need to adjust the coding for the fans then (at the moment it is already checked how many chassis fans are installed, as the larger models have two of them - but it is not checked if there is a CPU fan installed or not).

edit: nonsense below, please ignore

On the slow data updating: I found some weird results this morning in the data that are totally confusing me: Everything was properly updated every 5 seconds last night, even when the second HA instance (my developer PC) was switched off. That is contrary to my first findings yesterday evening.

And: Even the data that were 'missing' yesterday evening (the straight lines in my plot from 3:30-11:00 and from 20:15-22:00) seem to there now - the straight lines disappeared, instead it's showing normal data with 'ups' and 'downs' there now. I'll need to further investigate this tonight, that's pretty strange.

/tom

[Tom-Bom-badil]

[ edit: start ]

my mistake - I totally forgot that I made the screenshot on my developer system, which has of course it's own database.

🤦‍♂️ Then of course it absolutely makes sense that it is not recording any data while it's switched off (sic!). 🤦‍♂️
The live system (2nd sceenshot) shows that the same sensor was working fine there all the time.

So we are back at the beginning. So far I was able to reproduce some odd behaviour of data updating on a mobile device (iPad) only - when using HA in the browser, the same data are updated always after 5s and working as intended.

original nonsense below, please ignore:

[ edit end ]

Further to the above - I just made a post on the HA discord about this. This one leaves me completely clueless - why has the straight line disappeared overnight, and plausible data are now show there instead?

Will keep you posted.

/tom

[rexprojects]

Just to give some more information from my system

Yesterday at around 08:30 o'clock i connected to the Web UI of the NAS to check the information about the CPU_Fan Data.
I did not close that connection and my updates where very frequently. Now i logged out and the rapid updates stopped.

[Tom-Bom-badil]

Well, I meanwhile believe we are hunting a ghost here. The question should be:
Is it stopping to change because the 2nd recording is not active anymore and the NAS is in stable / idle condition now?

Both the Web UI and the mobile APP are pulling A LOT (!) of data, way more than the HA integration does: e.g. parallel updates of all plots through 'live' web sockets (many ws:// connections instead of http://), calculation of average values, permanent evaluation of disk/pool details etc. So UGreen's native tools are keeping the NAS way more busy than the HA integration does - resulting in spikes in your plots.

See e.g. these graphs that are showing my last night's data:

Looking into the code, this is how both above sensors are defined - they are both coming from /ugreen/v1/taskmgr/stat/get_all (see endpoint), it's just different nodes in the same JSON tree (see path), so obviously they are always read out at the same time:

UGREEN_STATIC_STATUS_ENDPOINTS = [
    [....]
    UgreenEntity(
        description=EntityDescription(
            key="cpu_temperature",
            name="CPU Temperature",
            icon="mdi:thermometer",
            unit_of_measurement=UnitOfTemperature.CELSIUS,
        ),
        decimal_places=0,
        endpoint="/ugreen/v1/taskmgr/stat/get_all",
        path="data.overview.cpu[0].temp",
        entity_category="Status",
    ),
    [....]
    UgreenEntity(
        description=EntityDescription(
            key="cpu_fan_speed",
            name="CPU Fan Speed",
            icon="mdi:fan",
            unit_of_measurement=REVOLUTIONS_PER_MINUTE,
        ),
        endpoint="/ugreen/v1/taskmgr/stat/get_all",
        path="data.overview.cpu_fan[0].speed",
        entity_category="Status",
    ),
    [....]

Meanwhile I believe cpu_temperature is not a good reference for checking if the integration is updating correctly - it will only change when the NAS is busy (e.g. by UGreens monitoring tools), and not much in idle conditions (e.g. when only HA integration is active).

/tom

[rexprojects]

Okay, i think i got it now.

The CPU-Temp or CPU-Usage is just not updated in the endpoint "/ugreen/v1/taskmgr/stat/get_all" by UGREEN when in idle, because you are right, Overall LAN Download (raw) or Device Fan Speed get frequent updates, even when Web UI or App are not open.

An when i connect, the system calculates those values more frequently.

So the integration is working, there is nothing wrong at my side, and if i would like to have this information with more updates i need to have an open connection or ask UGREEN to change the behavior.

Thanks for your time, learned some interesting things!

[Tom-Bom-badil]

As a quick fix for the DXP2800 until it is fixed in the next release:

Edit the file ha-root/custom_components/ugreen/api.py:

• Put a # at the beginning of each line between 410...420 (see here - this will comment out the cpu_fan sensor),
• save the file and restart HA.

The error should be gone now, and you can delete the cpu_fan sensor in settings --> devices and services --> entities.

/tom

[rexprojects]

I thought I could fix that issue by just adding a cpu fan, haha. Ordered a small noctua that maybe could fit.

If not I will do your fix, thanks.

[Tom-Bom-badil]

You could still comment out those lines, and uncomment them after fitting the fan.
Good luck! :)

/tom

[rammstone83]

I have the exact same problem, dxp2800 but this time with the android app. settings will only be refreshed when the app is open. as soon as a close it there is no refresh. i allready uncommented the cpu fan sensor ( line 410 to 420)

[rexprojects]

As for now I use glances in docker for temp and cpu usage. I used a guide from Marius Hosting.

https://mariushosting.com/how-to-install-glances-on-your-ugreen-nas/

You could try that.

[Tom-Bom-badil]

As I wasn't sure if it is missing updates of the API or just a stable idling system, I made a test. When you ssh into the NAS (e.g. with putty, or from Windows cmd: ssh tom@192.168.178.9), you have several commands avaliable for monitoring: top, mpstat 1 or uptime are some of them.

This is the output of mpstat 1 (at left in the background the current value of the HA sensor at constant 15%):

You can see that the cpu idle time is between 85.79% ... 90.80% - so CPU load is between 14.21% ... 9.2%, and not 15%. My guess is that those 15% is the overall average load since last restart until the system went idle and stopped updating the values, but this is unconfirmed. Also not sure if this is intended behaviour, or a bug in UGOS, because almost all other sensors are working 24/7 (see my screenshot above).

Conclusion: Yes, it's true that the NAS/UGOS is not updating the API for at least CPU Temp and CPU Usage when no Web GUI or Mobile App is active (but most other sensors are properly updated). No, the 'straight lines' in the plots are not caused by a stable, idling system that would lead to the same cpu percentage + temp the whole time.

/tom

[Tom-Bom-badil]

Quick update on this:

• ✅ Fans (CPU, Chassis) have been moved completely to dynamic sensors, so they are detected on startup of the integration. No more errors if no CPU fan is installed.
  
  
• ✅ Also implemented a 'count' on startup for the majority of the dynamic sensors, so it's easier to iterate through it in the code. Code to come to the :develop branch within the next days, still have to finalize some items.
  
  
• ❌ For the three currently known non-updating sensors (CPU Temp, CPU Load, RAM Load), I tried to pull them from different endpoints of the API. Same result - no updates if not either UGOS Web GUI or UGreen Mobile App are active in parallel.
  
  
• ⁉️ Over the last couple of days I digged deeply into the behaviour of the UGOS Web GUI. It seems that upon opening, it is establishing some ws: / wss: connections to specific endpoints that are kept open all the time:
  ws://<NAS>:9999/ugreen/v1/desktop/ws?...
ws://<NAS>:9999/ugreen/v1/web/sync/ws?...
  I am almost certain that this 'wakes up' the gathering of data internally in UGOS, and triggers the permanent updates of the three endpoints mentioned above.
  
  
• ⁉️ It seems those ws: / wss: permanent connections can be established either through a ugreen-internal key (ugk) in the request, or through the token we are already using. So something like the following should do the trick:
  ws://<NAS>:9999/ugreen/v1/desktop/ws?client_id=<UUID>-WEB&lang=de-DE&token=<ACCESS_TOKEN>
  
  
• ⁉️ My current idea is to extend the Token Proxy (that is permanently running on the NAS anyways) with a function to build up and keep alive a ws:// connection to a specific endpoint in UGOS. This should keep the API item updates triggered 24/7. But as I am not too deep into Docker and do not even have a ready-made development platform for it, this might take a bit.

@dobby5 Any thoughts on this from your end?

/tom

[Tom-Bom-badil]

Quick update @rexprojects: It seems I found a 'minimal invasive' way through the Docker container to keep the API updated. Code is not 'nice' yet, but seems functional - all three sensors are updated every 5s since 1 hour now - without having an active Web Dashboard or Mobile App:

Will monitor this for at least 1..2 days while making the code rollout-ready. It will be necessary to re-deploy the container on the NAS with the latest build once finished (no automatic updating through HACS possible). Will keep you posted - did your CPU fan already arrive?

/tom

[rexprojects]

That sounds great, happy to implement this in the Dashboard when ready 🙂

The PWM Fan arrived and I was able to install it, but there is not enough space without modding the heatsink. It's also not detected or shown by UGREEN. It still spins so removing some material on the heatsink could make it work, but I will probably not do that.

At least I found the reason for my high temps... One of the feet from the heatsink was not seated correctly. I had to apply much more pressure than I was comfortable with to get it in. Now the block actually is in contact with the dies and my temps are ok, lol.

[Tom-Bom-badil]

Update: Yep, looking quite promising. As usual, new 'challenges' have arisen during development. On it, already have plans how to cope. ;)








This is how it would look like with the 'old' method:





Keeping you posted.

/tom

[Tom-Bom-badil]

I have just released a new container that should fix this issue.
Further info here - please help testing.
Marking this discussion as solved, so all new comments go to that one central place.
Thanks again for reporting and your investigations, appreciated! :)

/tom