Proactive mail by UGREEN on unusual API Traffic

[olli991]

Hey there, i got a mail recently from UGREEN because they detected that i have unusual API traffic on my system... a bit sus because i don't even use UGREEN Link stuff so how are they monitoring me 🤔

It might be this integration though 🤣

[Tom-Bom-badil]

Weird - how did they get your eMail address? Have you been in contact with them before?

I do know that the NAS is reaching out quite frequently to addresses in China, Russia, Taiwan and Vietnam (has been reported by users several times, like here. I have fully blocked the NAS for that reason in my router, and only open for a short time for updating.

Technically: 200 requests sounds a lot. It should be 7 calls every 60s only (once for every known endpoint - 6 have been known for a while now, one more was discovered recently). Will need to check in the code why data are polled so often. Do you have the UGOS App open on your mobile device in parallel?

/tom

[olli991]

The update went like this:

They had my mail because i've activated the UGREEN Link stuff in the first place and created an UGREEN account so the "link" to my NAS is there in the database.

[Tom-Bom-badil]

Thanks the update - it would be interesting to know which 'API' is meant:

• the API offered by your own NAS/UGOS (which is used by the HA integration),
• or the API of UGreen's servers on the Internet (e.g. api-us.ugnas.com - your NAS contacts those servers frequently).

/tom

p.s. On the latter, also see the posts here on the UGreen NAS Discord.

[olli991]

I wrote them with those questions and also asked if they want to hop into this discussion here on git to maybe solve problems together if they're linked.

They seem very cooperative and wrote initially to help fix problems with the OS, so maybe both sides can learn from it :)

[Tom-Bom-badil]

MAJOR EDIT: See comment below.

It might be this integration though 🤣

I did a quick analysis of the code again, then asked Copilot to do the same (just to be on safe side).

The matching outcome of both: One API call per endpoint every 60s = 7 API calls per minute in total.

To reach 200 API calls per minute (as indicated by UGreens tech support in the eMail), you would need to change the update interval manually from 60s down to every 2s or lower (=30 calls per minute x 7 API endpoints = 210 API calls per minute).

Conclusion: I highlky doubt it's the integration causing this. But never say 'never' ...

/tom

p.s. I am in the planning phase for implementing an additional data coordinator in __init__.py for collecting important, fast changing operational data like CPU + RAM usage, Temperatures etc every 5s - that's why the above is important. If this isn't working right, there might be the risk to 'overload' or 'flood' either HA and/or the NAS API, especially on high latency connections like e.g. Wifi or remote operated systems.

[Tom-Bom-badil]

Went through the code in depth by hand again, and the UGreen Tech's are right. My apologies (especially to them, you guys doing an awesome job).

Conclusion: Never trust an AI to proof your point. It will always tell you there's no problem with the code, until you start digging deeper by yourself - see above.

Explanation: The integration was polling each single entity separately from the NAS with a full API request; 200 sensors, 200 requests per minute. A simple for statement was missing to centralize the end points and poll each of them once, before sorting the received values into the corresponding entities.

Corrected. I'll build a new release in a few minutes as there was no negative comment from @dobby5 so far on the bug fixes I did last week, and this one has some urgency.

/tom

p.s. for new release: I would appreciate if some of the users would help testing / confirm proper operation by setting the logging in line 93 of __init__.py from _LOGGER.debug to _LOGGER.warning. This will show you the polling times of the end points, so you can confirm each of them is polled only once per minute. Once confirmed, just set that line back to 'debug'. Example log:

[dobby5]

Hello there, an interesting conversation you had.
Just as interesting is that ugreen has intervened himself.

For an API, 200 requests is not really a lot. But what they are right about is that it is easily avoidable.
As Tom has already pointed out, it is not necessary to query each entity individually via the API.
The implementation of a solution for this is very easy to solve using a json reader.
I would be happy to take a look at your changes @Tom-Bom-badil .
I currently have not much time to work on the project due to my work, but I am pleased to read that there is interest.

PS: If there is something you would like me to look at, please write a reference to my name (@dobby5) so I can respond more quickly.

Have a nice evening.

[Tom-Bom-badil]

Hey @dobby5, no worries, we got this. You may have noticed that I released a new version yesterday evening, change notes here. That updates includes e.g. bug fixes to the wrong iteration of disks, but also reduces the number of calls from 200...400 per minute (depending on the NAS model and number of disks/pools) to 7 per minute only (1 per end point); see screenshot above.

There is a side observation I noticed after the update, and I didn't come to a full conclusion on it's meaning yet:
According to my Pi-hole DNS server, it seems that each of our API requests towards the NAS did trigger an 'outside call' by the NAS/UGOS. Those calls dropped dramatically in the moment the current update was installed:

I can only guess what that means, but the most obvious assumption is that the API triggers (or mirrors?) requests/responses to somewhere outside my network. 200..400 API calls per minute with full JSON response, multiplied with the number of installations the integration has in the mean time, might have significantly increased the system load on that remote receipient, leading to the eMail in the first post.

Again - all this is speculation; only the massively reduced number of DNS queries by the NAS instantly after the update is a fact.

/tom

[olli991]

I just looked into my AdGuard top list (non blocked). I can understand why they wrote :D How long do I have this installed? not even 30 days tho:

But i can also see that it went down to 4x / minute

[Tom-Bom-badil]

Thanks @olli991, that confirms my observation.

In my case the traffic is even doubled, because I have 2 separate HA instances querying the NAS (develop and production system). However, it didn't succeed here - since many years, all my NASes have always been fully blocked in the firewall - and I only temporarily disable blocking for the very short time of update installations (and before any NAS update, I always shutdown all VM's & Containers including HA, just to be on safe side - so nothing was mirrored to the outside). 😁

/tom