High network utilisation (large uploads) being reported when the Icloudpd container is inactive ?

[nodecentral]

Hi,

There seems to be a lot of network traffic, mainly uploads (and down) from the container.

This is slightly concerning, especially as I’ve left the container running after it failed to locate the required cookie to work.

Why is it doing that, and where is all that traffic going ?

[boredazfcuk]

Hi,

The container doesn't generate any traffic at all when it's idling. What are you using to report on the network traffic? It sounds like it's inaccurate.

You can see for yourself by connecting to the running container:

docker exec -it <name of icloudpd container> /bin/ash

then run

ps

This will show you all the processes that are running inside the container. You should only see the the following (during an idle period):
sync-icloud.sh - My script that tells the system what commands to run and when
ash - the shell that you're connected to
ps - the command you have run to see what processes are running
sleep <nnnn> - one of the commands invoked by my script to wait between synchronisation intervals.

If you run this while the container is synchronising, the sleep command will be replaced by icloudpd. If the container is sending an alert, it will be curl

If you want to look at traffic in real time, go back to your shell prompt and install a network monitoring tool:

apk add tcpdump

to see all tcp traffic going in and out of your network card type:

tcpdump -vv

This is the maximum logging level so it will generate huge amounts of data... if it sees anything at all (which it shouldn't until it synchronises). Press + C to exit. It should then display a summery showing no data was sent over the Ethernet adapter.

If doing this, you'll want to create the container with the syncrhronisation_delay variable set to 10 or 15 or something like that to give you enough time to get connected to the container, install the tcpdump utility, and set it running before a synchronisation occurs. Otherwise the initial sync may have completed, and it will be in it's sleep phase where you'll see nothing for however long your synchronisation_interval variable is set, which should be 6-12 hours.

Thanks.

[nodecentral]

Hi,

Thanks for responding, in regards to where I obtained that information; as I use QNAP’s Container Station to create/run Icloudpd , it has a chart that shows each container’s network utilisation. And Icloudpd was going crazy. (Likely inaccurate but I thought it best to raise it)

I’d stopped the container, but will run it again later , so I can monitor it and see if I can do some reporting, I’m also going to install a network monitoring tool too - called netdata. (Have you heard of that ?)

[boredazfcuk]

Is it not that Container Station is showing utilisation for the network that the container is attached to? So if you have 10 containers connected to the same network, it shows the utilisation for all 10 containers?

I've seen Netdata but not used it. I use both Zabbix/PRTG Network Monitor for monitoring all the infrastructure I look after.

[nodecentral]

It’s my understanding that the ‘network usage’ chart is per-container.
To check, I’ve just run the system resource monitor alongside a running container, and the values are very different..

[boredazfcuk]

I wouldn't pay much attention to it. Obtaining true network utilisation statistics on a per-container basis is not an easy task at all. It says as much in the documentation: https://docs.docker.com/config/containers/runmetrics/#network-metrics

It's probably showing cumulative totals. Does it reset to 0 when you restart the container and slowly climb the longer the container is on? Jumping up when the container is actively syncing?

The only you can be sure you're seeing accurate stats is if you have some sort of agent running inside the container that monitors traffic on a per-process basis. Just connect to the container and then install iptraf-ng:

apk add iptraf-ng

Run iptraf-ng from the command line and it will bring up a GUI with which you can interrogate the network traffic all you like. Connect to the server a second time and run the script a few times and watch what affect it has on the statistics in the other connection.

[boredazfcuk]

Closing this off as I wasn't able to replicate it. Running network monitoring software inside the container shows no traffic whatsoever while script is not running.