Group id, 0, already in use by the group

[mariopaolo]

Hello,
thanks for the wonderful app.

I am trying to deploy the TrueChart version of docker-icloudpd (latest_3.0.4) on a TrueNAS SCALE server (22.12.4.2).

Before opening this issue I browsed past issues like this one and this one but couldn't make it work.

Basically, if I just specify my Apple ID and deploy, I hit the common problem about not being able to run traceroute

2023-11-07 02:37:36 INFO     ***** boredazfcuk/icloudpd container for icloud_photo_downloader started *****
2023-11-07 02:37:36 INFO     ***** For support, please go here: https://github.com/boredazfcuk/docker-icloudpd *****
2023-11-07 02:37:36 INFO     ***** /usr/local/bin/sync-icloud.sh date: 2022/12/28_22:22 *****
2023-11-07 02:37:36 INFO     ***** /usr/local/bin/sync-icloud.sh hash: a4728cb97b760a5c9d7cc3f54cdb2f44 *****
2023-11-07 02:37:36 INFO     Alpine Linux 3.17.0
2023-11-07 02:37:36 INFO     Python version: 3.10.9
2023-11-07 02:37:37 INFO     icloudpd version: 1.7.2
2023-11-07 02:37:37 INFO     pyicloud-ipd version: 0.10.1
2023-11-07 02:37:37 INFO     Running user id: 0
2023-11-07 02:37:37 INFO     Running group id: 0
2023-11-07 02:37:37 INFO     Local user: apps:0
2023-11-07 02:37:37 INFO     Local group: apps:0
2023-11-07 02:37:37 INFO     Force GID: False
2023-11-07 02:37:37 INFO     LAN IP Address: 172.16.2.153
2023-11-07 02:37:37 INFO     Default gateway: 172.16.0.1
2023-11-07 02:37:37 INFO     DNS server: 172.17.0.10
2023-11-07 02:37:38 INFO     IP address for icloud.com: 17.253.144.10
2023-11-07 02:37:38 ERROR    No route to icloud.com found. Please check your container's network settings - exiting
2023-11-07 02:37:38 ERROR    Error debug - traceroute: socket(AF_INET,3,1): Operation not permitted

I browsed the issues in the repo and found this one, where you provide a working configuration for SCALE. Below is the exact same configuration from the next deploy attempt (all other settings being default):

If I try with these settings, I get past the error above, but I am now greeted by a new one:

2023-11-07 02:45:48 INFO     ***** boredazfcuk/icloudpd container for icloud_photo_downloader started *****
2023-11-07 02:45:48 INFO     ***** For support, please go here: https://github.com/boredazfcuk/docker-icloudpd *****
2023-11-07 02:45:48 INFO     ***** /usr/local/bin/sync-icloud.sh date: 2022/12/28_22:22 *****
2023-11-07 02:45:48 INFO     ***** /usr/local/bin/sync-icloud.sh hash: a4728cb97b760a5c9d7cc3f54cdb2f44 *****
2023-11-07 02:45:48 INFO     Alpine Linux 3.17.0
2023-11-07 02:45:48 INFO     Python version: 3.10.9
2023-11-07 02:45:49 INFO     icloudpd version: 1.7.2
2023-11-07 02:45:49 INFO     pyicloud-ipd version: 0.10.1
2023-11-07 02:45:49 INFO     Running user id: 0
2023-11-07 02:45:49 INFO     Running group id: 0
2023-11-07 02:45:49 INFO     Local user: apps:0
2023-11-07 02:45:49 INFO     Local group: apps:0
2023-11-07 02:45:49 INFO     Force GID: False
2023-11-07 02:45:49 INFO     LAN IP Address: 172.16.2.155
2023-11-07 02:45:49 INFO     Default gateway: 172.16.0.1
2023-11-07 02:45:49 INFO     DNS server: 172.17.0.10
2023-11-07 02:45:49 INFO     IP address for icloud.com: 17.253.144.10
2023-11-07 02:45:49 INFO     Route check to icloud.com successful
2023-11-07 02:45:49 INFO     Apple ID: my.appleid@icloud.com
2023-11-07 02:45:49 INFO     Authentication Type: 2FA
2023-11-07 02:45:49 INFO     Cookie path: /config/myappleidicloudcom
2023-11-07 02:45:49 INFO     Cookie expiry notification period: 7
2023-11-07 02:45:49 INFO     Download destination directory: /home/apps/iCloud
2023-11-07 02:45:49 INFO     Folder structure: {:%Y/%m/%d}
2023-11-07 02:45:49 INFO     Directory permissions: 750
2023-11-07 02:45:49 INFO     File permissions: 640
2023-11-07 02:45:49 INFO     Synchronisation interval: 86400
2023-11-07 02:45:49 INFO     Synchronisation delay (minutes): 0
2023-11-07 02:45:49 INFO     Set EXIF date/time: False
2023-11-07 02:45:49 INFO     Auto delete: False
2023-11-07 02:45:49 INFO     Photo size: original
2023-11-07 02:45:49 INFO     Single pass mode: False
2023-11-07 02:45:49 INFO     Skip download check: False
2023-11-07 02:45:49 INFO     Skip live photos: False
2023-11-07 02:45:49 INFO     Number of most recently added photos to download: Download All Photos
2023-11-07 02:45:49 INFO     Downloading photos from album: Download All Photos
2023-11-07 02:45:49 INFO     Stop downloading when prexisiting files count is: Download All Photos
2023-11-07 02:45:49 INFO     Live photo size: original
2023-11-07 02:45:49 INFO     Skip videos: False
2023-11-07 02:45:49 INFO     Convert HEIC to JPEG: False
2023-11-07 02:45:49 INFO     converted JPEGs path: /home/apps/iCloud
2023-11-07 02:45:49 INFO     JPEG conversion quality: 90
2023-11-07 02:45:49 INFO     Downloading from: icloud.com
2023-11-07 02:45:49 INFO     Nextcloud synchronisation trigger: Disabled
2023-11-07 02:45:49 INFO     Creating directory: /home/apps/.local/share/
2023-11-07 02:45:49 INFO     Creating symbolic link: /home/apps/.local/share/python_keyring/ to: /config/python_keyring/ directory
2023-11-07 02:45:49 ERROR    Group id, 0, already in use by the group: apps - exiting. If you must to add your user to this pre-existing system group, please set the force_gid variable to True

and since it mentions setting force_gid to True, I tried it adding it as an env var in a new attempt:

In this case I get the exact same log with additional lines (below), except the ERROR is now a WARNING (possibly because of force_gid)

2023-11-07 02:51:36 WARNING  Group id, 0, already in use by the group: root - continuing as force_gid variable has been set. Group name to use: root
2023-11-07 02:51:36 INFO     Creating user apps:0
useradd: UID 0 is not unique
2023-11-07 02:51:36 INFO     Correct owner on icloudpd temp directory, if required
find: unknown user apps
2023-11-07 02:51:36 INFO     Correct group on icloudpd temp directory, if required
2023-11-07 02:51:36 INFO     Correct owner on config directory, if required
find: unknown user apps
2023-11-07 02:51:36 INFO     Correct group on config directory, if required
2023-11-07 02:51:36 INFO     Correct owner on keyring directory, if required
find: unknown user apps
2023-11-07 02:51:36 INFO     Correct group on keyring directory, if required
2023-11-07 02:51:36 INFO     Configure password
2023-11-07 02:51:36 ERROR    Keyring file /config/python_keyring/keyring_pass.cfg does not exist
2023-11-07 02:51:36 INFO      - Please add the your password to the system keyring using the --Initialise script command line option
2023-11-07 02:51:36 INFO      - Syntax: docker exec -it <container name> sync-icloud.sh --Initialise
2023-11-07 02:51:36 INFO      - Example: docker exec -it icloudpd sync-icloud.sh --Initialise
2023-11-07 02:51:36 INFO     Waiting for keyring file to be created...

As you can see there are several failed commands in between log lines, like useradd: UID 0 is not unique or find: unknown user apps. In this state I tried to run /usr/local/bin/sync-icloud.sh --Initialise but got the same output and same errors (useradd: UID 0 is not unique etc...)

Now, I tried possibly every combination of user/group settings in the SCALE app config, but I never made it further.
I tried setting user/group to 568, but I go back to the first error (traceroute). Tried with everything set to 0 (including fsGroup), still don't get past the last error. Set Supplemental group to 0, it didn't make any difference.

I then started passing envvars for user/group names and ids, but the TrueChart app has hardcoded values that cannot be overridden even if manually specified https://github.com/truecharts/charts/blob/cfe948e5b0b07d2a704079a6fbf6783dfacfe7d8/charts/incubator/icloudpd/values.yaml#L77

I have run out of options so far, hence the ticket. Not sure if this depends on the app, or also on TrueCharts implementation of the helm chart, but seeing you helped other users with SCALE in the past, I tried :)

P.S.: for the ticket I deleted my first attempt and redid all the steps with a vanilla deploy, so that other options wouldn't interfere. Nevertheless, I was able to setup Discord notifications and other stuff in my first attempt, so I realized the problem was just about permissions. I then confirmed it looking at several other tickets dealing with similar issues.

Thanks again for this amazing app.

[boredazfcuk]

The issues with not being able to run traceroute is due to the TrueNAS configuration. By default, TrueNAS runs containers unprivileged, so the traceroute command fails. "Privileged mode" is the the configuration option needed to be checked to allow that to work:

However, running as root isn't actually supported:

2023-11-07 02:45:49 INFO     Local user: apps:0
2023-11-07 02:45:49 INFO     Local group: apps:0

The container will create the user that is specified, and as the root user always exists within the container, it will fail.

Set user=apps, group=apps, user_id=568 and group_id=568 and I think it will work.

[partnerinflight]

That doesn't seem to work either -- if the container is in Privileged mode and the script is running as 568, traceroute once again gets denied.

[mariopaolo]

hey thanks for the reply @boredazfcuk

"Privileged mode" is the the configuration option needed to be checked to allow that to work:

unfortunately, as I stated in my original ticket (and showed in the first screenshot enclosed), before opening this issue I made sure to follow your instructions in ticket #381.
by the way I can confirm @partnerinflight report, still no luck with the TrueCharts version of icloudpd on my SCALE server

EDIT: as I mentioned earlier, we can't pass any of those envvars (user, group, user_id and group_id) since they are hardcoded and the chart won't deploy if present in the config (even with identical values).

[partnerinflight]

oh at least as far as user_id and group_id far as I can tell we just set the appropriate value in the runAsUserId/runAsGroupId boxes -- those map to user_id/group_id in the config file.

But setting them to those values -- as I said -- breaks traceroute permissions again.

[boredazfcuk]

Something must be off as it's working on mine with those settings:

[mariopaolo]

Something must be off as it's working on mine with those settings:

I see you are not using the TrueCharts version of icloudpd, you are using a custom-app instead to run the app.
I know it might be feasible with a custom-app but wanted to investigate the issue about the official app.

Regaridng the issue at hand, you can reproduce it by just deploying the current icloudpd app on the stable train.

thanks

[boredazfcuk]

I see you are not using the TrueCharts version of icloudpd, you are using a custom-app instead to run the app.

Ahh I get you, I'm using the TrueCharts custom-app but they have a separate app for icloudpd as well.

I know it might be feasible with a custom-app but wanted to investigate the issue about the official app.

I only publish the container to Dockerhub, so that's the only place the truly official version of my container can be obtained. I configured the 'custom-app' to download the container from Dockerhub, so I know it's the official version.

I have no knowledge of the TrueCharts "official" app. It's entirely possible they are making their own modifications to the container, repackaging it and hosting it in their own repository. If their "official" app isn't working, then it would be best to report the problem to them. They must be handling things differently in their official app to how the custom app does things. Maybe they can identify the issue and fix it, but it's not something I have access to.

[partnerinflight]

Ok, I got the custom-app to work using instructions above, and runAsUser/runAsGroup set to 0.