Extensions are not being installed with deployment profiles and allowed images list

[IsaSih]

Actual Behavior

I am not sure if the deployment profiles and the allowed images list are somehow breaking lima, but none of the extensions regardless if they are allowed or not in the deployment profiles, are failing to download

Error says : limactl exited with code 1

Steps to Reproduce

1. Run a factory reset
2. Create system and user's deployment profiles (the files can be found in the session Additional Information)
3. Start Rancher Desktop

Result

All the extensions are failing to download

Important information from the logs:
extensions.log:

023-06-09T02:50:17.725Z: Failed to read metadata for docker/disk-usage-extension: Error: /Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl exited with code 1

2023-06-09T02:50:20.925Z: Failed to read metadata for ignatandrei/blockly-automation: Error: /Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl exited with code 1

2023-06-09T02:51:24.267Z: Failed to read metadata for docker/logs-explorer-extension: Error: /Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl exited with code 1

2023-06-09T02:53:09.118Z: Failed to read metadata for joycelin79/newman-extension: Error: /Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl exited with code 1

from lima.log:

2023-06-09T02:53:09.118Z: Lima: executing: /usr/local/bin/nerdctl --namespace rancher-desktop-extensions create --entrypoint=/ joycelin79/newman-extension:0.0.7: Error: /Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl exited with code 1

2023-06-09T02:53:09.118Z: stdout: 
2023-06-09T02:53:09.118Z: stderr: time="2023-06-08T19:53:09-07:00" level=debug msg="changeDirCmd=\"cd . || exit 1\""
time="2023-06-08T19:53:09-07:00" level=debug msg="OpenSSH version 8.6.1 detected"
time="2023-06-08T19:53:09-07:00" level=debug msg="AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com"
time="2023-06-08T19:53:09-07:00" level=debug msg="executing ssh (may take a long)): [/usr/bin/ssh -F /dev/null -o IdentityFile=\"/Users/isasih/Library/Application Support/rancher-desktop/lima/_config/user\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=isasih -o ControlMaster=auto -o ControlPath=\"/Users/isasih/Library/Application Support/rancher-desktop/lima/0/ssh.sock\" -o ControlPersist=5m -q -p 56789 127.0.0.1 -- cd . || exit 1 ; exec \"$SHELL\" --login -c '/usr/local/bin/nerdctl --namespace rancher-desktop-extensions create --entrypoint=/ joycelin79/newman-extension:0.0.7']"
time="2023-06-09T02:53:07Z" level=fatal msg="failed to resolve reference \"docker.io/joycelin79/newman-extension:0.0.7\": unexpected status from HEAD request to https://registry-1.docker.io/v2/joycelin79/newman-extension/manifests/0.0.7: 403 Forbidden"

From serial.log:

 * Waiting for uevents to be processed ... [ ok ]
 * Starting networking ... *   lo ... [ ok ]
 *   eth0 ...udhcpc: started, v1.36.1
udhcpc: broadcasting discover
udhcpc: broadcasting select for 192.168.5.15, server 192.168.5.2
udhcpc: lease of 192.168.5.15 obtained from 192.168.5.2, lease time 86400
 [ ok ]
 * The binfmt-misc module needs to be loaded by the modules service or built in.
 * Mounting misc binary format filesystem ... [ ok ]
 * Registering QEMU binaries in binfmt misc ... [ ok ]
 [ ok ]
 * Caching service dependencies ... [ ok ]
ssh-keygen: generating new host keys: RSA ECDSA ED25519 
 * Starting sshd ... [ ok ]


Welcome to Alpine Linux 3.18

Kernel 6.1.32-0-virt on an x86_64 (/dev/ttyS0)



lima-rancher-desktop login: 

Archive.zip

lima-logs.zip

Expected Behavior

Extensions that belong to the allowed images list or the allowed extensions list should be installed with no errors. Extensions that are not in any of these list should be preventing from installing with a bad request error.

Additional Information

Here are the files containing each of the deployment profiles used

System locked profile - profile.txt
System default profile - sys-profile-default.txt
User locked profile - user-profile-locked.txt
User default profile -
user-profile-defaults.txt

Rancher Desktop Version

1.7.0-1669-g22e4b6c2

Rancher Desktop K8s Version

1.26.2

Which container engine are you using?

containerd (nerdctl)

What operating system are you using?

macOS

Operating System / Build Version

macOS Monterey

What CPU architecture are you using?

x64

Linux only: what package format did you use to install Rancher Desktop?

None

Windows User Only

No response

[xianjin-xu]

yes，i have the same problem, i guess the reason is that the timeout 230 seconds is the load balance of azure app, my network speed just at 500~600 kb/s, i retry many times, every there was more than 40m left can't be downloaded.

[IsaSih]

Issue happens with both dockerd and containerd. Both system and user deployment profiles

[mook-as]

failed to resolve reference "docker.io/joycelin79/newman-extension:0.0.7": … 403 Forbidden

That sounds like Docker Hub doesn't like us. If you're doing this remotely (using Remote Desktop or something to a machine in the lab), perhaps there's been too many requests from that IP address and you're being rate limited? That should be HTTP 429 instead, though…

[mook-as]

Oh, it's the image allow list; the system locked profile has:

  <key>containerEngine</key>
  <dict>
    <key>allowedImages</key>
    <dict>
      <key>enabled</key>
      <true/>
      <key>patterns</key>
      <array>
        <string>nginx</string>
        <string>busybox</string>
      </array>
    </dict>
  </dict>

That doesn't cover docker.io/joycelin79/newman-extension:0.0.7…

[IsaSih]

I'm using my own machine.

[IsaSih]

But if we have it allowed in the extensions list shouldn't this suffice? the image needs to be in both images and extensions list?

[mook-as]

Correct, it needs to show up in both lists.

We could consider a new enhancement issue (but then it gets confusing for the admins going the other way too).

[jandubois]

But if we have it allowed in the extensions list shouldn't this suffice?

Yes, Rancher Desktop is supposed to add all allowed extensions to the Allowed Images list internally (not included in settings.json, but included in the proxy filter rules).

I don't know if this got implemented or not; @mook-as do you remember?

[mook-as]

@jandubois I don't recall that requirement at all…

[IsaSih]

So, what is the purpose of the allowed extensions list? Should this be used then just to specify tags but list the image ID in the allowed images list? It's now a bit unclear on what should be the expected behaviour when we have both lists enabled.

[jandubois]

I don't recall that requirement at all…

I thought it was in the original story, but I can't find it right now either. I did talk about it in #4655 (comment)

Anyways, I'll create an issue for it, and then we can discuss if there are reasons not to do this.

[jandubois]

what is the purpose of the allowed extensions list?

The purpose of the list is that an administrator can explicitly allow specific extensions to be loaded. This can be more narrow than the general Allowed Images list:

• You can allow all images from docker/* to be used in the container engine or k8s, but only allow a single specific extension.
• You can lock an extension to a specific tag. The allowed images list doesn't allow you to specify a tag.

Extensions are more closely restricted because they can execute code on the host. Regular images only run inside the container runtime inside the VM.

It's now a bit unclear on what should be the expected behaviour when we have both lists enabled.

The Allowed Images list will restrict which images can be pulled from a registry. Since extensions need to be pulled, they will need to be allowed (but can't limit to a specific tag).

Even if the extension image can be pulled, it still needs to be included in the Allowed Extensions list (including tag, if specified). Otherwise the extension will not be enabled, even though it may have been pulled already.

I've created #4926 to add all allowed extensions to the Allowed Images list automatically. It doesn't make sense to allow an extension if you don't intend to allow the user to pull the corresponding image. So there is no need to require the admin to add it to both lists.

Note that you can lock down the extension list without having the Allowed Images list enabled.

[IsaSih]

Ok. So, in case we have the image in the allowed images list, but not in the allowed extensions, user will just be able to run them with the container runtime in the VM. But not to instal an extension from that image, correct?

[jandubois]

That is correct.

[gaktive]

We'll look at this more for 1.10. @jandubois to put something in release notes for 1.9

[ericpromislow]

I saw one problem: <string>docker/disk-usage-extension:0.2.7</string> should be <string>docker/disk-usage-extension</string> (no version tag).

After making that change, when containerEngine.allowedImages.enabled is set to false, both the disk-usage and newman extensions show up in the catalog and are installable. No other extensions appeared in the catalog, because they weren't whitelisted, as expected.

When containerEngine.allowedImages.enabled is set to false in the system locked file, and the extensions list is left as is,

[ericpromislow]

I had closed this, but reopened due to Gary's comment at #4920 (comment) -- but I would mark this closed because I can now run and install the allowed extensions as long as I turn off the allowed-images field.