python:3.13-alpine permission denied in apk update

[tgermain-impirica]

we have been using the python:3.13-alpine image for some time. The recent update from 3.13.6 to 3.13.7 has a breaking change. Apk update (or apk add) results in permission denied errors.

# from terminal
% docker run --rm -it --entrypoint /bin/sh python:3.13-alpine

# from container terminal
apk update

#output
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/main/aarch64/APKINDEX.tar.gz
20CD3EB6FFFF0000:error:0480006C:PEM routines:get_name:no start line:crypto/pem/pem_lib.c:797:Expecting: TRUSTED CERTIFICATE
20CD3EB6FFFF0000:error:05800087:x509 certificate routines:X509_load_cert_file_ex:no certificate found:crypto/x509/by_file.c:123:
20CD3EB6FFFF0000:error:0480006C:PEM routines:get_name:no start line:crypto/pem/pem_lib.c:797:Expecting: TRUSTED CERTIFICATE
20CD3EB6FFFF0000:error:05800087:x509 certificate routines:X509_load_cert_file_ex:no certificate found:crypto/x509/by_file.c:123:
20CD3EB6FFFF0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2123:
WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.22/main: Permission denied
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/community/aarch64/APKINDEX.tar.gz
20CD3EB6FFFF0000:error:0480006C:PEM routines:get_name:no start line:crypto/pem/pem_lib.c:797:Expecting: TRUSTED CERTIFICATE
20CD3EB6FFFF0000:error:05800087:x509 certificate routines:X509_load_cert_file_ex:no certificate found:crypto/x509/by_file.c:123:
20CD3EB6FFFF0000:error:0480006C:PEM routines:get_name:no start line:crypto/pem/pem_lib.c:797:Expecting: TRUSTED CERTIFICATE
20CD3EB6FFFF0000:error:05800087:x509 certificate routines:X509_load_cert_file_ex:no certificate found:crypto/x509/by_file.c:123:
20CD3EB6FFFF0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2123:
WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.22/community: Permission denied
4 unavailable, 0 stale; 29 distinct packages available

[tianon]

Hmm, I can't seem to reproduce -- maybe seccomp or something?

$ docker run --rm --pull=always python:3.13-alpine apk update
3.13-alpine: Pulling from library/python
Digest: sha256:9ba6d8cbebf0fb6546ae71f2a1c14f6ffd2fdab83af7fa5669734ef30ad48844
Status: Image is up to date for python:3.13-alpine
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/community/x86_64/APKINDEX.tar.gz
v3.22.1-326-ge10f1967b2c [https://dl-cdn.alpinelinux.org/alpine/v3.22/main]
v3.22.1-336-gbc8bda71ff2 [https://dl-cdn.alpinelinux.org/alpine/v3.22/community]
OK: 26323 distinct packages available

[tgermain-impirica]

additional info

# get alpine version in broken image
cat /etc/alpine-release

#output
3.22.1

# try apk in alpine 3.22.1 native
docker run --rm -it alpine:3.22.1

# from within alpine container
apk update

#output
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/community/aarch64/APKINDEX.tar.gz
v3.22.1-326-ge10f1967b2c [https://dl-cdn.alpinelinux.org/alpine/v3.22/main]
v3.22.1-336-gbc8bda71ff2 [https://dl-cdn.alpinelinux.org/alpine/v3.22/community]
OK: 26161 distinct packages available

[tgermain-impirica]

Hmm, I can't seem to reproduce -- maybe seccomp or something?

$ docker run --rm --pull=always python:3.13-alpine apk update
3.13-alpine: Pulling from library/python
Digest: sha256:9ba6d8cbebf0fb6546ae71f2a1c14f6ffd2fdab83af7fa5669734ef30ad48844
Status: Image is up to date for python:3.13-alpine
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/community/x86_64/APKINDEX.tar.gz
v3.22.1-326-ge10f1967b2c [https://dl-cdn.alpinelinux.org/alpine/v3.22/main]
v3.22.1-336-gbc8bda71ff2 [https://dl-cdn.alpinelinux.org/alpine/v3.22/community]
OK: 26323 distinct packages available

Just tried your exact command

~  % docker run --rm --pull=always python:3.13-alpine apk update
3.13-alpine: Pulling from library/python
Digest: sha256:9ba6d8cbebf0fb6546ae71f2a1c14f6ffd2fdab83af7fa5669734ef30ad48844
Status: Image is up to date for python:3.13-alpine
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/main/aarch64/APKINDEX.tar.gz
204DC1B2FFFF0000:error:0480006C:PEM routines:get_name:no start line:crypto/pem/pem_lib.c:797:Expecting: TRUSTED CERTIFICATE
204DC1B2FFFF0000:error:05800087:x509 certificate routines:X509_load_cert_file_ex:no certificate found:crypto/x509/by_file.c:123:
204DC1B2FFFF0000:error:0480006C:PEM routines:get_name:no start line:crypto/pem/pem_lib.c:797:Expecting: TRUSTED CERTIFICATE
204DC1B2FFFF0000:error:05800087:x509 certificate routines:X509_load_cert_file_ex:no certificate found:crypto/x509/by_file.c:123:
204DC1B2FFFF0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2123:
WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.22/main: Permission denied
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/community/aarch64/APKINDEX.tar.gz
204DC1B2FFFF0000:error:0480006C:PEM routines:get_name:no start line:crypto/pem/pem_lib.c:797:Expecting: TRUSTED CERTIFICATE
204DC1B2FFFF0000:error:05800087:x509 certificate routines:X509_load_cert_file_ex:no certificate found:crypto/x509/by_file.c:123:
204DC1B2FFFF0000:error:0480006C:PEM routines:get_name:no start line:crypto/pem/pem_lib.c:797:Expecting: TRUSTED CERTIFICATE
204DC1B2FFFF0000:error:05800087:x509 certificate routines:X509_load_cert_file_ex:no certificate found:crypto/x509/by_file.c:123:
204DC1B2FFFF0000:error:0A000086:SSL routines:tls_post_process_server_certificate:certificate verify failed:ssl/statem/statem_clnt.c:2123:
WARNING: updating and opening https://dl-cdn.alpinelinux.org/alpine/v3.22/community: Permission denied
4 unavailable, 0 stale; 29 distinct packages available

[yosifkit]

Guessing that it could be architecture related, I tried both amd64 (native) and then arm64v8 (emulated) but I am also unable to reproduce the error. It could possibly be corrupt images (Did the Docker Desktop VM run out of space at some point? Try deleting them completely and pulling again) or perhaps it is libseccomp related (docker-library/official-images#16829)

$ docker run --rm --pull=always python:3.13-alpine apk update
3.13-alpine: Pulling from library/python
ea971c774474: Pull complete
5eb8a9c59764: Pull complete
c987cde7f297: Pull complete
Digest: sha256:9ba6d8cbebf0fb6546ae71f2a1c14f6ffd2fdab83af7fa5669734ef30ad48844
Status: Image is up to date for python:3.13-alpine
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/community/x86_64/APKINDEX.tar.gz
v3.22.1-326-ge10f1967b2c [https://dl-cdn.alpinelinux.org/alpine/v3.22/main]
v3.22.1-336-gbc8bda71ff2 [https://dl-cdn.alpinelinux.org/alpine/v3.22/community]
OK: 26323 distinct packages available
$ docker run --rm --pull=always --platform linux/arm64 python:3.13-alpine apk update
3.13-alpine: Pulling from library/python
Digest: sha256:9ba6d8cbebf0fb6546ae71f2a1c14f6ffd2fdab83af7fa5669734ef30ad48844
Status: Image is up to date for python:3.13-alpine
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/main/aarch64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.22/community/aarch64/APKINDEX.tar.gz
v3.22.1-326-ge10f1967b2c [https://dl-cdn.alpinelinux.org/alpine/v3.22/main]
v3.22.1-336-gbc8bda71ff2 [https://dl-cdn.alpinelinux.org/alpine/v3.22/community]
OK: 26162 distinct packages available

[tgermain-impirica]

Excellent point @yosifkit

This is happening on docker desktop on Mac. Maybe it is architecture related.

This command succeeds

docker run --rm --platform linux/amd64 --pull=always python:3.13-alpine apk update 

This command fails

docker run --rm --pull=always python:3.13-alpine apk update