PermissionError when creating symlink to certifi certificate bundle on Cirrus CI

[sgillies]

Description

Here's the traceback:

 -- creating symlink to certifi certificate bundle
Traceback (most recent call last):
  File "/private/var/folders/0n/_7v_bpwd1w71f8l0b0kjw5br0000gn/T/cirrus-ci-build/venv/lib/python3.12/site-packages/cibuildwheel/resources/install_certifi.py", line 54, in <module>
    main()
  File "/private/var/folders/0n/_7v_bpwd1w71f8l0b0kjw5br0000gn/T/cirrus-ci-build/venv/lib/python3.12/site-packages/cibuildwheel/resources/install_certifi.py", line 46, in main
    os.symlink(relpath_to_certifi_cafile, openssl_cafile)
PermissionError: [Errno 13] Permission denied: '../../../../Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/certifi/cacert.pem' -> 'cert.pem'

Python 3.9's etc/openssl directory isn't writable by the Cirrus CI runner?

% ls -l /Library/Frameworks/Python.framework/Versions/3.9/etc/
total 0
drwxrwxr-x  2 root  admin  64 May 17  2022 openssl

Build log

https://cirrus-ci.com/task/5534003191611392?logs=build_wheels#L302

CI config

https://github.com/sgillies/fiona-wheels/blob/83f638b2100b3162f705d1af3e78740130e2e84f/.cirrus.yml#L49

[mayeut]

Python 3.9's etc/openssl directory isn't writable by the Cirrus CI runner?

It is since the Cirrus CI runner user is in the admin group.
The relative path shown in the PermissionError traceback is weird and suggests that the target for the symlink installation is wrong.
It should be ../../lib/python3.9/site-packages/certifi/cacert.pem

I wonder if there's something going on with the environment which would change the target installation.
Could you add a printenv | sort before running cibuildwheel ?

[sgillies]

@mayeut thanks for the support! Here is the environment:

printenv | sort
ANDROID_HOME=/Users/admin/android-sdk
ANDROID_SDK_ROOT=/Users/admin/android-sdk
BRANCH=rel-1.10.0
BUILD_COMMIT=rel-1.10.0
BUILD_DEPENDS=setuptools cython~=3.0.2 wheel
BUILD_PREFIX=/private/tmp/local
CI=true
CIBW_ARCH=arm64
CIBW_PRERELEASE_PYTHONS=True
CIBW_SKIP=pp*
CIBW_TEST_COMMAND=cp -R {package}/tests tests && GDAL_ENABLE_DEPRECATED_DRIVER_GTM=YES python -m pytest -vv tests -k 'not test_collection_zip_http and not test_mask_polygon_triangle and not test_show_versions and not test_append_or_driver_error and not [PCIDSK] and not cannot_append[FlatGeobuf]'
CIBW_TEST_REQUIRES=attrs pytest click mock boto3 packaging hypothesis wheel pytz fsspec aiohttp requests pyparsing shapely
CIBW_TEST_SKIP=*:arm64
CIRRUS_ARCH=arm64
CIRRUS_BRANCH=rel-1.10.0
CIRRUS_BUILD_ID=5360053627125760
CIRRUS_BUILD_SOURCE=github
CIRRUS_CHANGE_IN_REPO=3d7bc08c63a8038050e852629237541c58c401af
CIRRUS_CHANGE_MESSAGE=Even more debugging
CIRRUS_CHANGE_TIMESTAMP=1723822332000
CIRRUS_CHANGE_TITLE=Even more debugging
CIRRUS_CI=true
CIRRUS_CLONE_SUBMODULES=true
CIRRUS_COMMIT_MESSAGE=Even more debugging
CIRRUS_CPU=4
CIRRUS_DEFAULT_BRANCH=main
CIRRUS_ENV=/var/folders/0n/_7v_bpwd1w71f8l0b0kjw5br0000gn/T/cirrus-env-task-6683631115042816-d09e9aae-0b42-4546-b21a-48b8a5ecf6b5
CIRRUS_HTTP_CACHE_HOST=127.0.0.1:12321
CIRRUS_OIDC_TOKEN=HIDDEN-BY-CIRRUS-CI
CIRRUS_OS=darwin
CIRRUS_REPO_CLONE_HOST=github.com
CIRRUS_REPO_CLONE_TOKEN=HIDDEN-BY-CIRRUS-CI
CIRRUS_REPO_CLONE_URL=https://github.com/sgillies/fiona-wheels.git
CIRRUS_REPO_FULL_NAME=sgillies/fiona-wheels
CIRRUS_REPO_ID=6002860001853440
CIRRUS_REPO_NAME=fiona-wheels
CIRRUS_REPO_OWNER=sgillies
CIRRUS_RESOLUTION_STRATEGY=SAME_SHA
CIRRUS_TASK_ID=6683631115042816
CIRRUS_TASK_NAME=Build Rasterio wheels for Mac ARM64.
CIRRUS_USER_COLLABORATOR=true
CIRRUS_USER_PERMISSION=admin
CIRRUS_VM_ID=cirrus-cli-6683631115042816-ac84466e-e0bc-4da6-9eac-59a68f531562
CIRRUS_WORKING_DIR=/var/folders/0n/_7v_bpwd1w71f8l0b0kjw5br0000gn/T/cirrus-ci-build
CI_NODE_INDEX=0
CI_NODE_TOTAL=1
CONTINUOUS_INTEGRATION=true
DYLD_LIBRARY_PATH=/private/tmp/local/lib:/private/tmp/local/lib:/private/tmp/local/lib:/private/tmp/local/lib:/private/tmp/local/lib:/private/tmp/local/lib:/private/tmp/local/lib:/private/tmp/local/lib:/private/tmp/local/lib:${BUILD_PREFIX}/lib:${LD_LIBRARY_PATH}
FLUTTER_HOME=/Users/admin/flutter
GITHUB_CHECK_SUITE_ID=27260022554
HOME=/Users/admin
HOMEBREW_CELLAR=/opt/homebrew/Cellar
HOMEBREW_NO_AUTO_UPDATE=1
HOMEBREW_NO_INSTALL_CLEANUP=1
HOMEBREW_PREFIX=/opt/homebrew
HOMEBREW_REPOSITORY=/opt/homebrew
INFOPATH=/opt/homebrew/share/info:
LANG=en_US.UTF-8
LD_LIBRARY_PATH=${BUILD_PREFIX}/lib:${LD_LIBRARY_PATH}
LOGNAME=admin
MACOSX_DEPLOYMENT_TARGET=11.0
OS=darwin
PATH=/private/var/folders/0n/_7v_bpwd1w71f8l0b0kjw5br0000gn/T/cirrus-ci-build/venv/bin:/private/tmp/local/bin:/opt/homebrew/opt/python@3.10/bin:/Users/admin/flutter:/Users/admin/flutter/bin/:/Users/admin/flutter/bin/cache/dart-sdk/bin:/opt/homebrew/opt/openjdk@17/bin:/opt/homebrew/opt/node@20/bin:/Users/admin/.rbenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Library/Apple/usr/bin:/Users/admin/android-sdk/cmdline-tools/latest/bin:/Users/admin/android-sdk/platform-tools:/Users/admin/android-sdk/emulator
PLAT=arm64
PROJ_DATA=/private/tmp/local/share/proj
PS1=(venv) 
PWD=/private/var/folders/0n/_7v_bpwd1w71f8l0b0kjw5br0000gn/T/cirrus-ci-build
RBENV_SHELL=zsh
REPO_DIR=Fiona
SHELL=/bin/zsh
SHLVL=2
SSH_CLIENT=192.168.64.1 56040 22
SSH_CONNECTION=192.168.64.1 56040 192.168.64.6 22
TERM=xterm-256color
TEST_DEPENDS=attrs pytest click mock boto3 packaging hypothesis wheel pytz fsspec aiohttp requests pyparsing shapely
TMPDIR=/var/folders/0n/_7v_bpwd1w71f8l0b0kjw5br0000gn/T/
USER=admin
VIRTUAL_ENV=/private/var/folders/0n/_7v_bpwd1w71f8l0b0kjw5br0000gn/T/cirrus-ci-build/venv
VIRTUAL_ENV_PROMPT=(venv) 
_=/usr/bin/printenv
pwd
/private/var/folders/0n/_7v_bpwd1w71f8l0b0kjw5br0000gn/T/cirrus-ci-build
python -m cibuildwheel --output-dir wheelhouse Fiona

[mayeut]

my best guess would be the wrong openssl library gets loaded because of DYLD_LIBRARY_PATH.

[sgillies]

@mayeut I agree. And now that you mention it, I think this explains why I've seen another project only set library paths within CIBW_REPAIR_WHEEL_COMMAND. Thanks for the help and apologies for the distraction!