Skip to content

Commit

Permalink
Address issues with latest AWS-LC and OpenBSD (#569)
Browse files Browse the repository at this point in the history
**Issue:**
The latest AWS-LC was crashing on OpenBSD 7.4, when running test `test.test_http_client.TestClient.test_connect_pq_tlsv1_0_2021_05`

**Investigation:**
AWS-LC added [OpenBSD 7.4 and 7.5 Support](aws/aws-lc#1437) in [v1.26.0](https://github.com/aws/aws-lc/releases/tag/v1.26.0). [Ironically](https://www.youtube.com/watch?v=Jne9t8sHpUc), these changes broke our existing OpenBSD 7.4 CI. My understanding is: "support OpenBSD" means "support fancy assembly math, instead of using vanilla C code math" on OpenBSD. This fancy assembly math currently reads from the .text section of the library, which is forbidden if a library is linked with the `--execute-only` flag, which OpenBSD 7.4+ uses by default.

**Description of changes:**
- Update to AWS-LC v1.24.0 -> v1.28.0
- Set '-Wl,--no-execute-only' flag when building for OpenBSD and using AWS-LC
- Add OpenBSD 7.4 and 7.5 to CI (OpenBSD supports its two most recent releases)
  • Loading branch information
graebm authored Jun 7, 2024
1 parent 4a4fd22 commit c9c84bc
Show file tree
Hide file tree
Showing 3 changed files with 21 additions and 8 deletions.
11 changes: 9 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -192,21 +192,28 @@ jobs:
openbsd:
runs-on: ubuntu-22.04 # latest
strategy:
fail-fast: false
matrix:
# OpenBSD only supports the two most recent releases
version: ['7.4', '7.5']
steps:
# Cannot use builder to checkout as OpenBSD doesn't ship git in the base install
- uses: actions/checkout@v3
with:
submodules: true
- name: Build ${{ env.PACKAGE_NAME }} + consumers
uses: cross-platform-actions/action@v0.23.0
uses: cross-platform-actions/action@v0.24.0
with:
operating_system: openbsd
version: '7.4'
version: ${{ matrix.version }}
cpu_count: 4
shell: bash
environment_variables: AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_DEFAULT_REGION AWS_REGION
run: |
sudo pkg_add awscli py3-pip py3-urllib3
python3 -m venv .venv
source .venv/bin/activate
python3 -c "from urllib.request import urlretrieve; urlretrieve('${{ env.BUILDER_HOST }}/${{ env.BUILDER_SOURCE }}/${{ env.BUILDER_VERSION }}/builder.pyz', 'builder')"
chmod a+x builder
./builder build -p ${{ env.PACKAGE_NAME }}
Expand Down
2 changes: 1 addition & 1 deletion crt/aws-lc
16 changes: 11 additions & 5 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -334,17 +334,23 @@ def awscrt_ext():

if using_system_libcrypto():
libraries += ['crypto']
else:
# hide the symbols from libcrypto.a
# this prevents weird crashes if an application also ends up using
# libcrypto.so from the system's OpenSSL installation.
extra_link_args += ['-Wl,--exclude-libs,libcrypto.a']

# OpenBSD 7.4+ defaults to linking with --execute-only, which is bad for AWS-LC.
# See: https://github.com/aws/aws-lc/blob/4b07805bddc55f68e5ce8c42f215da51c7a4e099/CMakeLists.txt#L44-L53
# (If AWS-LC's CMakeLists.txt removes these lines in the future, we can remove this hack here as well)
if sys.platform.startswith('openbsd'):
extra_link_args += ['-Wl,--no-execute-only']

# FreeBSD doesn't have execinfo as a part of libc like other Unix variant.
# Passing linker flag to link execinfo properly
if sys.platform.startswith('freebsd'):
extra_link_args += ['-lexecinfo']

# hide the symbols from libcrypto.a
# this prevents weird crashes if an application also ends up using
# libcrypto.so from the system's OpenSSL installation.
extra_link_args += ['-Wl,--exclude-libs,libcrypto.a']

# python usually adds -pthread automatically, but we've observed
# rare cases where that didn't happen, so let's be explicit.
extra_link_args += ['-pthread']
Expand Down

0 comments on commit c9c84bc

Please sign in to comment.