Skip to content

Commit

Permalink
build: use statically linked libssl (#546)
Browse files Browse the repository at this point in the history
We build our binaries on ubuntu:20 which uses libssl1.1 (through our
direct dependency of `reqwest`). Libssl is dynamically linked and with
systems moving to ubuntu:22 that has libssl3.0 our binary breaks. An
example is Google's Collab.

We also build multiplatform linux docker image using `cross-rs`.
`cross-rs` currently builds for ubuntu:20 on their `master` branch and
there is currently an open PR
(cross-rs/cross#973) to move to ubuntu:22.

The proposed solution with this PR is to enable the feature on `reqwest`
that builds the libssl crate (and the libssl library) statically in the
sparrow binaries rather than relying on dynamic linking to pick the
system's libssl library.

We are *not* moving our CI to use ubuntu:22 until there we have a
solution for building multiarch images on ubuntu:22 (through `cross-rs`
or otherwise). Otherwise, if we move to ubuntu:22 today with out any
other changes our docker image will break on linux on arm (anyone on a
Mac with an M chip running a docker container).




1. sparrow depends on `reqwest` which brings in libssl. 
2. added an `if` condition on the release CI that we missed


I verified that the binaries 

Before: 
```
❯ ldd sparrow-main
	linux-vdso.so.1 (0x00007fff56643000)
	libssl.so.3 => /lib/x86_64-linux-gnu/libssl.so.3 (0x00007f3387ee8000)
	libcrypto.so.3 => /lib/x86_64-linux-gnu/libcrypto.so.3 (0x00007f3383e00000)
	libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3383a00000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f3387ec8000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3384321000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3383c1f000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f3387fa4000)

```


After 

```
❯ ldd sparrow-main
        linux-vdso.so.1 (0x00007ffc7d3f2000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f3989800000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f398d9aa000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f3989b21000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f398961f000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f398d9dd000)
```


The final `release` binary size increases by 2MB 

Before:
 
```
❯ ls -lh sparrow-main
-rwxr-xr-x 2 therapon therapon 85M Jul 24 13:09 sparrow-main

```

After

```
❯ ls -lh sparrow-main             
-rwxr-xr-x 2 therapon therapon 87M Jul 24 13:06 sparrow-main

```
  • Loading branch information
therapon authored Jul 25, 2023
1 parent 74ed812 commit fafdf58
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 1 deletion.
1 change: 1 addition & 0 deletions .github/workflows/release_engine.yml
Original file line number Diff line number Diff line change
Expand Up @@ -374,6 +374,7 @@ jobs:
- name: Build and push Docker images for Jupyter Beta
uses: docker/build-push-action@v4
if: startsWith(github.ref, 'refs/tags/engine@v') && contains(github.ref, 'beta')
with:
context: .
platforms: linux/amd64, linux/arm64
Expand Down
10 changes: 10 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ prost-wkt-build = "0.4.1"
prost-wkt-types = "0.4.1"
pulsar = { version = "5.1.0", default-features = false, features = ["async-std-runtime", "tokio-runtime", "lz4"] }
rand = "0.8.5"
reqwest = "0.11.14"
reqwest = { version = "0.11.14", features = ["native-tls-vendored"] }
serde = { version = "1.0.159", features = ["derive", "rc"] }
serde_json = "1.0.95"
serde_yaml = "0.9.19"
Expand Down

0 comments on commit fafdf58

Please sign in to comment.