Skip to content

Commit

Permalink
Update the base image for ci-build's Dockerfile (#2674)
Browse files Browse the repository at this point in the history
## Motivation and Context
~~**The PR will be a draft until
#2675 is merged into `main`.
Once that's merged, this PR can remove the change to `Cargo.toml` in
`aws-smithy-client`.**~~

**Now that it's passed tests in CI, ysaito1001 will merge this PR once
it has passed the internal tests**

Fixes #2500.

## Description
This PR updates the base image for ci-build's Dockerfile as the latest
image for [amazonlinux](https://gallery.ecr.aws/amazonlinux/amazonlinux)
has been updated to 2023.

As part of it, the changes to Dockerfile include
- Fixing conflicting curl's libraries by specifying `--allowerasing`
Without it, we'd get an error as follows:
```
package curl-minimal-7.88.1-1.amzn2023.0.1.x86_64 conflicts with curl provided by curl-7.87.0-2.amzn2023.0.2.x86_64
```
- Adding perl explicitly as a new version of openssl now requires it
Without `perl`, we'd get an error as follows:
```
Step 24/48 : RUN cargo +${rust_nightly_version} -Z sparse-registry install cargo-deny --locked --version ${cargo_deny_version}
 ---> Running in 3c3431881cfa
...
error: failed to run custom build command for `openssl-sys v0.9.76`
...
  --- stderr
  Can't locate FindBin.pm in @inc (you may need to install the FindBin module) (@inc contains: /usr/local/lib64/perl5/5.32 /usr/local/share/perl5/5.32 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at ./Configure line 15.
  BEGIN failed--compilation aborted at ./Configure line 15.
  thread 'main' panicked at '
```

Also, there is a change to `Cargo.toml` in `aws-smithy-client`:
- Forcing `aws-smithy-client` with `native-tls` feature to use a more
recent `openssl-sys` crate
Without it, `cargo minimal-versions check` would fail in
`aws-smithy-client` trying to build `openssl-sys` v0.9.39 where OpenSSL
in a container is more up-to-date, leading to a build failure (See [this
issue](sfackler/rust-openssl#1724) for more
details)

Finally, updating CI for `Run Tests on Windows`
- Manually installing `openssl` as
[suggested](sfackler/rust-openssl#1542 (comment)).
Without it, after introducing a more recent `openssl` 0.10.52
`dev-dependencies` in `aws-smithy-client`, we'd get this on Windows:
```
  --- stderr
  thread 'main' panicked at '

  Could not find directory of OpenSSL installation, and this `-sys` crate cannot
  proceed without this knowledge. If OpenSSL is installed and this crate had
  trouble finding it,  you can set the `OPENSSL_DIR` environment variable for the
  compilation process.

  Make sure you also have the development packages of openssl installed.
  For example, `libssl-dev` on Ubuntu or `openssl-devel` on Fedora.

  If you're in a situation where you think the directory *should* be found
  automatically, please open a bug at https://github.com/sfackler/rust-openssl
  and include information about your system as well as this message.

  $HOST = x86_64-pc-windows-msvc
  $TARGET = x86_64-pc-windows-msvc
  openssl-sys = 0.9.90
```

----

_By submitting this pull request, I confirm that you can use, modify,
copy, and redistribute this contribution, under the terms of your
choice._

---------

Co-authored-by: Yuki Saito <awsaito@amazon.com>
Co-authored-by: Zelda Hessler <zhessler@amazon.com>
  • Loading branch information
3 people committed Sep 7, 2023
1 parent be9093c commit 8a3b8f3
Show file tree
Hide file tree
Showing 3 changed files with 15 additions and 4 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,9 @@ jobs:
with:
toolchain: ${{ env.rust_version }}
components: ${{ env.rust_toolchain_components }}
# To fix OpenSSL not found on Windows: https://github.com/sfackler/rust-openssl/issues/1542
- run: echo "VCPKG_ROOT=$env:VCPKG_INSTALLATION_ROOT" | Out-File -FilePath $env:GITHUB_ENV -Append
- run: vcpkg install openssl:x64-windows-static-md
- name: Run tests
shell: bash
run: |
Expand Down
6 changes: 6 additions & 0 deletions rust-runtime/aws-smithy-client/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ hyper = { version = "0.14.26", default-features = false, optional = true }
# https://github.com/rust-lang/cargo/issues/1596
hyper-rustls = { version = "0.24", optional = true, features = ["rustls-native-certs", "http2"] }
hyper-tls = { version = "0.5.0", optional = true }
# This forces a more recent version of `openssl-sys` to be brought in.
# Without this `cargo minimal-versions check` would fail in a container whose base image is `amazonlinux:2023`
# with the failure symptom: https://github.com/sfackler/rust-openssl/issues/1724
openssl = { version = "0.10.52", optional = true }
rustls = { version = "0.21.1", optional = true }
lazy_static = { version = "1", optional = true }
pin-project-lite = "0.2.7"
Expand All @@ -45,6 +49,8 @@ tracing = "0.1"
[dev-dependencies]
aws-smithy-async = { path = "../aws-smithy-async", features = ["rt-tokio"] }
hyper-tls = { version = "0.5.0" }
# Dependency on `openssl` above needs to be repeated here.
openssl = "0.10.52"
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1.23.1", features = ["full", "test-util"] }
Expand Down
10 changes: 6 additions & 4 deletions tools/ci-build/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,15 @@

# This is the base Docker build image used by CI

ARG base_image=public.ecr.aws/amazonlinux/amazonlinux:2
ARG base_image=public.ecr.aws/amazonlinux/amazonlinux:2023
ARG rust_stable_version=1.70.0
ARG rust_nightly_version=nightly-2023-05-31

FROM ${base_image} AS bare_base_image
RUN yum -y updateinfo

FROM bare_base_image as musl_toolchain
RUN yum -y install tar gzip gcc make
RUN yum -y install --allowerasing tar gzip gcc make
RUN curl https://musl.libc.org/releases/musl-1.2.3.tar.gz -o musl-1.2.3.tar.gz \
&& ls \
&& tar xvzf musl-1.2.3.tar.gz \
Expand All @@ -30,7 +30,7 @@ ENV RUSTUP_HOME=/opt/rustup \
PATH=/opt/cargo/bin/:${PATH} \
CARGO_INCREMENTAL=0
WORKDIR /root
RUN yum -y install \
RUN yum -y install --allowerasing \
autoconf \
automake \
binutils \
Expand All @@ -41,6 +41,7 @@ RUN yum -y install \
git \
make \
openssl-devel \
perl \
pkgconfig \
tar \
xz && \
Expand Down Expand Up @@ -153,12 +154,13 @@ FROM bare_base_image AS final_image
ARG rust_stable_version
ARG rust_nightly_version
RUN set -eux; \
yum -y install \
yum -y install --allowerasing \
bc \
ca-certificates \
clang \
gcc \
git \
glibc-langpack-en \
java-11-amazon-corretto-headless \
make \
openssl-devel \
Expand Down

0 comments on commit 8a3b8f3

Please sign in to comment.