I would like to compile my rust application to a statically linked binary so that I can use it inside a FROM scratch docker image. However, it seems like this crate is incompatible with musl.
To reproduce this, I have create a somewhat lengthy dockerfile that attempts to run a simple application that smoke-tests ktls. You can find it here:
Dockerfile
FROM rust:1-bookworm AS base
WORKDIR /workspace
RUN apt-get update \
&& apt-get install -y --no-install-recommends pkg-config musl-tools \
&& rm -rf /var/lib/apt/lists/*
RUN rustup target add x86_64-unknown-linux-musl
FROM base AS deps
COPY Cargo.lock Cargo.lock
COPY ktls/Cargo.toml ktls/Cargo.toml
COPY ktls-sys/Cargo.toml ktls-sys/Cargo.toml
RUN cat > Cargo.toml << 'TOML'
[workspace]
members = ["ktls", "ktls-sys", "app"]
resolver = "2"
[workspace.package]
edition = "2021"
rust-version = "1.75.0"
authors = ["Amos Wenger <amos@bearcove.net>"]
license = "MIT OR Apache-2.0"
readme = "README.md"
repository = "https://github.com/rustls/ktls"
TOML
RUN mkdir -p app/src && cat > app/Cargo.toml << 'TOML'
[package]
name = "ktls-smoke"
version = "0.1.0"
edition = "2021"
[dependencies]
ktls = { path = "../ktls", default-features = false, features = ["ring", "tls12"] }
tokio = { version = "1", features = ["full"] }
tokio-rustls = { version = "0.26", default-features = false, features = ["ring", "tls12"] }
rustls = { version = "0.23", default-features = false, features = ["ring", "tls12"] }
rcgen = "0.13"
TOML
RUN mkdir -p ktls/src ktls-sys/src && \
touch ktls/src/lib.rs ktls-sys/src/lib.rs && \
printf 'fn main() {}\n' > app/src/main.rs
RUN cargo build --release -p ktls-smoke
RUN cargo build --release --target x86_64-unknown-linux-musl -p ktls-smoke
# LLM-generated ktls smoke test file for a minimal repro:
RUN cat > app/src/main.rs << 'RUST'
//! Minimal end-to-end kTLS smoke test:
//! - generates a self-signed certificate
//! - starts a TLS server with kernel TLS offload enabled (kTLS)
//! - connects a plain TLS client
//! - exchanges a round-trip payload and verifies it
use std::sync::Arc;
use ktls::CorkStream;
use rcgen::generate_simple_self_signed;
use rustls::{ClientConfig, RootCertStore, ServerConfig};
use tokio::{
io::{AsyncReadExt, AsyncWriteExt},
net::{TcpListener, TcpStream},
};
use tokio_rustls::TlsConnector;
const PING: &[u8] = b"ping from ktls client";
const PONG: &[u8] = b"pong from ktls server";
#[tokio::main]
async fn main() {
rustls::crypto::ring::default_provider()
.install_default()
.expect("failed to install ring crypto provider");
let cert = generate_simple_self_signed(vec!["localhost".to_string()])
.expect("cert generation failed");
let mut server_cfg = ServerConfig::builder()
.with_no_client_auth()
.with_single_cert(
vec![cert.cert.der().clone()],
rustls::pki_types::PrivatePkcs8KeyDer::from(cert.key_pair.serialize_der()).into(),
)
.expect("server config error");
server_cfg.enable_secret_extraction = true;
let acceptor = tokio_rustls::TlsAcceptor::from(Arc::new(server_cfg));
let ln = TcpListener::bind("127.0.0.1:0").await.expect("bind failed");
let addr = ln.local_addr().unwrap();
let server = tokio::spawn(async move {
let (tcp, _) = ln.accept().await.expect("accept failed");
// CorkStream must wrap the TCP socket before the TLS handshake so
// that ktls can drain the rustls internal buffer at the record boundary.
let tls = acceptor
.accept(CorkStream::new(tcp))
.await
.expect("TLS handshake failed");
let mut stream = ktls::config_ktls_server(tls)
.await
.expect("kTLS offload setup failed — is the tls kernel module loaded?");
let mut buf = vec![0u8; PING.len()];
stream.read_exact(&mut buf).await.expect("server read failed");
assert_eq!(buf, PING, "server received unexpected payload");
stream.write_all(PONG).await.expect("server write failed");
stream.flush().await.expect("server flush failed");
stream.shutdown().await.expect("server shutdown failed");
});
let mut root_store = RootCertStore::empty();
root_store.add(cert.cert.der().clone()).unwrap();
let client_cfg = ClientConfig::builder()
.with_root_certificates(root_store)
.with_no_client_auth();
let connector = TlsConnector::from(Arc::new(client_cfg));
let tcp = TcpStream::connect(addr).await.expect("connect failed");
let mut tls = connector
.connect("localhost".try_into().unwrap(), tcp)
.await
.expect("client TLS handshake failed");
tls.write_all(PING).await.expect("client write failed");
tls.flush().await.expect("client flush failed");
let mut buf = vec![0u8; PONG.len()];
tls.read_exact(&mut buf).await.expect("client read failed");
assert_eq!(buf, PONG, "client received unexpected payload");
server.await.unwrap();
eprintln!("ktls smoke test passed");
}
RUST
FROM deps AS build-gnu
COPY ktls/src/ ktls/src/
COPY ktls-sys/src/ ktls-sys/src/
# mark all files as dirty
RUN find ktls ktls-sys app -name '*.rs' | xargs touch
RUN cargo build --release -p ktls-smoke
FROM deps AS build-musl
COPY ktls/src/ ktls/src/
COPY ktls-sys/src/ ktls-sys/src/
# mark all files as dirty
RUN find ktls ktls-sys app -name '*.rs' | xargs touch
RUN cargo build --release --target x86_64-unknown-linux-musl -p ktls-smoke
FROM scratch
COPY --from=build-musl /workspace/target/x86_64-unknown-linux-musl/release/ktls-smoke /ktls-smoke
ENTRYPOINT ["/ktls-smoke"]
When I build it, I see this error:
$ docker build --progress=plain -t ktls-repro .
#0 building with "default" instance using docker driver
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 5.67kB done
#1 DONE 0.0s
#2 resolve image config for docker-image://docker.io/docker/dockerfile:1
#2 DONE 2.8s
#3 docker-image://docker.io/docker/dockerfile:1@sha256:2780b5c3bab67f1f76c781860de469442999ed1a0d7992a5efdf2cffc0e3d769
#3 CACHED
#4 [internal] load metadata for docker.io/library/rust:1-bookworm
#4 DONE 1.3s
#5 [internal] load .dockerignore
#5 transferring context: 2B done
#5 DONE 0.0s
#6 [base 1/4] FROM docker.io/library/rust:1-bookworm@sha256:adab7941580c74513aa3347f2d2a1f975498280743d29ec62978ba12e3540d3a
#6 DONE 0.0s
#7 [internal] load build context
#7 transferring context: 11.27kB done
#7 DONE 0.0s
#8 [deps 4/9] RUN cat > Cargo.toml << 'TOML'
#8 CACHED
#9 [deps 6/9] RUN mkdir -p ktls/src ktls-sys/src && touch ktls/src/lib.rs ktls-sys/src/lib.rs && printf 'fn main() {}\n' > app/src/main.rs
#9 CACHED
#10 [deps 7/9] RUN cargo build --release -p ktls-smoke
#10 CACHED
#11 [base 4/4] RUN rustup target add x86_64-unknown-linux-musl
#11 CACHED
#12 [deps 8/9] RUN cargo build --release --target x86_64-unknown-linux-musl -p ktls-smoke
#12 CACHED
#13 [deps 1/9] COPY Cargo.lock Cargo.lock
#13 CACHED
#14 [base 2/4] WORKDIR /workspace
#14 CACHED
#15 [base 3/4] RUN apt-get update && apt-get install -y --no-install-recommends pkg-config musl-tools && rm -rf /var/lib/apt/lists/*
#15 CACHED
#16 [deps 3/9] COPY ktls-sys/Cargo.toml ktls-sys/Cargo.toml
#16 CACHED
#17 [deps 2/9] COPY ktls/Cargo.toml ktls/Cargo.toml
#17 CACHED
#18 [deps 5/9] RUN mkdir -p app/src && cat > app/Cargo.toml << 'TOML'
#18 CACHED
#19 [deps 9/9] RUN cat > app/src/main.rs << 'RUST'
#19 CACHED
#20 [build-musl 1/4] COPY ktls/src/ ktls/src/
#20 DONE 0.0s
#21 [build-musl 2/4] COPY ktls-sys/src/ ktls-sys/src/
#21 DONE 0.0s
#22 [build-musl 3/4] RUN find ktls ktls-sys app -name '*.rs' | xargs touch
#22 DONE 0.2s
#23 [build-musl 4/4] RUN cargo build --release --target x86_64-unknown-linux-musl -p ktls-smoke
#23 0.245 Compiling ktls v6.0.2 (/workspace/ktls)
#23 0.369 error[E0063]: missing field `__pad1` in initializer of `cmsghdr`
#23 0.369 --> ktls/src/ffi.rs:255:18
#23 0.369 |
#23 0.369 255 | hdr: libc::cmsghdr {
#23 0.369 | ^^^^^^^^^^^^^ missing `__pad1`
#23 0.369
#23 0.370 error: cannot construct `msghdr` with struct literal syntax due to private fields
#23 0.370 --> ktls/src/ffi.rs:275:15
#23 0.370 |
#23 0.370 275 | let msg = libc::msghdr {
#23 0.370 | ^^^^^^^^^^^^
#23 0.370 |
#23 0.370 = note: ...and other private fields `__pad1` and `__pad2` that were not provided
#23 0.370
#23 0.434 For more information about this error, try `rustc --explain E0063`.
#23 0.437 error: could not compile `ktls` (lib) due to 2 previous errors
#23 ERROR: process "/bin/sh -c cargo build --release --target x86_64-unknown-linux-musl -p ktls-smoke" did not complete successfully: exit code: 101
------
> [build-musl 4/4] RUN cargo build --release --target x86_64-unknown-linux-musl -p ktls-smoke:
0.370 error: cannot construct `msghdr` with struct literal syntax due to private fields
0.370 --> ktls/src/ffi.rs:275:15
0.370 |
0.370 275 | let msg = libc::msghdr {
0.370 | ^^^^^^^^^^^^
0.370 |
0.370 = note: ...and other private fields `__pad1` and `__pad2` that were not provided
0.370
0.434 For more information about this error, try `rustc --explain E0063`.
0.437 error: could not compile `ktls` (lib) due to 2 previous errors
------
Dockerfile:173
--------------------
171 | RUN find ktls ktls-sys app -name '*.rs' | xargs touch
172 |
173 | >>> RUN cargo build --release --target x86_64-unknown-linux-musl -p ktls-smoke
174 |
175 | FROM scratch
--------------------
ERROR: failed to build: failed to solve: process "/bin/sh -c cargo build --release --target x86_64-unknown-linux-musl -p ktls-smoke" did not complete successfully: exit code: 101From what I can tell, this means that the ktls crate does not compile against the libc structs because they have hidden fields.
I would like to compile my rust application to a statically linked binary so that I can use it inside a
FROM scratchdocker image. However, it seems like this crate is incompatible with musl.To reproduce this, I have create a somewhat lengthy dockerfile that attempts to run a simple application that smoke-tests ktls. You can find it here:
Dockerfile
When I build it, I see this error:
From what I can tell, this means that the ktls crate does not compile against the libc structs because they have hidden fields.