Custom ALPN protocol

[mkroman]

I'm working on a project that communicates with a Teleport auth service.

Their protocol is using gRPC, for which I am using Tonic.

They have a feature where they proxy multiple protocols into a single TLS connection based on SNI and ALPN values: https://github.com/gravitational/teleport/blob/master/rfd/0039-sni-alpn-teleport-proxy-routing.md

So I'm trying to create a HTTP client for Tonic based on this example. However, I noticed that hyper-rustls won't let me set the protocol as the ClientConfig::alpn_protocols field is required to be empty during construction:

hyper-rustls/src/connector/builder.rs

Lines 40 to 45 in 0a34dd9

	/// The [`alpn_protocols`](ClientConfig::alpn_protocols) field is
	/// required to be empty (or the function will panic) and will be
	/// rewritten to match the enabled schemes (see
	/// [`enable_http1`](ConnectorBuilder::enable_http1),
	/// [`enable_http2`](ConnectorBuilder::enable_http2)) before the
	/// connector is built.

Ideally the assertion should be removed, but I'm not sure if this creates problems elsewhere. I could also imagine a scenario where one would want http2 communication without the h2 protocol being part of the ALPN (which enable_http2() sets), but I'm not even sure if Teleport supports that.

[djc]

Sorry, I don't have time to go read through the Teleport documentation in detail. What do you want/need to send in the ALPN protocols to enable the Teleport use case? I'd be happy to enable this use case but would still like the configuration process to be as misuse-resistant as possible.

[mkroman]

I don't think I'll continue working on this project, so I'm just gonna close this issue and open a new one it if it becomes relevant again.

I do however think that my initial goal may have been misguided, as I wanted hyper-rustls to handle the ALPN, when it may be more appropriate to do that with just rustls and then wrap the stream with hyper-rustls when necessary.

[rcanderson23]

Sorry to bump an old, closed thread but I started working on exactly this a while back and recently have time to pick it back up and ran into the same issue. Not sure if this a hyper-rustls problem to support but figured it was worth asking. Essentially the proxy server is expecting a specific ALPN to route to the auth server behind it. This was the code I wrote to get it working:

async fn connect() -> Result<(), Box<dyn std::error::Error>> {
    let ca = load_ca()?;
    let user_cert = load_user_cert()?;
    let user_key = load_private_key()?;

    let mut roots = RootCertStore::empty();

    roots.add_parsable_certificates(&ca);

    let mut tls = ClientConfig::builder()
        .with_safe_defaults()
        .with_root_certificates(roots)
        .with_client_auth_cert(vec![user_cert], user_key)?;

    let encoded_cluster = hex::encode("teleport.carsonanderson.net");

    // Order here matters
    tls.alpn_protocols
        .push(format!("teleport-auth@{}.teleport.cluster.local", encoded_cluster).into());
    tls.alpn_protocols.push("h2".into());

    let stream = tokio::net::TcpStream::connect("teleport.carsonanderson.net:443").await?;

    let connector = TlsConnector::from(Arc::new(tls));

    // being presented the auth server cert which is teleport.cluster.local
    let domain = tokio_rustls::rustls::ServerName::try_from("teleport.cluster.local")?;
    let stream = connector.connect(domain, stream).await?;
    let (sender, conn) = hyper::client::conn::Builder::new()
        .http2_only(true)
        .handshake(stream)
        .await?;
    tokio::spawn(async {
        if let Err(err) = conn.await {
            eprintln!("error servicing connection: {err}");
        }
    });
    let svc = ServiceBuilder::new().buffer(256).service(sender);

    let mut client = teleport::auth::auth_service_client::AuthServiceClient::with_origin(
        svc,
        Uri::from_static("https://teleport.carsonanderson.net"),
    );
    let users = client
        .get_users(tonic::Request::new(teleport::auth::GetUsersRequest {
            with_secrets: false,
        }))
        .await?;
    let mut users = users.into_inner();
    while let Some(user) = users.message().await? {
        println!("{}", user.metadata.unwrap().name);
    }

    Ok(())
}

The teleport specific ALPN and h2 are both required and ordering there matters. Removal of the h2 header results in an error Error: Custom { kind: InvalidData, error: AlertReceived(NoApplicationProtocol) }.