[Question] Logging TLS 1.3 traffic secrets

[Benedicht]

I'm trying to write the traffic secrets to an SSL keylog file so i could see my traffic in Wireshark.

With TLS 1.2 i had this modified CalculateKeyBlock function in TlsUtilities.cs:

internal static byte[] CalculateKeyBlock(TlsContext context, int size)
{
    SecurityParameters securityParameters = context.SecurityParameters;
    byte[] master_secret = securityParameters.MasterSecret;
    byte[] seed = Concat(securityParameters.ServerRandom, securityParameters.ClientRandom);

    if (IsSsl(context))
        return CalculateKeyBlock_Ssl(master_secret, seed, size);

#if UNITY_EDITOR
    // https://www.m00nie.com/2015/05/decrypt-https-ssltls-with-wireshark/
    // https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format
    string SSLKEYLOGFILE = Environment.GetEnvironmentVariable("SSLKEYLOGFILE", EnvironmentVariableTarget.User);
    if (!string.IsNullOrEmpty(SSLKEYLOGFILE))
        using (var writer = new StreamWriter(System.IO.File.Open(SSLKEYLOGFILE, FileMode.Append, FileAccess.Write, FileShare.ReadWrite)))
            writer.Write(string.Format("# Generated by BestHTTP\r\nCLIENT_RANDOM {0} {1}",
                BouncyCastle.Utilities.Encoders.Hex.ToHexString(securityParameters.ClientRandom),
                BouncyCastle.Utilities.Encoders.Hex.ToHexString(master_secret))); 
#endif

    return PRF(context, master_secret, ExporterLabel.key_expansion, seed, size);
}

I don't know whether it was the best place to add, but it worked for my use case so i was happy with it.

However, I'm in trouble with TLS 1.3. For TLS 1.3 there are 7 labels (CLIENT_EARLY_TRAFFIC_SECRET, CLIENT_HANDSHAKE_TRAFFIC_SECRET, ..., EXPORTER_SECRET) and my understanding is that i should use at least 5 of them.

My guess is all the required fields can be found in the SecurityParameters class (that i can access through TlsClientProtocol's m_tlsClientContext.SecurityParameters), but i'm not sure what field matches what label (except for ClientRandom :) ).

To summ it up, I have two questions:
1.) What fields can be used for the following labels:

• CLIENT_EARLY_TRAFFIC_SECRET: the hex-encoded early traffic secret for the client side (for TLS 1.3)
• CLIENT_HANDSHAKE_TRAFFIC_SECRET: the hex-encoded handshake traffic secret for the client side (for TLS 1.3)
• SERVER_HANDSHAKE_TRAFFIC_SECRET: the hex-encoded handshake traffic secret for the server side (for TLS 1.3)
• CLIENT_TRAFFIC_SECRET_0: the first hex-encoded application traffic secret for the client side (for TLS 1.3)
• SERVER_TRAFFIC_SECRET_0: the first hex-encoded application traffic secret for the server side (for TLS 1.3)
• EARLY_EXPORTER_SECRET: the hex-encoded early exporter secret (for TLS 1.3).
• EXPORTER_SECRET: the hex-encoded exporter secret (for TLS 1.3)

2.) What could be the best place to write these out to the SSL Keylog file?

Thanks in advance!

[peterdettman]

TLS 1.3 support is only in the new Org.BouncyCastle.Tls namespace and not in the old Org.BouncyCastle.Crypto.Tls (the API changes aren't massive).

The traffic secret fields are used for both handshake and application traffic (i.e. they get overwritten). We don't currently support early traffic, so never calculate those secrets. Early exporter secret I am just noticing is not set, although I don't think there's any reason we couldn't be doing so; I need to review.

So I expect you're interested in these fields of SecurityParameters:

internal TlsSecret m_earlyExporterMasterSecret = null;
internal TlsSecret m_exporterMasterSecret = null;
internal TlsSecret m_trafficSecretClient = null;
internal TlsSecret m_trafficSecretServer = null;

I would think logging could most easily be added in the TlsUtilities methods that switch phases:

Establish13PhaseEarly
Establish13PhaseHandshake
Establish13PhaseApplication

We could probably look at providing a proper API for supplying this sort of information at some point.

Anyway, let me know how you go.

[Benedicht]

Thanks for the quick reply, i could make it work!

As mentioned early traffic isn't supported and indeed Establish13PhaseEarly wasn't called in my tests, so i changed only Establish13PhaseHandshake and Establish13PhaseApplication.

Pasting the code here, maybe it will be helpful for someone:

internal static void Establish13PhaseApplication(TlsContext context, byte[] serverFinishedTranscriptHash,
    RecordStream recordStream)
{
    SecurityParameters securityParameters = context.SecurityParameters;
    TlsSecret phaseSecret = securityParameters.MasterSecret;

    Establish13TrafficSecrets(context, serverFinishedTranscriptHash, phaseSecret, "c ap traffic",
        "s ap traffic", recordStream);

    securityParameters.m_exporterMasterSecret = DeriveSecret(securityParameters, phaseSecret, "exp master",
        serverFinishedTranscriptHash);

    KeyLogFileWriter.WriteLabel(Labels.CLIENT_TRAFFIC_SECRET_0, securityParameters.ClientRandom, securityParameters.TrafficSecretClient);
    KeyLogFileWriter.WriteLabel(Labels.SERVER_TRAFFIC_SECRET_0, securityParameters.ClientRandom, securityParameters.TrafficSecretServer);
    KeyLogFileWriter.WriteLabel(Labels.EXPORTER_SECRET, securityParameters.ClientRandom, securityParameters.ExporterMasterSecret);
}

internal static void Establish13PhaseHandshake(TlsContext context, byte[] serverHelloTranscriptHash,
    RecordStream recordStream)
{
    SecurityParameters securityParameters = context.SecurityParameters;
    TlsSecret phaseSecret = securityParameters.HandshakeSecret;

    Establish13TrafficSecrets(context, serverHelloTranscriptHash, phaseSecret, "c hs traffic", "s hs traffic",
        recordStream);

    KeyLogFileWriter.WriteLabel(Labels.CLIENT_HANDSHAKE_TRAFFIC_SECRET, securityParameters.ClientRandom, securityParameters.TrafficSecretClient);
    KeyLogFileWriter.WriteLabel(Labels.SERVER_HANDSHAKE_TRAFFIC_SECRET, securityParameters.ClientRandom, securityParameters.TrafficSecretServer);
            
    securityParameters.m_baseKeyClient = securityParameters.TrafficSecretClient;
    securityParameters.m_baseKeyServer = securityParameters.TrafficSecretServer;
}

My KeyLogFileWriter:

internal enum Labels
{
    CLIENT_EARLY_TRAFFIC_SECRET,
    CLIENT_HANDSHAKE_TRAFFIC_SECRET,
    SERVER_HANDSHAKE_TRAFFIC_SECRET,
    CLIENT_TRAFFIC_SECRET_0,
    SERVER_TRAFFIC_SECRET_0,
    EARLY_EXPORTER_SECRET,
    EXPORTER_SECRET
}

internal static class KeyLogFileWriter
{
    private static string GetKeylogFileName() => Environment.GetEnvironmentVariable("SSLKEYLOGFILE", EnvironmentVariableTarget.User);

    [Conditional("UNITY_EDITOR")]
    public static void WriteLabel(Labels label, byte[] clientRandom, TlsSecret secret)
    {
        string SSLKEYLOGFILE = GetKeylogFileName();

        if (!string.IsNullOrEmpty(SSLKEYLOGFILE) && clientRandom != null && secret != null)
            using (var writer = new StreamWriter(System.IO.File.Open(SSLKEYLOGFILE, FileMode.Append, FileAccess.Write, FileShare.ReadWrite)))
                writer.WriteLine($"{label} {Hex.ToHexString(clientRandom)} {Hex.ToHexString((secret as AbstractTlsSecret).CopyData())}");
    }

    [Conditional("UNITY_EDITOR")]
    public static void WriteLabel(Labels label, SecurityParameters securityParameters)
    {
        string SSLKEYLOGFILE = GetKeylogFileName();
        if (!string.IsNullOrEmpty(SSLKEYLOGFILE))
            using (var writer = new StreamWriter(System.IO.File.Open(SSLKEYLOGFILE, FileMode.Append, FileAccess.Write, FileShare.ReadWrite)))
            {
                TlsSecret secret = null;
                switch (label)
                {
                    case Labels.CLIENT_EARLY_TRAFFIC_SECRET: break;
                    case Labels.CLIENT_HANDSHAKE_TRAFFIC_SECRET: secret = securityParameters.TrafficSecretClient; break;
                    case Labels.SERVER_HANDSHAKE_TRAFFIC_SECRET: secret = securityParameters.TrafficSecretServer; break;
                    case Labels.CLIENT_TRAFFIC_SECRET_0: secret = securityParameters.TrafficSecretClient; break;
                    case Labels.SERVER_TRAFFIC_SECRET_0: secret = securityParameters.TrafficSecretServer; break;
                    case Labels.EARLY_EXPORTER_SECRET: break;
                    case Labels.EXPORTER_SECRET: secret = securityParameters.ExporterMasterSecret; break;
                }

                if (secret != null)
                    writer.Write($"\r\n{label} {Hex.ToHexString(securityParameters.ClientRandom)} {Hex.ToHexString((secret as AbstractTlsSecret).CopyData())}");
            }
    }
}