Write TLS session keys to $SSLKEYLOGFILE

[miroR]

It's simple. While this is may Youtube-dl:
$ youtube-dl --version
2016.12.22-gentoo_no_offensive_sites
$
I prepared a page telling what my desired feature would be, by comparison with
what the Wget does. And I don't know the details how, I'm just a user.

Pls. see:
https://www.croatiafidelis.hr/foss/cap/cap-170105_wget-ssl/

and notice that Wget now decrypts SSL traffic, since you can decrypt the:
https://www.croatiafidelis.hr/foss/cap/cap-170105_wget-ssl/dump_170105_1733_g0n.pcap
by using the two effemeral keys in the
https://www.croatiafidelis.hr/foss/cap/cap-170105_wget-ssl/dump_170105_1733_g0n_SSLKEYLOGFILE.txt

such as by downloading those and issuing:

$ wireshark -o "ssl.keylog_file: dump_170105_1733_g0n_SSLKEYLOGFILE.txt" \
dump_170105_1733_g0n.pcap

My wish is that Youtube-dl would do that too! Thanks for your kind
consideration!

[dstftw]

So, are you requesting youtube-dl to be able to write TLS session keys in a file pointed by SSLKEYLOGFILE in order to able to decrypt caps later in wireshark?

[miroR]

Yes, I believe, that is what Wget has recently started doing, IIUC.
(IIUC: previously it was not the case! Now it is, as anybody can see, also with, say tshark-streams.sh from my:
https://github.com/miroR/tshark-streams repo, also, say:
$ tshark-streams.sh -r dump_170105_1733_g0n.pcap -k dump_170105_1733_g0n_SSLKEYLOGFILE.txt -Y 'tcp.stream==5'
and then:
$ cat dump_170105_1733_g0n_s005-ssl.txt | tail -450 | head -449 > dump_170105_1733_g0n_s005-ssl.html
and anybody would get:
$ <your-browser> dump_170105_1733_g0n_s005-ssl.html

that http-over-tls RFC from the video on that www.CroatiaFidelis.hr page that I linked above.
Would be terribly interested to know how Wget does it, but programming real is still overkill for me...

Wget surely are fine and capable people, but there are capable people here, I'm sure Youtube-dl devs can do it...

[yan12125]

What's the format of keylog files? In youtube-dl multiple hosts are involved in each invocation, and, as SSL session resumption (http://bugs.python.org/issue19500) is not implemented in youtube-dl, there are multiple session keys even for the same host. How to generate a keylog file for such scenarios?

UPDATE: the format can be found at https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format

[yan12125]

By the way, Python does not support exposing session keys yet. A patched Python is necessary.

References:
https://www.openssl.org/docs/manmaster/man3/SSL_SESSION_get_master_key.html
https://hg.python.org/cpython/file/tip/Modules/_ssl.c#l4427

[yan12125]

Depends on openssl/openssl#1646, too

[luiso1979]

There is any updates regarding this issue? Assuming that I'm using the patched Python and the right openssl, this is a solvable problem?

[yan12125]

Looks like with Python 3.8 and OpenSSL 1.1.1, keys will be logged if you specify the environment variable $SSLKEYLOGFILE [1][2]. Closing as there is nothing to do in youtube-dl.

[1] python/cpython@c7f7069
[2] openssl/openssl#2287 (comment)