Write TLS session keys to $SSLKEYLOGFILE #11614

Open
miroR opened this Issue Jan 5, 2017 · 5 comments

Projects

None yet

3 participants

@miroR
miroR commented Jan 5, 2017 edited

It's simple. While this is may Youtube-dl:
$ youtube-dl --version
2016.12.22-gentoo_no_offensive_sites
$
I prepared a page telling what my desired feature would be, by comparison with
what the Wget does. And I don't know the details how, I'm just a user.

Pls. see:
http://www.croatiafidelis.hr/foss/cap/cap-170105_wget-ssl/

and notice that Wget now decrypts SSL traffic, since you can decrypt the:
http://www.croatiafidelis.hr/foss/cap/cap-170105_wget-ssl/dump_170105_1733_g0n.pcap
by using the two effemeral keys in the
http://www.croatiafidelis.hr/foss/cap/cap-170105_wget-ssl/dump_170105_1733_g0n_SSLKEYLOGFILE.txt

such as by downloading those and issuing:

$ wireshark -o "ssl.keylog_file: dump_170105_1733_g0n_SSLKEYLOGFILE.txt" \
dump_170105_1733_g0n.pcap

My wish is that Youtube-dl would do that too! Thanks for your kind
consideration!

@dstftw
Collaborator
dstftw commented Jan 5, 2017

So, are you requesting youtube-dl to be able to write TLS session keys in a file pointed by SSLKEYLOGFILE in order to able to decrypt caps later in wireshark?

@miroR
miroR commented Jan 5, 2017 edited

Yes, I believe, that is what Wget has recently started doing, IIUC.
(IIUC: previously it was not the case! Now it is, as anybody can see, also with, say tshark-streams.sh from my:
https://github.com/miroR/tshark-streams repo, also, say:
$ tshark-streams.sh -r dump_170105_1733_g0n.pcap -k dump_170105_1733_g0n_SSLKEYLOGFILE.txt -Y 'tcp.stream==5'
and then:
$ cat dump_170105_1733_g0n_s005-ssl.txt | tail -450 | head -449 > dump_170105_1733_g0n_s005-ssl.html
and anybody would get:
$ <your-browser> dump_170105_1733_g0n_s005-ssl.html

that http-over-tls RFC from the video on that www.CroatiaFidelis.hr page that I linked above.
Would be terribly interested to know how Wget does it, but programming real is still overkill for me...

Wget surely are fine and capable people, but there are capable people here, I'm sure Youtube-dl devs can do it...

@dstftw dstftw changed the title from SSL decryption via the $SSLKEYLOGFILE setup? to Write TLS session keys to $SSLKEYLOGFILE Jan 5, 2017
@dstftw dstftw added the request label Jan 5, 2017
@yan12125
Collaborator
yan12125 commented Jan 6, 2017 edited

What's the format of keylog files? In youtube-dl multiple hosts are involved in each invocation, and, as SSL session resumption (http://bugs.python.org/issue19500) is not implemented in youtube-dl, there are multiple session keys even for the same host. How to generate a keylog file for such scenarios?

UPDATE: the format can be found at https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format

@yan12125
Collaborator
yan12125 commented Jan 6, 2017

By the way, Python does not support exposing session keys yet. A patched Python is necessary.

References:
https://www.openssl.org/docs/manmaster/man3/SSL_SESSION_get_master_key.html
https://hg.python.org/cpython/file/tip/Modules/_ssl.c#l4427

@yan12125
Collaborator
yan12125 commented Jan 7, 2017

Depends on openssl/openssl#1646, too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment