New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Write TLS session keys to $SSLKEYLOGFILE #11614

Open
miroR opened this Issue Jan 5, 2017 · 5 comments

Comments

Projects
None yet
3 participants
@miroR

miroR commented Jan 5, 2017

It's simple. While this is may Youtube-dl:
$ youtube-dl --version
2016.12.22-gentoo_no_offensive_sites
$
I prepared a page telling what my desired feature would be, by comparison with
what the Wget does. And I don't know the details how, I'm just a user.

Pls. see:
https://www.croatiafidelis.hr/foss/cap/cap-170105_wget-ssl/

and notice that Wget now decrypts SSL traffic, since you can decrypt the:
https://www.croatiafidelis.hr/foss/cap/cap-170105_wget-ssl/dump_170105_1733_g0n.pcap
by using the two effemeral keys in the
https://www.croatiafidelis.hr/foss/cap/cap-170105_wget-ssl/dump_170105_1733_g0n_SSLKEYLOGFILE.txt

such as by downloading those and issuing:

$ wireshark -o "ssl.keylog_file: dump_170105_1733_g0n_SSLKEYLOGFILE.txt" \
dump_170105_1733_g0n.pcap

My wish is that Youtube-dl would do that too! Thanks for your kind
consideration!

@dstftw

This comment has been minimized.

Show comment
Hide comment
@dstftw

dstftw Jan 5, 2017

Collaborator

So, are you requesting youtube-dl to be able to write TLS session keys in a file pointed by SSLKEYLOGFILE in order to able to decrypt caps later in wireshark?

Collaborator

dstftw commented Jan 5, 2017

So, are you requesting youtube-dl to be able to write TLS session keys in a file pointed by SSLKEYLOGFILE in order to able to decrypt caps later in wireshark?

@miroR

This comment has been minimized.

Show comment
Hide comment
@miroR

miroR Jan 5, 2017

Yes, I believe, that is what Wget has recently started doing, IIUC.
(IIUC: previously it was not the case! Now it is, as anybody can see, also with, say tshark-streams.sh from my:
https://github.com/miroR/tshark-streams repo, also, say:
$ tshark-streams.sh -r dump_170105_1733_g0n.pcap -k dump_170105_1733_g0n_SSLKEYLOGFILE.txt -Y 'tcp.stream==5'
and then:
$ cat dump_170105_1733_g0n_s005-ssl.txt | tail -450 | head -449 > dump_170105_1733_g0n_s005-ssl.html
and anybody would get:
$ <your-browser> dump_170105_1733_g0n_s005-ssl.html

that http-over-tls RFC from the video on that www.CroatiaFidelis.hr page that I linked above.
Would be terribly interested to know how Wget does it, but programming real is still overkill for me...

Wget surely are fine and capable people, but there are capable people here, I'm sure Youtube-dl devs can do it...

miroR commented Jan 5, 2017

Yes, I believe, that is what Wget has recently started doing, IIUC.
(IIUC: previously it was not the case! Now it is, as anybody can see, also with, say tshark-streams.sh from my:
https://github.com/miroR/tshark-streams repo, also, say:
$ tshark-streams.sh -r dump_170105_1733_g0n.pcap -k dump_170105_1733_g0n_SSLKEYLOGFILE.txt -Y 'tcp.stream==5'
and then:
$ cat dump_170105_1733_g0n_s005-ssl.txt | tail -450 | head -449 > dump_170105_1733_g0n_s005-ssl.html
and anybody would get:
$ <your-browser> dump_170105_1733_g0n_s005-ssl.html

that http-over-tls RFC from the video on that www.CroatiaFidelis.hr page that I linked above.
Would be terribly interested to know how Wget does it, but programming real is still overkill for me...

Wget surely are fine and capable people, but there are capable people here, I'm sure Youtube-dl devs can do it...

@dstftw dstftw changed the title from SSL decryption via the $SSLKEYLOGFILE setup? to Write TLS session keys to $SSLKEYLOGFILE Jan 5, 2017

@dstftw dstftw added the request label Jan 5, 2017

@yan12125

This comment has been minimized.

Show comment
Hide comment
@yan12125

yan12125 Jan 6, 2017

Collaborator

What's the format of keylog files? In youtube-dl multiple hosts are involved in each invocation, and, as SSL session resumption (http://bugs.python.org/issue19500) is not implemented in youtube-dl, there are multiple session keys even for the same host. How to generate a keylog file for such scenarios?

UPDATE: the format can be found at https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format

Collaborator

yan12125 commented Jan 6, 2017

What's the format of keylog files? In youtube-dl multiple hosts are involved in each invocation, and, as SSL session resumption (http://bugs.python.org/issue19500) is not implemented in youtube-dl, there are multiple session keys even for the same host. How to generate a keylog file for such scenarios?

UPDATE: the format can be found at https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format

@yan12125

This comment has been minimized.

Show comment
Hide comment
@yan12125

yan12125 Jan 6, 2017

Collaborator

By the way, Python does not support exposing session keys yet. A patched Python is necessary.

References:
https://www.openssl.org/docs/manmaster/man3/SSL_SESSION_get_master_key.html
https://hg.python.org/cpython/file/tip/Modules/_ssl.c#l4427

Collaborator

yan12125 commented Jan 6, 2017

By the way, Python does not support exposing session keys yet. A patched Python is necessary.

References:
https://www.openssl.org/docs/manmaster/man3/SSL_SESSION_get_master_key.html
https://hg.python.org/cpython/file/tip/Modules/_ssl.c#l4427

@yan12125

This comment has been minimized.

Show comment
Hide comment
@yan12125

yan12125 Jan 7, 2017

Collaborator

Depends on openssl/openssl#1646, too

Collaborator

yan12125 commented Jan 7, 2017

Depends on openssl/openssl#1646, too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment