New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TCP fragmentation caused by \x0a sent to socket #1130

Closed
drwetter opened this Issue Sep 21, 2018 · 12 comments

Comments

Projects
None yet
2 participants
@drwetter
Owner

drwetter commented Sep 21, 2018

As noted in the discussion of PR #1113 it seems that printf -- "$data" >&5 2>/dev/null does not do what it is intended. $data contains a ClientHello, 5 is the fd of a socket. "$data" is a ClientHello like

\x16\x03\x01\x2\x00\x01\x00\x1\xfc\x03\x03\x54\x51\x1e\x7a\xde\xad\xbe\xef\x31\x33\x07\x00\x00\x00\x00\x00\xcf\xbd\x39\x04\xcc\x16\x0a\...

Each \x0a like the last one causes a new TCP fragment to begin which can be spotted when using wireshark while running e.g.

testssl.sh --assume-http -p testssl.sh

Starting from the SSLv3 ClientHello the first reassembled packet ends with 0a.

This ticket is for continuing the discussion from #1113 (comment) on

@drwetter drwetter added this to the 3.0 milestone Sep 21, 2018

dcooper16 added a commit to dcooper16/testssl.sh that referenced this issue Sep 21, 2018

Avoid unnecessary '0a' characters in ClientHello
As noted in drwetter#1130, the current implementation of socksend_tls_clienthello() results in packets being fragmented wherever a '0a' character appears in the message. This cannot be avoided, but there are a few places where a '0a' character appears in which the character could easily be replaced:

* In the session_id for a TLSv1.3 ClientHello.
* In the 32-byte client random value
* In any public key sent in the key_share extension

This PR removes those uses of the '0a' character. While this does not do much to address the problem, it does result in a slight reduction in the amount of fragmentation of messages.
@drwetter

This comment has been minimized.

Show comment
Hide comment
@drwetter

drwetter Sep 22, 2018

Owner

dd might be worth looking into it .I am afraid that it is a "feature" of bash

Owner

drwetter commented Sep 22, 2018

dd might be worth looking into it .I am afraid that it is a "feature" of bash

@drwetter

This comment has been minimized.

Show comment
Hide comment
@drwetter

drwetter Sep 23, 2018

Owner

It turned out not to be a feature of bash + sockets but of the internal printf:

prompt> strace -ewrite bash -c 'printf "one\ntwo\n"' >/dev/null
write(1, "one\n", 4)                    = 4
write(1, "two\n", 4)                    = 4
+++ exited with 0 +++
prompt> strace -etrace=write /usr/bin/printf "one\two\n" > /dev/null
write(1, "one\two\n", 7)                = 7
+++ exited with 0 +++

We could also use stdbuf or dd (latter either directly for writing into the socket or as a pipe after any printf). :

prompt> strace -etrace=write bash -c 'stdbuf -o1M printf "one\ntwo\n"  ' > /dev/null
write(1, "one\ntwo\n", 8)               = 8
+++ exited with 0 +++
prompt> bash -c 'printf "one\ntwo\n" | strace -e write,read -o/tmp/log.txt dd status=none bs=1M' > /dev/null
prompt> cat /tmp/log.txt
[..]
read(0, "one\ntwo\n", 1048576)          = 8
write(1, "one\ntwo\n", 8)               = 8
read(0, "", 1048576)                    = 0
+++ exited with 0 +++
Owner

drwetter commented Sep 23, 2018

It turned out not to be a feature of bash + sockets but of the internal printf:

prompt> strace -ewrite bash -c 'printf "one\ntwo\n"' >/dev/null
write(1, "one\n", 4)                    = 4
write(1, "two\n", 4)                    = 4
+++ exited with 0 +++
prompt> strace -etrace=write /usr/bin/printf "one\two\n" > /dev/null
write(1, "one\two\n", 7)                = 7
+++ exited with 0 +++

We could also use stdbuf or dd (latter either directly for writing into the socket or as a pipe after any printf). :

prompt> strace -etrace=write bash -c 'stdbuf -o1M printf "one\ntwo\n"  ' > /dev/null
write(1, "one\ntwo\n", 8)               = 8
+++ exited with 0 +++
prompt> bash -c 'printf "one\ntwo\n" | strace -e write,read -o/tmp/log.txt dd status=none bs=1M' > /dev/null
prompt> cat /tmp/log.txt
[..]
read(0, "one\ntwo\n", 1048576)          = 8
write(1, "one\ntwo\n", 8)               = 8
read(0, "", 1048576)                    = 0
+++ exited with 0 +++

@drwetter drwetter removed the performance label Sep 23, 2018

drwetter added a commit that referenced this issue Sep 23, 2018

Fix TCP fragmentation
As @tomato42 pointed out in #1113 '\x0a' causes the printf buffer
to flush before all data was sent. As a result any '\x0a' in
a ClientHello causes a new TCP fragment.

This commit changes all TCP sockets write to use an external
printf if available which doesn't show this behaviour, see #1130.
It was checked against wireshark.

The external printf was available for Linux, FreeBSD 9 and OpenBSD,
so I do not expect any problems with MacOS X either.

There might be further solutions like 'stdbuf' or 'dd' which
are shown in #1130.
@drwetter

This comment has been minimized.

Show comment
Hide comment
@drwetter

drwetter Sep 23, 2018

Owner

See commit. Solved

Owner

drwetter commented Sep 23, 2018

See commit. Solved

@drwetter drwetter closed this Sep 23, 2018

@drwetter drwetter reopened this Sep 25, 2018

@drwetter

This comment has been minimized.

Show comment
Hide comment
@drwetter
Owner

drwetter commented Sep 25, 2018

see 305eefc and #1134, #1135

drwetter added a commit that referenced this issue Sep 26, 2018

Put temporary workaounrd in place for Linux + TCP fragmentation
This commit basically reverses the previous commit 305eefc for
Linux only. Here the external printf is working fine, where as
the BSDish counterpart is not (see e.g. #1134).

For Linux is basically addresses #1130 / #1113.

A compatible solution for all OS needs to be found.
@drwetter

This comment has been minimized.

Show comment
Hide comment
@drwetter

drwetter Sep 26, 2018

Owner

BSDish systems are still open -- at least if they do a write flush after "\x0a" which I don't know but assume.

@dcooper16 : You mind checking? There's a small chance that OS X libc behaves different

Owner

drwetter commented Sep 26, 2018

BSDish systems are still open -- at least if they do a write flush after "\x0a" which I don't know but assume.

@dcooper16 : You mind checking? There's a small chance that OS X libc behaves different

@drwetter

This comment has been minimized.

Show comment
Hide comment
@drwetter

drwetter Sep 26, 2018

Owner

note to self: /usr/bin/printf --version 2>&1 | grep -q GNU would have been better

Owner

drwetter commented Sep 26, 2018

note to self: /usr/bin/printf --version 2>&1 | grep -q GNU would have been better

@dcooper16

This comment has been minimized.

Show comment
Hide comment
@dcooper16

dcooper16 Sep 26, 2018

Contributor

As far as I can tell, it is the same with OS X. I am seeing the ClientHello message being broken into fragments.

Contributor

dcooper16 commented Sep 26, 2018

As far as I can tell, it is the same with OS X. I am seeing the ClientHello message being broken into fragments.

@drwetter

This comment has been minimized.

Show comment
Hide comment
@drwetter

drwetter Sep 26, 2018

Owner

:-( but thanks. Long / mid term solution is probably dd. Do you have stdbuf on OSX?

Owner

drwetter commented Sep 26, 2018

:-( but thanks. Long / mid term solution is probably dd. Do you have stdbuf on OSX?

@dcooper16

This comment has been minimized.

Show comment
Hide comment
@dcooper16

dcooper16 Sep 26, 2018

Contributor

Do you have stdbuf on OSX?

No. At least not on my computer.

Contributor

dcooper16 commented Sep 26, 2018

Do you have stdbuf on OSX?

No. At least not on my computer.

@drwetter

This comment has been minimized.

Show comment
Hide comment
@drwetter

drwetter Sep 26, 2018

Owner

sigh

Owner

drwetter commented Sep 26, 2018

sigh

@drwetter

This comment has been minimized.

Show comment
Hide comment
@drwetter

drwetter Oct 5, 2018

Owner

Any good ideas how to convert hex to binary?

Just a few thoughts I thought to share:
I tried a few things today but it seems to me ``printf``` in general is still the best option.

Atm the best solution I see for OS X and FreeBSD in order to avoid fragmentation is piping printf through dd or cat.

Interesting: The internal echo -e. The internal one also does fragmentation btw.

Owner

drwetter commented Oct 5, 2018

Any good ideas how to convert hex to binary?

Just a few thoughts I thought to share:
I tried a few things today but it seems to me ``printf``` in general is still the best option.

Atm the best solution I see for OS X and FreeBSD in order to avoid fragmentation is piping printf through dd or cat.

Interesting: The internal echo -e. The internal one also does fragmentation btw.

@drwetter

This comment has been minimized.

Show comment
Hide comment
@drwetter

drwetter Oct 11, 2018

Owner

see also commit 1b52834

Owner

drwetter commented Oct 11, 2018

see also commit 1b52834

drwetter added a commit that referenced this issue Oct 11, 2018

Fix fragmentation also under FreeBSD and OS X
This PR addresses the remaining TCP fragmentation by piping the line buffered
internal print through cat, see also #1130.
It extends 1b52834 which was the same doing for Linux and
OpenBSD.

This PR also consolidates the last remaining low level socket calls
in client_simulation_sockets() into socksend_clienthello().

An negative performance effect is barely measurable.

It also does a check whether the fd 5 is taken by a tty as
I see this while writing the commit message ;-). We might
want to make that line better instead of just echoing. :-)

@drwetter drwetter closed this Oct 11, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment