Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incorrectly used ipv6 in proxy connection #1226

Closed
mmmds opened this issue Jul 25, 2019 · 5 comments
Closed

incorrectly used ipv6 in proxy connection #1226

mmmds opened this issue Jul 25, 2019 · 5 comments
Assignees

Comments

@mmmds
Copy link

mmmds commented Jul 25, 2019

A socket structure is incorrectly used as ipv6 while connecting to a proxy server.

PoC

echo -e "GET /test8/test.html HTTP/1.0\nUser-Agent: python\n\n" | nc 127.0.0.1 80

ASAN

==27029==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600001cd98 at pc 0x55ea3748a492 bp 0x7f6bc42eca90 sp 0x7f6bc42eca80
READ of size 16 at 0x60600001cd98 thread T1
    #0 0x55ea3748a491 in cherokee_socket_update_from_addrinfo /home/mmm/fuzz/webserver/cherokee/socket.c:396
    #1 0x55ea3752b4c0 in cherokee_handler_proxy_conn_init_socket /home/mmm/fuzz/webserver/cherokee/proxy_hosts.c:465
    #2 0x55ea37521029 in cherokee_handler_proxy_init /home/mmm/fuzz/webserver/cherokee/handler_proxy.c:975
    #3 0x55ea3758544c in cherokee_handler_init /home/mmm/fuzz/webserver/cherokee/handler.c:93
    #4 0x55ea37582233 in cherokee_connection_open_request /home/mmm/fuzz/webserver/cherokee/connection.c:2678
    #5 0x55ea374be889 in process_active_connections /home/mmm/fuzz/webserver/cherokee/thread.c:1165
    #6 0x55ea374c4549 in cherokee_thread_step_MULTI_THREAD /home/mmm/fuzz/webserver/cherokee/thread.c:2086
    #7 0x55ea374b8300 in thread_routine /home/mmm/fuzz/webserver/cherokee/thread.c:99
    #8 0x7f6bc7b316da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
    #9 0x7f6bc765688e in __clone (/lib/x86_64-linux-gnu/libc.so.6+0x12188e)

0x60600001cda0 is located 0 bytes to the right of 64-byte region [0x60600001cd60,0x60600001cda0)
allocated by thread T0 here:
    #0 0x7f6bc805fb50 in __interceptor_malloc (/usr/lib/x86_64-linux-gnu/libasan.so.4+0xdeb50)
    #1 0x7f6bc763a4c4  (/lib/x86_64-linux-gnu/libc.so.6+0x1054c4)
    #2 0x7f6bc763cce3 in getaddrinfo (/lib/x86_64-linux-gnu/libc.so.6+0x107ce3)
    #3 0x7f6bc801825a  (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x9725a)
    #4 0x55ea3744c08a in cherokee_gethostbyname /home/mmm/fuzz/webserver/cherokee/util.c:793
    #5 0x55ea37477a31 in entry_fill_up /home/mmm/fuzz/webserver/cherokee/resolv_cache.c:96
    #6 0x55ea374784b4 in table_add_new_entry /home/mmm/fuzz/webserver/cherokee/resolv_cache.c:227
    #7 0x55ea3747888b in cherokee_resolv_cache_get_ipstr /home/mmm/fuzz/webserver/cherokee/resolv_cache.c:263
    #8 0x55ea374d0a09 in set_host /home/mmm/fuzz/webserver/cherokee/source.c:289
    #9 0x55ea374d0b3c in cherokee_source_configure /home/mmm/fuzz/webserver/cherokee/source.c:304
    #10 0x55ea374a4c9f in add_source /home/mmm/fuzz/webserver/cherokee/server.c:1243
    #11 0x55ea374981e7 in cherokee_config_node_while /home/mmm/fuzz/webserver/cherokee/config_node.c:226
    #12 0x55ea374a8f37 in configure_server /home/mmm/fuzz/webserver/cherokee/server.c:1657
    #13 0x55ea374a98ad in cherokee_server_read_config_file /home/mmm/fuzz/webserver/cherokee/server.c:1751
    #14 0x55ea37447644 in common_server_initialization /home/mmm/fuzz/webserver/cherokee/main_worker.c:245
    #15 0x55ea374481f7 in main /home/mmm/fuzz/webserver/cherokee/main_worker.c:393
    #16 0x7f6bc7556b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

Thread T1 created by T0 here:   
    #0 0x7f6bc7fb8d2f in __interceptor_pthread_create (/usr/lib/x86_64-linux-gnu/libasan.so.4+0x37d2f)
    #1 0x55ea374b9219 in cherokee_thread_new /home/mmm/fuzz/webserver/cherokee/thread.c:247
    #2 0x55ea374a173f in initialize_server_threads /home/mmm/fuzz/webserver/cherokee/server.c:671
    #3 0x55ea374a3a05 in cherokee_server_initialize /home/mmm/fuzz/webserver/cherokee/server.c:1053
    #4 0x55ea3744776f in common_server_initialization /home/mmm/fuzz/webserver/cherokee/main_worker.c:255
    #5 0x55ea374481f7 in main /home/mmm/fuzz/webserver/cherokee/main_worker.c:393
    #6 0x7f6bc7556b96 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x21b96)

SUMMARY: AddressSanitizer: heap-buffer-overflow /home/mmm/fuzz/webserver/cherokee/socket.c:396 in cherokee_socket_update_from_addrinfo
Shadow bytes around the buggy address:
  0x0c0c7fffb960: fa fa fa fa 00 00 00 00 00 00 03 fa fa fa fa fa
  0x0c0c7fffb970: 00 00 00 00 00 00 06 fa fa fa fa fa 00 00 00 00
  0x0c0c7fffb980: 00 00 03 fa fa fa fa fa 00 00 00 00 00 00 06 fa
  0x0c0c7fffb990: fa fa fa fa 00 00 00 00 00 00 04 fa fa fa fa fa
  0x0c0c7fffb9a0: 00 00 00 00 00 00 00 fa fa fa fa fa 00 00 00 00
=>0x0c0c7fffb9b0: 00 00 00[00]fa fa fa fa 00 00 00 00 00 00 00 fa
  0x0c0c7fffb9c0: fa fa fa fa 00 00 00 00 00 00 00 fa fa fa fa fa
  0x0c0c7fffb9d0: 00 00 00 00 00 00 00 fa fa fa fa fa 00 00 00 00
  0x0c0c7fffb9e0: 00 00 00 00 fa fa fa fa 00 00 00 00 00 00 00 fa
  0x0c0c7fffb9f0: fa fa fa fa 00 00 00 00 00 00 00 01 fa fa fa fa
  0x0c0c7fffba00: 00 00 00 00 00 00 00 fa fa fa fa fa fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==27029==ABORTING

Setup:

  • Ubuntu 18.04 64 bit
  • source code from github, commit 9a75e65
  • build command:
ac_cv_func_realloc_0_nonnull=yes ac_cv_func_malloc_0_nonnull=yes LDFLAGS="-lasan" LDADD="-lasan" CFLAGS="-fsanitize=address -ggdb -O0 -fprofile-arcs -ftest-coverage" ./configure --prefix=`pwd`/bin --enable-trace --enable-static-module=all --enable-static --enable-shared=no
make
  • files in webroot mkdir /var/www/test{1..20}; for i in seq 1 20; do echo test > test$i/test.html; done
  • configuration file cherokee.txt

found by: Mateusz Kocielski, Michał Dardas from LogicalTrust

@skinkie skinkie self-assigned this Jul 25, 2019
@skinkie
Copy link
Member

skinkie commented Jul 25, 2019

@mmmds could you elaborate where the relationship IPv6 structure comes into place?

@mmmds
Copy link
Author

mmmds commented Jul 25, 2019

I've set breakpoint on socket.c:391 and examined flow on a second hit.

   391  switch (addr->ai_family) {
   392  case AF_INET:
   393          memcpy (&SOCKET_SIN_ADDR(socket), &((struct sockaddr_in *) ai->ai_addr)->sin_addr, sizeof(struct in_addr));
   394          break;
   395  case AF_INET6:
   396          memcpy (&SOCKET_SIN6_ADDR(socket), &((struct sockaddr_in6 *) ai->ai_addr) >sin6_addr, sizeof(struct in6_addr));
   397          break;

The application falls into case AF_INET6 and crashes on line 396. However, if I change

set var addr->ai_family=2

the application falls into AF_INET case and it proceeds without a crash. By that I assume that ai holds struct sockaddr_in with 4 bytes allocated, but memcpy tries to read 16 bytes.

> print &((struct sockaddr_in *) ai->ai_addr)->sin_addr
$3 = (struct in_addr *) 0x60600001cdf4
==23299==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600001cdf8 at pc 0x55555568e492 bp 0x7fffffffdaf0 sp 0x7fffffffdae0
READ of size 16 at 0x60600001cdf8 thread T0

@skinkie skinkie added the t:bug label Jul 26, 2019
@skinkie
Copy link
Member

skinkie commented Nov 9, 2019

Extra information, in order to make this error work. /etc/hosts should contain:

127.0.0.1       localhost 
::1             localhost

If either of one is defined, it will not trigger. Hence the approach where ipv6-localhost is defined (for example Ubuntu) will not have issues. It took some time to figure this out, better write about it.

@skinkie
Copy link
Member

skinkie commented Nov 9, 2019

This might be a related issue.

CHEROKEE_TRACE=proxy /tmp/cherokee/sbin/cherokee-worker 
Cherokee Web Server 1.2.104 (Nov  9 2019): Listening on port ALL:8000, TLS
disabled, IPv6 enabled, using epoll, 4096 fds system limit, max. 2041
connections, 20 threads, 102 connections per thread, standard scheduling
policy, tracing 'proxy'
   handler_proxy.c:0901 (   cherokee_handler_proxy_init): Entering init 'get conn'
     proxy_hosts.c:0453 (cherokee_handler_proxy_conn_init_socket): Initializing proxy socket
     proxy_hosts.c:0493 (cherokee_handler_proxy_conn_init_socket): Proxy socket Initialized: IPv6, target: localhost, IP: ::1
   handler_proxy.c:0939 (   cherokee_handler_proxy_init): Entering phase 'preconnect'
   handler_proxy.c:0984 (   cherokee_handler_proxy_init): Entering phase 'connect': pconn=0x61100001a1c0
   handler_proxy.c:0939 (   cherokee_handler_proxy_init): Entering phase 'preconnect'
     proxy_hosts.c:0453 (cherokee_handler_proxy_conn_init_socket): Initializing proxy socket
=================================================================
==20662==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6060000211e0 at pc 0x561b7abe6bed bp 0x7f041cace840 sp 0x7f041cace830
READ of size 4 at 0x6060000211e0 thread T17
    #0 0x561b7abe6bec in cherokee_ntop /home/skinkie/Sources/webserver/cherokee/util.c:2223
    #1 0x561b7acbf72a in cherokee_handler_proxy_conn_init_socket /home/skinkie/Sources/webserver/cherokee/proxy_hosts.c:491
    #2 0x561b7acb4f60 in cherokee_handler_proxy_init /home/skinkie/Sources/webserver/cherokee/handler_proxy.c:975
    #3 0x561b7ad2abbd in cherokee_handler_init /home/skinkie/Sources/webserver/cherokee/handler.c:93
    #4 0x561b7ad279bd in cherokee_connection_open_request /home/skinkie/Sources/webserver/cherokee/connection.c:2678
    #5 0x561b7ac52b6b in process_active_connections /home/skinkie/Sources/webserver/cherokee/thread.c:1165
    #6 0x561b7ac58800 in cherokee_thread_step_MULTI_THREAD /home/skinkie/Sources/webserver/cherokee/thread.c:2086
    #7 0x561b7ac4c589 in thread_routine /home/skinkie/Sources/webserver/cherokee/thread.c:99
    #8 0x7f0452344016 in start_thread (/lib64/libpthread.so.0+0x8016)
    #9 0x7f0451bffafe in clone (/lib64/libc.so.6+0x106afe)

@skinkie
Copy link
Member

skinkie commented Nov 9, 2019

@mmmds one of the other things that your message has put me on the wrong foot is that your actual request does not confirm the HTTP standard. It should have been CRLF, not just LF.

echo -e "GET /test8/test.html HTTP/1.0\r\nUser-Agent: python\r\n\r\n" | nc 10.9.0.2 8000

skinkie added a commit that referenced this issue Nov 9, 2019
This commit changes something fundamental in the proxy handling. Previously the proxy was iterating over multiple possible IP addresses received from the source definition. This
issue #1226 shows that this fails when mixing IPv4 and IPv6 results. This could be resolved by just taking the "right" addrinfo result, but this is executed after the socket has been
created, hence: if we would have an IPv6 result, but created an IPv4 socket, we would miss all checks that happen upon socket creation.

The iteration over the getaddrinfo results has now been pulled into the proxy specific code, this might still be able to be abstracted.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants