ASAN tripping on traffic_server -F #6766
Description
Running with traffic_server -f -F on docs, I get
Traffic Server 9.0.0 May 12 2020 16:29:24 qa1.rax.boot.org
traffic_server: using root directory '/opt/ats'
=================================================================
==26501==ERROR: AddressSanitizer: heap-use-after-free on address 0x619000031e88 at pc 0x00000073fb88 bp 0x7ff349f3d220 sp 0x7ff349f3d210
READ of size 8 at 0x619000031e88 thread T7 ([ET_NET 5])
#0 0x73fb87 in Http1ClientSession::state_keep_alive(int, void*) /usr/local/src/trafficserver/proxy/http/Http1ClientSession.cc:399
#1 0x65aea3 in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:190
#2 0xd2d88d in read_signal_and_update /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:83
#3 0xd2e1b9 in read_signal_done /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:144
#4 0xd35287 in UnixNetVConnection::readSignalDone(int, NetHandler*) /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:1000
#5 0xcbbd19 in SSLNetVConnection::net_read_io(NetHandler*, EThread*) /usr/local/src/trafficserver/iocore/net/SSLNetVConnection.cc:713
#6 0xd158fd in NetHandler::process_ready_list() /usr/local/src/trafficserver/iocore/net/UnixNet.cc:400
#7 0xd17214 in NetHandler::waitForActivity(long) /usr/local/src/trafficserver/iocore/net/UnixNet.cc:535
#8 0xdb25cd in EThread::execute_regular() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:284
#9 0xdb2e02 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:345
#10 0xdaf485 in spawn_thread_internal /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:92
#11 0x7ff351ac9ea4 in start_thread (/lib64/libpthread.so.0+0x7ea4)
#12 0x7ff350dcf8dc in __clone (/lib64/libc.so.6+0xfe8dc)
0x619000031e88 is located 776 bytes inside of 1088-byte region [0x619000031b80,0x619000031fc0)
freed by thread T7 ([ET_NET 5]) here:
#0 0x7ff353827508 in __interceptor_free (/lib64/libasan.so.4+0xde508)
#1 0x7ff35336dba1 in ats_memalign_free /usr/local/src/trafficserver/src/tscore/ink_memory.cc:138
#2 0x7ff353392a51 in jearena::JemallocNodumpAllocator::deallocate(_InkFreeList*, void*) /usr/local/src/trafficserver/src/tscore/JeAllocator.cc:139
#3 0x7ff3533706cc in malloc_free /usr/local/src/trafficserver/src/tscore/ink_queue.cc:323
#4 0x7ff3533700ee in ink_freelist_free /usr/local/src/trafficserver/src/tscore/ink_queue.cc:277
#5 0x74363c in ClassAllocator<Http1ClientSession>::free(Http1ClientSession*) (/opt/ats/bin/traffic_server+0x74363c)
#6 0x743448 in void thread_free<Http1ClientSession>(ClassAllocator<Http1ClientSession>&, Http1ClientSession*) (/opt/ats/bin/traffic_server+0x743448)
#7 0x73cabf in Http1ClientSession::free() /usr/local/src/trafficserver/proxy/http/Http1ClientSession.cc:126
#8 0xc72e0c in ProxySession::handle_api_return(int) /usr/local/src/trafficserver/proxy/ProxySession.cc:168
#9 0xc72b1d in ProxySession::do_api_callout(TSHttpHookID) /usr/local/src/trafficserver/proxy/ProxySession.cc:145
#10 0x73c42a in Http1ClientSession::destroy() /usr/local/src/trafficserver/proxy/http/Http1ClientSession.cc:79
#11 0x73eab2 in Http1ClientSession::do_io_close(int) /usr/local/src/trafficserver/proxy/http/Http1ClientSession.cc:295
#12 0x73fb5d in Http1ClientSession::state_keep_alive(int, void*) /usr/local/src/trafficserver/proxy/http/Http1ClientSession.cc:398
#13 0x65aea3 in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:190
#14 0xd2d88d in read_signal_and_update /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:83
#15 0xd2e1b9 in read_signal_done /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:144
#16 0xd35287 in UnixNetVConnection::readSignalDone(int, NetHandler*) /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:1000
#17 0xcbbd19 in SSLNetVConnection::net_read_io(NetHandler*, EThread*) /usr/local/src/trafficserver/iocore/net/SSLNetVConnection.cc:713
#18 0xd158fd in NetHandler::process_ready_list() /usr/local/src/trafficserver/iocore/net/UnixNet.cc:400
#19 0xd17214 in NetHandler::waitForActivity(long) /usr/local/src/trafficserver/iocore/net/UnixNet.cc:535
#20 0xdb25cd in EThread::execute_regular() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:284
#21 0xdb2e02 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:345
#22 0xdaf485 in spawn_thread_internal /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:92
#23 0x7ff351ac9ea4 in start_thread (/lib64/libpthread.so.0+0x7ea4)
previously allocated by thread T7 ([ET_NET 5]) here:
#0 0x7ff3538284f0 in posix_memalign (/lib64/libasan.so.4+0xdf4f0)
#1 0x7ff35336d954 in ats_memalign /usr/local/src/trafficserver/src/tscore/ink_memory.cc:102
#2 0x7ff3533929ce in jearena::JemallocNodumpAllocator::allocate(_InkFreeList*) /usr/local/src/trafficserver/src/tscore/JeAllocator.cc:118
#3 0x7ff35336ffb2 in malloc_new /usr/local/src/trafficserver/src/tscore/ink_queue.cc:264
#4 0x7ff35336f1fd in ink_freelist_new /usr/local/src/trafficserver/src/tscore/ink_queue.cc:187
#5 0x728bf9 in ClassAllocator<Http1ClientSession>::alloc() ../../include/tscore/Allocator.h:131
#6 0x728bba in Http1ClientSession* thread_alloc_init<Http1ClientSession>(ClassAllocator<Http1ClientSession>&, ProxyAllocator&) /usr/local/src/trafficserver/iocore/eventsystem/I_ProxyAllocator.h:73
#7 0x7277b2 in HttpSessionAccept::accept(NetVConnection*, MIOBuffer*, IOBufferReader*) /usr/local/src/trafficserver/proxy/http/HttpSessionAccept.cc:52
#8 0xc713cc in ProtocolProbeTrampoline::ioCompletionEvent(int, void*) /usr/local/src/trafficserver/proxy/ProtocolProbeSessionAccept.cc:147
#9 0x65aea3 in Continuation::handleEvent(int, void*) /usr/local/src/trafficserver/iocore/eventsystem/I_Continuation.h:190
#10 0xd2d88d in read_signal_and_update /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:83
#11 0xd352a9 in UnixNetVConnection::readSignalAndUpdate(int) /usr/local/src/trafficserver/iocore/net/UnixNetVConnection.cc:1006
#12 0xcbb8a9 in SSLNetVConnection::net_read_io(NetHandler*, EThread*) /usr/local/src/trafficserver/iocore/net/SSLNetVConnection.cc:671
#13 0xd158fd in NetHandler::process_ready_list() /usr/local/src/trafficserver/iocore/net/UnixNet.cc:400
#14 0xd17214 in NetHandler::waitForActivity(long) /usr/local/src/trafficserver/iocore/net/UnixNet.cc:535
#15 0xdb25cd in EThread::execute_regular() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:284
#16 0xdb2e02 in EThread::execute() /usr/local/src/trafficserver/iocore/eventsystem/UnixEThread.cc:345
#17 0xdaf485 in spawn_thread_internal /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:92
#18 0x7ff351ac9ea4 in start_thread (/lib64/libpthread.so.0+0x7ea4)
Thread T7 ([ET_NET 5]) created by T0 ([TS_MAIN]) here:
#0 0x7ff353780a7f in pthread_create (/lib64/libasan.so.4+0x37a7f)
#1 0xdaecf4 in ink_thread_create ../../include/tscore/ink_thread.h:159
#2 0xdaf5b3 in Thread::start(char const*, void*, unsigned long, std::function<void ()> const&) /usr/local/src/trafficserver/iocore/eventsystem/Thread.cc:109
#3 0xdb907a in EventProcessor::spawn_event_threads(int, int, unsigned long) /usr/local/src/trafficserver/iocore/eventsystem/UnixEventProcessor.cc:392
#4 0xdb9920 in EventProcessor::start(int, unsigned long) /usr/local/src/trafficserver/iocore/eventsystem/UnixEventProcessor.cc:455
#5 0x6e1d25 in main traffic_server/traffic_server.cc:1970
#6 0x7ff350cf3554 in __libc_start_main (/lib64/libc.so.6+0x22554)
SUMMARY: AddressSanitizer: heap-use-after-free /usr/local/src/trafficserver/proxy/http/Http1ClientSession.cc:399 in Http1ClientSession::state_keep_alive(int, void*)
Shadow bytes around the buggy address:
0x0c327fffe380: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c327fffe390: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c327fffe3a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c327fffe3b0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c327fffe3c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c327fffe3d0: fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c327fffe3e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c327fffe3f0: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
0x0c327fffe400: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c327fffe410: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c327fffe420: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==26501==ABORTING