Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

quiche: TSAN failure in quic_protocol_integration_test #15414

Closed
danzh2010 opened this issue Mar 10, 2021 · 1 comment
Closed

quiche: TSAN failure in quic_protocol_integration_test #15414

danzh2010 opened this issue Mar 10, 2021 · 1 comment

Comments

@danzh2010
Copy link
Contributor

danzh2010 commented Mar 10, 2021

There is a use after free case in EnvoyQuicClientSessionWithExtra. The EnvoyQuicConnectionHelper in this class needs to out live its base class actually. conn_helper_, alarm_factory_ and supported_versions_ are configured per connection pool. So the best way to fix it is to move them out of EnvoyQuicClientSessionWithExtra into a standalone object living in connection pool.

At the same time, the other two members: server_id_ and crypto_config_ can be moved to QuicUpstreamTransportSocketFactory.

TSAN error:
[ RUN ] UpstreamProtocols/ProtocolIntegrationTest.TestDownstreamResetIdleTimeout/IPv6_HttpDownstream_Http3Upstream
=========== IntegrationCodecClient add connection callback 0x7b30000089d0==================
WARNING: ThreadSanitizer: heap-use-after-free (virtual call vs free) (pid=14)
Read of size 8 at 0x7b2800015430 by thread T9:
#0 quic::QuicBufferDeleter::operator()(char*) ??:? (quic_protocol_integration_test+0x3b9e236)
#1 quic::QuicMemSliceImpl::QuicMemSliceImpl(std::unique_ptr<char [], quic::QuicBufferDeleter>, unsigned long)::$_0::operator()(void const*, unsigned long, Envoy::Buffer::BufferFragmentImpl const*) const quic_mem_slice_impl.cc:? (quic_protocol_integration_test+0x60b1fb4)
#2 void std::__invoke_impl<void, quic::QuicMemSliceImpl::QuicMemSliceImpl(std::unique_ptr<char [], quic::QuicBufferDeleter>, unsigned long)::$_0&, void const*, unsigned long, Envoy::Buffer::BufferFragmentImpl const*>(std::__invoke_other, quic::QuicMemSliceImpl::QuicMemSliceImpl(std::unique_ptr<char [], quic::QuicBufferDeleter>, unsigned long)::$_0&, void const*&&, unsigned long&&, Envoy::Buffer::BufferFragmentImpl const*&&) quic_mem_slice_impl.cc:? (quic_protocol_integration_test+0x60b1f00)
#3 std::enable_if<is_invocable_r_v<void, quic::QuicMemSliceImpl::QuicMemSliceImpl(std::unique_ptr<char [], quic::QuicBufferDeleter>, unsigned long)::$_0&, void const*, unsigned long, Envoy::Buffer::BufferFragmentImpl const*>, void>::type std::__invoke_r<void, quic::QuicMemSliceImpl::QuicMemSliceImpl(std::unique_ptr<char [], quic::QuicBufferDeleter>, unsigned long)::$_0&, void const*, unsigned long, Envoy::Buffer::BufferFragmentImpl const*>(quic::QuicMemSliceImpl::QuicMemSliceImpl(std::unique_ptr<char [], quic::QuicBufferDeleter>, unsigned long)::$_0&, void const*&&, unsigned long&&, Envoy::Buffer::BufferFragmentImpl const*&&) quic_mem_slice_impl.cc:? (quic_protocol_integration_test+0x60b1da7)
#4 std::_Function_handler<void (void const*, unsigned long, Envoy::Buffer::BufferFragmentImpl const*), quic::QuicMemSliceImpl::QuicMemSliceImpl(std::unique_ptr<char [], quic::QuicBufferDeleter>, unsigned long)::$_0>::_M_invoke(std::_Any_data const&, void const*&&, unsigned long&&, Envoy::Buffer::BufferFragmentImpl const*&&) quic_mem_slice_impl.cc:? (quic_protocol_integration_test+0x60b1b6c)
#5 std::function<void (void const*, unsigned long, Envoy::Buffer::BufferFragmentImpl const*)>::operator()(void const*, unsigned long, Envoy::Buffer::BufferFragmentImpl const*) const ??:? (quic_protocol_integration_test+0x39bc6f5)
#6 Envoy::Buffer::BufferFragmentImpl::done() ??:? (quic_protocol_integration_test+0x39bc542)
#7 Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::{lambda()#1}::operator()() const ??:? (quic_protocol_integration_test+0x82c3602)
#8 void std::__invoke_impl<void, Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::{lambda()#1}&>(std::__invoke_other, Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::{lambda()#1}&) ??:? (quic_protocol_integration_test+0x82c3570)
#9 std::enable_if<is_invocable_r_v<void, Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::{lambda()#1}&>, std::enable_if>::type std::__invoke_r<void, Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::{lambda()#1}&>(void&&, (Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::{lambda()#1}&)...) ??:? (quic_protocol_integration_test+0x82c34a0)
#10 std::_Function_handler<void (), Envoy::Buffer::Slice::Slice(Envoy::Buffer::BufferFragment&)::{lambda()#1}>::_M_invoke(std::_Any_data const&) ??:? (quic_protocol_integration_test+0x82c32ca)
#11 std::function<void ()>::operator()() const ??:? (quic_protocol_integration_test+0x3940046)
#12 Envoy::Buffer::Slice::callAndClearDrainTrackers() ??:? (quic_protocol_integration_test+0x393f64f)
#13 Envoy::Buffer::Slice::~Slice() ??:? (quic_protocol_integration_test+0x393f1bf)
#14 Envoy::Buffer::SliceDeque::~SliceDeque() ??:? (quic_protocol_integration_test+0x393f05d)
#15 Envoy::Buffer::OwnedImpl::~OwnedImpl() ??:? (quic_protocol_integration_test+0x393eb65)
#16 quic::QuicMemSliceImpl::~QuicMemSliceImpl() ??:? (quic_protocol_integration_test+0x3afdac9)
#17 quic::QuicMemSlice::~QuicMemSlice() ??:? (quic_protocol_integration_test+0x3afc038)
#18 quic::BufferedSlice::~BufferedSlice() ??:? (quic_protocol_integration_test+0x3b967b8)
#19 quic::QuicCircularDeque<quic::BufferedSlice, 3ul, std::allocatorquic::BufferedSlice >::DestroyByAddress(quic::BufferedSlice*) const ??:? (quic_protocol_integration_test+0x3b9d6e4)
#20 quic::QuicCircularDeque<quic::BufferedSlice, 3ul, std::allocatorquic::BufferedSlice >::DestroyByIndex(unsigned long) const ??:? (quic_protocol_integration_test+0x3b9d69b)
#21 quic::QuicCircularDeque<quic::BufferedSlice, 3ul, std::allocatorquic::BufferedSlice >::DestroyUnwrappedRange(unsigned long, unsigned long) const ??:? (quic_protocol_integration_test+0x3b9d59f)
#22 quic::QuicCircularDeque<quic::BufferedSlice, 3ul, std::allocatorquic::BufferedSlice >::DestroyRange(unsigned long, unsigned long) const ??:? (quic_protocol_integration_test+0x3b9d27e)
#23 quic::QuicCircularDeque<quic::BufferedSlice, 3ul, std::allocatorquic::BufferedSlice >::DestroyAndDeallocateAll() ??:? (quic_protocol_integration_test+0x3b9cff5)
#24 quic::QuicCircularDeque<quic::BufferedSlice, 3ul, std::allocatorquic::BufferedSlice >::~QuicCircularDeque() ??:? (quic_protocol_integration_test+0x3b9cf1f)
#25 quic::QuicIntervalDeque<quic::BufferedSlice, quic::QuicCircularDeque<quic::BufferedSlice, 3ul, std::allocatorquic::BufferedSlice > >::~QuicIntervalDeque() ??:? (quic_protocol_integration_test+0x3b9b318)
#26 quic::QuicStreamSendBuffer::~QuicStreamSendBuffer() ??:? (quic_protocol_integration_test+0x3b96b6d)
#27 quic::QuicStream::~QuicStream() ??:? (quic_protocol_integration_test+0x3b43391)
#28 quic::QuicSpdyStream::~QuicSpdyStream() ??:? (quic_protocol_integration_test+0x3a6fbd5)
#29 quic::QuicSpdyClientStream::~QuicSpdyClientStream() ??:? (quic_protocol_integration_test+0x39dce79)
#30 Envoy::Quic::EnvoyQuicClientStream::~EnvoyQuicClientStream() ??:? (quic_protocol_integration_test+0x3973bc3)
#31 Envoy::Quic::EnvoyQuicClientStream::~EnvoyQuicClientStream() ??:? (quic_protocol_integration_test+0x396d076)
#32 Envoy::Quic::EnvoyQuicClientStream::~EnvoyQuicClientStream() ??:? (quic_protocol_integration_test+0x396d0cf)
#33 std::default_deletequic::QuicStream::operator()(quic::QuicStream*) const ??:? (quic_protocol_integration_test+0x39c8cb6)
#34 std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >::~unique_ptr() ??:? (quic_protocol_integration_test+0x39c7213)
#35 void std::_Destroy<std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream > >(std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >) ??:? (quic_protocol_integration_test+0x3b2ceb8)
#36 void std::_Destroy_aux::__destroy<std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >
>(std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >, std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >) ??:? (quic_protocol_integration_test+0x3b2ce6a)
#37 void std::_Destroy<std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >>(std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >, std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >) ??:? (quic_protocol_integration_test+0x3b2cdf8)
#38 void std::_Destroy<std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >
, std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream > >(std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >, std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >, std::allocator<std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream > >&) ??:? (quic_protocol_integration_test+0x3b2ccd4)
#39 std::vector<std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream >, std::allocator<std::unique_ptr<quic::QuicStream, std::default_deletequic::QuicStream > > >::~vector() ??:? (quic_protocol_integration_test+0x3b23aaa)
#40 quic::QuicSession::~QuicSession() ??:? (quic_protocol_integration_test+0x3b05625)
#41 quic::QuicSpdySession::~QuicSpdySession() ??:? (quic_protocol_integration_test+0x3a4b654)
#42 quic::QuicSpdyClientSessionBase::~QuicSpdyClientSessionBase() ??:? (quic_protocol_integration_test+0x39cb6bc)
#43 quic::QuicSpdyClientSession::~QuicSpdyClientSession() ??:? (quic_protocol_integration_test+0x39c4b05)
#44 Envoy::Quic::EnvoyQuicClientSession::~EnvoyQuicClientSession() ??:? (quic_protocol_integration_test+0x393ad65)
#45 Envoy::Quic::EnvoyQuicClientSessionWithExtras::~EnvoyQuicClientSessionWithExtras() ??:?

Previous write of size 8 at 0x7b2800015430 by thread T9:
#0 free ??:? (quic_protocol_integration_test+0x36e94f8)
#1 std::default_deleteEnvoy::Quic::QuicUpstreamData::operator()(Envoy::Quic::QuicUpstreamData*) const ??:? (quic_protocol_integration_test+0x380a7c2)
#2 std::unique_ptr<Envoy::Quic::QuicUpstreamData, std::default_deleteEnvoy::Quic::QuicUpstreamData >::~unique_ptr() ??:? (quic_protocol_integration_test+0x3808d63)
#3 Envoy::Quic::EnvoyQuicClientSessionWithExtras::~EnvoyQuicClientSessionWithExtras() ??:? (quic_protocol_integration_test+0x381659e)
#4 Envoy::Quic::EnvoyQuicClientSessionWithExtras::~EnvoyQuicClientSessionWithExtras() ??:? (quic_protocol_integration_test+0x3813a16)
#5 Envoy::Quic::EnvoyQuicClientSessionWithExtras::~EnvoyQuicClientSessionWithExtras() ??:? (quic_protocol_integration_test+0x3813a6f)
#6 non-virtual thunk to Envoy::Quic::EnvoyQuicClientSessionWithExtras::~EnvoyQuicClientSessionWithExtras() ??:? (quic_protocol_integration_test+0x38143a2)
#7 std::default_deleteEnvoy::Network::ClientConnection::operator()(Envoy::Network::ClientConnection*) const ??:? (quic_protocol_integration_test+0x3df2136)
#8 std::unique_ptr<Envoy::Network::ClientConnection, std::default_deleteEnvoy::Network::ClientConnection >::~unique_ptr() ??:? (quic_protocol_integration_test+0x3ddf4d3)
#9 Envoy::Http::CodecClient::~CodecClient() ??:? (quic_protocol_integration_test+0x633d00d)
#10 Envoy::Http::CodecClientProd::~CodecClientProd() ??:? (quic_protocol_integration_test+0x3dde80b)
#11 Envoy::Http::CodecClientProd::~CodecClientProd() ??:? (quic_protocol_integration_test+0x634446f)
#12 std::default_deleteEnvoy::Http::CodecClient::operator()(Envoy::Http::CodecClient*) const ??:? (quic_protocol_integration_test+0x49b2966)
#13 std::unique_ptr<Envoy::Http::CodecClient, std::default_deleteEnvoy::Http::CodecClient >::~unique_ptr() ??:? (quic_protocol_integration_test+0x49af6c3)
#14 Envoy::Http::ActiveClient::~ActiveClient() ??:? (quic_protocol_integration_test+0x49bdf86)
#15 Envoy::Http::MultiplexedActiveClientBase::~MultiplexedActiveClientBase() ??:? (quic_protocol_integration_test+0x49c4a59)
#16 Envoy::Http::Http3::ActiveClient::~ActiveClient() ??:? (quic_protocol_integration_test+0x49c482b)
#17 Envoy::Http::Http3::ActiveClient::~ActiveClient() ??:? (quic_protocol_integration_test+0x49c486f)
#18 non-virtual thunk to Envoy::Http::Http3::ActiveClient::~ActiveClient() ??:? (quic_protocol_integration_test+0x49c4a12)
#19 std::default_deleteEnvoy::Event::DeferredDeletable::operator()(Envoy::Event::DeferredDeletable*) const ??:? (quic_protocol_integration_test+0x419bd66)
#20 std::__uniq_ptr_impl<Envoy::Event::DeferredDeletable, std::default_deleteEnvoy::Event::DeferredDeletable >::reset(Envoy::Event::DeferredDeletable*) ??:? (quic_protocol_integration_test+0x68e61f3)

@danzh2010 danzh2010 added bug triage Issue requires triage labels Mar 10, 2021
@mattklein123 mattklein123 added area/test flakes and removed bug triage Issue requires triage labels Mar 11, 2021
@alyssawilk
Copy link
Contributor

Oops, I think this was fixed by #15444

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants