Skip to content

Commit

Permalink
test: fix a race in http2 flood integration test
Browse files Browse the repository at this point in the history
This test sends a large number of metadata frames in order to trigger a disconnect. However, it was possible for the disconnect to happen and the connection to be torn down before all the metadata frames had been sent. If that happened, ASAN detected a UAF:
```
==95==ERROR: AddressSanitizer: heap-use-after-free on address 0x60700037e5a0 at pc 0x000004811f9e bp 0x7ffc903af990 sp 0x7ffc903af988
READ of size 8 at 0x60700037e5a0 thread T0
    #0 0x4811f9d in Envoy::IntegrationCodecClient::sendMetadata(Envoy::Http::RequestEncoder&, Envoy::Http::MetadataMap) /proc/self/cwd/test/integration/http_integration.cc:168:3
    envoyproxy#1 0x46ed711 in Envoy::Http2FloodMitigationTest_RequestMetadata_Test::TestBody() /proc/self/cwd/test/integration/http2_flood_integration_test.cc:1486:20
    envoyproxy#2 0xd380e64 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2580:10
    envoyproxy#3 0xd348dc2 in testing::Test::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2655:5
    envoyproxy#4 0xd34a927 in testing::TestInfo::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2832:11
    envoyproxy#5 0xd34ccc4 in testing::TestSuite::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2986:28
    envoyproxy#6 0xd36f07a in testing::internal::UnitTestImpl::RunAllTests() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5697:44
    envoyproxy#7 0xd384e63 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/
gtest.cc:2580:10
    envoyproxy#8 0xd36dd86 in testing::UnitTest::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5280:10
    envoyproxy#9 0xa0e53a4 in Envoy::TestRunner::RunTests(int, char**) /proc/self/cwd/external/com_google_googletest/googletest/include/gtest/gtest.h:2485:46
    envoyproxy#10 0xa0e0af7 in main /proc/self/cwd/test/main.cc:34:10
    envoyproxy#11 0x7f442ef69082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
    envoyproxy#12 0x45ed36d in _start (/mnt/ssd/cas/work/1/exec/bazel-out/k8-dbg/bin/test/integration/http2_flood_integration_test.runfiles/envoy/test/integration/http2_flood_integration_test+0x45ed36d)

0x60700037e5a0 is located 48 bytes inside of 80-byte region [0x60700037e570,0x60700037e5c0)
freed by thread T0 here:
    #0 0x466f7d2 in free /local/mnt/workspace/bcain_clang_hu-bcain-lv_22036/final/llvm-project/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3
    envoyproxy#1 0x831dde8 in Envoy::Http::CodecClient::ActiveRequest::~ActiveRequest() /proc/self/cwd/./source/common/http/codec_client.h:220:10
    envoyproxy#2 0x5aa33f9 in std::__1::unique_ptr<Envoy::Event::DeferredDeletable, std::__1::default_delete<Envoy::Event::DeferredDeletable> >::reset(Envoy::Event::DeferredDeletable*) /opt/llvm/bin/../include/c++/v1/__memory/unique_ptr.h:54:5
    envoyproxy#3 0xa3218e8 in Envoy::Event::DispatcherImpl::clearDeferredDeleteList() /proc/self/cwd/source/common/event/dispatcher_impl.cc:142:21
    envoyproxy#4 0xa3348df in void std::__1::__invoke_void_return_wrapper<void, true>::__call<Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Envoy::Thread::ThreadFactory&, Envoy::TimeSource&
, Envoy::Random::RandomGenerator&, Envoy::Filesystem::Instance&, Envoy::Event::TimeSystem&, std::__1::function<std::__1::unique_ptr<Envoy::Event::ScaledRangeTimerManager, std::__1::default_delete<Envoy::Event::ScaledRangeTimerManager> > (Envoy::Event::Dispatcher&)> const&
, std::__1::shared_ptr<Envoy::Buffer::WatermarkFactory> const&)::$_2&>(Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Envoy::Thread::ThreadFactory&, Envoy::TimeSource&, Envoy::Rand
om::RandomGenerator&, Envoy::Filesystem::Instance&, Envoy::Event::TimeSystem&, std::__1::function<std::__1::unique_ptr<Envoy::Event::ScaledRangeTimerManager, std::__1::default_delete<Envoy::Event::ScaledRangeTimerManager> > (Envoy::Event::Dispatcher&)> const&, std::__1::s
hared_ptr<Envoy::Buffer::WatermarkFactory> const&)::$_2&) /proc/self/cwd/source/common/event/dispatcher_impl.cc:79:30
    envoyproxy#5 0xa334603 in std::__1::__function::__func<Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Envoy::Thread::ThreadFactory&, Envoy::TimeSource&, Envoy::Random::RandomGenerator&,
Envoy::Filesystem::Instance&, Envoy::Event::TimeSystem&, std::__1::function<std::__1::unique_ptr<Envoy::Event::ScaledRangeTimerManager, std::__1::default_delete<Envoy::Event::ScaledRangeTimerManager> > (Envoy::Event::Dispatcher&)> const&, std::__1::shared_ptr<Envoy::Buffe
r::WatermarkFactory> const&)::$_2, std::__1::allocator<Envoy::Event::DispatcherImpl::DispatcherImpl(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, Envoy::Thread::ThreadFactory&, Envoy::TimeSource&, Envoy::Random::RandomGenera
tor&, Envoy::Filesystem::Instance&, Envoy::Event::TimeSystem&, std::__1::function<std::__1::unique_ptr<Envoy::Event::ScaledRangeTimerManager, std::__1::default_delete<Envoy::Event::ScaledRangeTimerManager> > (Envoy::Event::Dispatcher&)> const&, std::__1::shared_ptr<Envoy:
:Buffer::WatermarkFactory> const&)::$_2>, void ()>::operator()() /opt/llvm/bin/../include/c++/v1/__functional/function.h:180:16
    envoyproxy#6 0x4897039 in std::__1::__function::__value_func<void ()>::operator()() const /opt/llvm/bin/../include/c++/v1/__functional/function.h:507:16
    envoyproxy#7 0xa8e6aa4 in Envoy::Event::SchedulableCallbackImpl::SchedulableCallbackImpl(Envoy::CSmartPtr<event_base, &(event_base_free)>&, std::__1::function<void ()>)::$_0::__invoke(int, short, void*) /opt/llvm/bin/../include/c++/v1/__functional/function.h:1184:12
    envoyproxy#8 0xb557c5e in event_process_active_single_queue /mnt/ssd/cas/work/2/exec/external/com_github_libevent_libevent/event.c:1713:4
    envoyproxy#9 0xb539252 in event_process_active /mnt/ssd/cas/work/2/exec/external/com_github_libevent_libevent/event.c
    envoyproxy#10 0xb539252 in event_base_loop /mnt/ssd/cas/work/2/exec/external/com_github_libevent_libevent/event.c:2047:12
    envoyproxy#11 0xa8e1e3c in Envoy::Event::LibeventScheduler::run(Envoy::Event::Dispatcher::RunType) /proc/self/cwd/source/common/event/libevent_scheduler.cc:60:3
    envoyproxy#12 0xa32bd94 in Envoy::Event::DispatcherImpl::run(Envoy::Event::Dispatcher::RunType) /proc/self/cwd/source/common/event/dispatcher_impl.cc:299:19
    envoyproxy#13 0x480faad in Envoy::IntegrationCodecClient::flushWrite() /proc/self/cwd/test/integration/http_integration.cc:100:29
    envoyproxy#14 0x4811e94 in Envoy::IntegrationCodecClient::sendMetadata(Envoy::Http::RequestEncoder&, Envoy::Http::MetadataMap) /proc/self/cwd/test/integration/http_integration.cc:169:3
    envoyproxy#15 0x46ed711 in Envoy::Http2FloodMitigationTest_RequestMetadata_Test::TestBody() /proc/self/cwd/test/integration/http2_flood_integration_test.cc:1486:20
    envoyproxy#16 0xd380e64 in void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2580:10
    envoyproxy#17 0xd348dc2 in testing::Test::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2655:5
    envoyproxy#18 0xd34a927 in testing::TestInfo::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2832:11
    envoyproxy#19 0xd34ccc4 in testing::TestSuite::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2986:28
    envoyproxy#20 0xd36f07a in testing::internal::UnitTestImpl::RunAllTests() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5697:44
    envoyproxy#21 0xd384e63 in bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*) /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:2580:10
    envoyproxy#22 0xd36dd86 in testing::UnitTest::Run() /proc/self/cwd/external/com_google_googletest/googletest/src/gtest.cc:5280:10
    envoyproxy#23 0xa0e53a4 in Envoy::TestRunner::RunTests(int, char**) /proc/self/cwd/external/com_google_googletest/googletest/include/gtest/gtest.h:2485:46
    envoyproxy#24 0xa0e0af7 in main /proc/self/cwd/test/main.cc:34:10
    envoyproxy#25 0x7f442ef69082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
```

To fix that, write all metadata frames at once.

Signed-off-by: Benjamin Peterson <benjamin@engflow.com>
  • Loading branch information
benjaminp committed Sep 13, 2022
1 parent 7ba4c86 commit a500db5
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion test/integration/http2_flood_integration_test.cc
Expand Up @@ -1482,9 +1482,12 @@ TEST_P(Http2FloodMitigationTest, RequestMetadata) {
{"header_key1", "header_value1"},
{"header_key2", "header_value2"},
};
Http::MetadataMapVector metadata_map_vector;
metadata_map_vector.push_back(std::make_unique<Http::MetadataMap>(metadata_map));
for (uint32_t frame = 0; frame < AllFrameFloodLimit + 1; ++frame) {
codec_client_->sendMetadata(*request_encoder_, metadata_map);
request_encoder_->encodeMetadata(metadata_map_vector);
}
codec_client_->connection()->dispatcher().run(Event::Dispatcher::RunType::NonBlock);

// Upstream connection should be disconnected
// Downstream client should receive 503 since upstream did not send response headers yet
Expand Down

0 comments on commit a500db5

Please sign in to comment.