envoy:filter_fuzz_test: ASSERT: base_interval_ > 0 #26840
Conversation
Envoy::JitteredExponentialBackOffStrategy::JitteredExponentialBackOffStrategy(unsigned long, unsigned long, Envoy::Random::RandomGenerator&) /proc/self/cwd/source/common/common/backoff_strategy.cc:0 Signed-off-by: Yanjun Xiang <yanjunxiang@google.com>
|
From the configuration, the base_interval_ms is a 64-bit number: 0x100000000. In source/config/utility.cc, it is converted into a 32-bit number base_interval_ms, and become zero. Thus the ASSERT(base_interval_ > 0) crashes. This is a regression issue by #24701, which changed the base_interval_ms from a uint64_t number into a uint32_t number. Consider the duration second is a uint64_t number, also the base_interval_ms in class JitteredExponentialBackOffStrategy is also a uint64_t number, let's change it back to uint64_t as well. The alternative solution is to add a lte PGV in the proto: . However, that change will be some sort of a broken change? even realistically nobody will config a base_interval_ms that big. Considering this is a regression issue, change it back to uint64_t here make more sense to me. |
|
/assign @yanavlasov @KBaichoo @adisuissa |
|
Another issue here is that after we fix the above ASSERT crash, there is a WASM filter crash observed with below traceback. This looks to me like a separate issue. I would think we should deal with it separately. [ RUN ] CorpusExamples/FuzzerCorpusTest.RunOneCorpusFile/30 source/extensions/filters/http/wasm/wasm_filter.h:34:54: runtime error: member call on null pointer of type 'Envoy::Extensions::Common::Wasm::PluginHandleSharedPtrThreadLocal' SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior source/extensions/filters/http/wasm/wasm_filter.h:34:54 in |
HI, @stevenzzzz , Could you please take a look this crash in wasm_filter code? It is easy to reproduce once you download the file: test/extensions/filters/http/common/fuzz/filter_corpus/clusterfuzz-minimized-filter_fuzz_test-6701612180832256 and copied to your workspace this directory, then run:~/envoy_fuzz_test_issue/envoy$ bazel test -c dbg --config clang-asan test/extensions/filters/http/common/fuzz:filter_fuzz_test |
|
I can take a look. |
|
This looks like some constraint annotations are missing from the API. /wait-any |
👍 |
|
/wait-any |
|
This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions! |
github-actions
bot
added
the
stale
|
This pull request has been automatically closed because it has not had activity in the last 37 days. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions! |
|
Below is the test logs. @mpwarres @stevenzzzz coud you please take a look? It looks to me once the configure has remote code fetching config with a uri, then it crashed: |
|
Function call path: which create the FilterConfig object and the callback function: It also create the the callback function to call tls_slot->set() However, it looks like the tls_slot_->set() is not called during call function createWasm() here: crashed here when call this callback function, which in turn call createFilter(): |
This fuzz test ASSERT is triggered by a large proto config:
name: "envoy.filters.http.wasm"
typed_config {
[type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm] {
config {
root_id: "||"
vm_config {
code {
remote {
http_uri {
uri: "envoy.filters.http.rbac"
cluster: "2"
timeout {
}
}
sha256: "z"
retry_policy {
retry_back_off {
base_interval {
seconds: 4294967296 ->>> hex: 100000000 , if converted into uint32_t, becomes zero.
}
}
}
}
}
}
}
detail logs:
https://oss-fuzz.com/testcase-detail/6701612180832256
tracebacks:
Running: /mnt/scratch0/clusterfuzz/bot/inputs/fuzzer-testcases/testcase
| [2023-04-19 14:39:39.538][27194][critical][assert] [source/common/common/backoff_strategy.cc:9] assert failure: base_interval_ > 0.
| AddressSanitizer:DEADLYSIGNAL
| =================================================================
| ==27194==ERROR: AddressSanitizer: ABRT on unknown address 0x053900006a3a (pc 0x79a5d4adc00b bp 0x7ffffc76a6b0 sp 0x7ffffc76a3f0 T0)
| SCARINESS: 10 (signal)
| #0 0x79a5d4adc00b in raise /build/glibc-SzIz7B/glibc-2.31/sysdeps/unix/sysv/linux/raise.c:51:1
| #1 0x79a5d4abb858 in abort /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:79:7
| #2 0x5f7930a in Envoy::JitteredExponentialBackOffStrategy::JitteredExponentialBackOffStrategy(unsigned long, unsigned long, Envoy::Random::RandomGenerator&) /proc/self/cwd/source/common/common/backoff_strategy.cc:0
| #3 0x5f782be in make_unique<Envoy::JitteredExponentialBackOffStrategy, unsigned int &, unsigned int &, Envoy::Random::RandomGenerator &> /usr/local/include/c++/v1/__memory/unique_ptr.h:724:32
| #4 0x5f782be in Envoy::Config::Utility::buildJitteredExponentialBackOffStrategy(std::__1::optional<envoy::config::core::v3::BackoffStrategy const>, Envoy::Random::RandomGenerator&, unsigned int, std::__1::optional) /proc/self/cwd/source/common/config/utility.cc:324:12
| #5 0x5dd1102 in std::__1::unique_ptr<Envoy::JitteredExponentialBackOffStrategy, std::__1::default_deleteEnvoy::JitteredExponentialBackOffStrategy > Envoy::Config::Utility::prepareJitteredExponentialBackOffStrategyenvoy::config::core::v3::RemoteDataSource(envoy::config::core::v3::RemoteDataSource const&, Envoy::Random::RandomGenerator&, unsigned int, std::__1::optional) /proc/self/cwd/source/common/config/utility.h:576:14
| #6 0x5dd0410 in Envoy::Config::DataSource::RemoteAsyncDataProvider::RemoteAsyncDataProvider(Envoy::Upstream::ClusterManager&, Envoy::Init::Manager&, envoy::config::core::v3::RemoteDataSource const&, Envoy::Event::Dispatcher&, Envoy::Random::RandomGenerator&, bool, std::__1::function<void (std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&)>&&) /proc/self/cwd/source/common/config/datasource.cc:69:23
| #7 0x57dc5e1 in make_unique<Envoy::Config::DataSource::RemoteAsyncDataProvider, Envoy::Upstream::ClusterManager &, Envoy::Init::Manager &, const envoy::config::core::v3::RemoteDataSource &, Envoy::Event::Dispatcher &, Envoy::Random::RandomGenerator &, bool, (lambda at source/extensions/common/wasm/wasm.cc:414:27) &> /usr/local/include/c++/v1/__memory/unique_ptr.h:724:32
| #8 0x57dc5e1 in Envoy::Extensions::Common::Wasm::createWasm(std::__1::shared_ptrEnvoy::Extensions::Common::Wasm::Plugin const&, std::__1::shared_ptrEnvoy::Stats::Scope const&, Envoy::Upstream::ClusterManager&, Envoy::Init::Manager&, Envoy::Event::Dispatcher&, Envoy::Api::Api&, Envoy::Server::ServerLifecycleNotifier&, std::__1::unique_ptr<Envoy::Config::DataSource::RemoteAsyncDataProvider, std::__1::default_deleteEnvoy::Config::DataSource::RemoteAsyncDataProvider >&, std::__1::function<void (std::__1::shared_ptrEnvoy::Extensions::Common::Wasm::WasmHandle)>&&, std::__1::function<proxy_wasm::ContextBase* (Envoy::Extensions::Common::Wasm::Wasm*, std::__1::shared_ptrEnvoy::Extensions::Common::Wasm::Plugin const&)>) /proc/self/cwd/source/extensions/common/wasm/wasm.cc:457:30
| #9 0x5746fe6 in Envoy::Extensions::HttpFilters::Wasm::FilterConfig::FilterConfig(envoy::extensions::filters::http::wasm::v3::Wasm const&, Envoy::Server::Configuration::FactoryContext&) /proc/self/cwd/source/extensions/filters/http/wasm/wasm_filter.cc:23:8
| #10 0x574039c in __shared_ptr_emplace<const envoy::extensions::filters::http::wasm::v3::Wasm &, Envoy::Server::Configuration::FactoryContext &> /usr/local/include/c++/v1/__memory/shared_ptr.h:294:37
| #11 0x574039c in allocate_shared<Envoy::Extensions::HttpFilters::Wasm::FilterConfig, std::__1::allocatorEnvoy::Extensions::HttpFilters::Wasm::FilterConfig, const envoy::extensions::filters::http::wasm::v3::Wasm &, Envoy::Server::Configuration::FactoryContext &, void> /usr/local/include/c++/v1/__memory/shared_ptr.h:953:55
| #12 0x574039c in make_shared<Envoy::Extensions::HttpFilters::Wasm::FilterConfig, const envoy::extensions::filters::http::wasm::v3::Wasm &, Envoy::Server::Configuration::FactoryContext &, void> /usr/local/include/c++/v1/__memory/shared_ptr.h:962:12
| #13 0x574039c in Envoy::Extensions::HttpFilters::Wasm::WasmFilterConfig::createFilterFactoryFromProtoTyped(envoy::extensions::filters::http::wasm::v3::Wasm const&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, Envoy::Server::Configuration::FactoryContext&) /proc/self/cwd/source/extensions/filters/http/wasm/config.cc:21:24
| #14 0x5741d37 in createFilterFactoryFromProto /proc/self/cwd/source/extensions/filters/http/common/factory_base.h:71:12
| #15 0x5741d37 in non-virtual thunk to Envoy::Extensions::HttpFilters::Common::FactoryBase<envoy::extensions::filters::http::wasm::v3::Wasm, envoy::extensions::filters::http::wasm::v3::Wasm>::createFilterFactoryFromProto(google::protobuf::Message const&, std::__1::basic_string<char, std::__1::char_traits, std::__1::allocator > const&, Envoy::Server::Configuration::FactoryContext&) /proc/self/cwd/source/extensions/filters/http/common/factory_base.h:0
| #16 0x20bb2cf in Envoy::Extensions::HttpFilters::UberFilterFuzzer::fuzz(envoy::extensions::filters::network::http_connection_manager::v3::HttpFilter const&, test::fuzz::HttpData const&, test::fuzz::HttpData const&) /proc/self/cwd/test/extensions/filters/http/common/fuzz/uber_filter.cc:76:19
| #17 0x1fa3a7c in TestOneProtoInput /proc/self/cwd/test/extensions/filters/http/common/fuzz/filter_fuzz_test.cc:78:12
| #18 0x1fa3a7c in LLVMFuzzerTestOneInput /proc/self/cwd/test/extensions/filters/http/common/fuzz/filter_fuzz_test.cc:13:1
| #19 0x1e76913 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerLoop.cpp:611:15
| #20 0x1e61422 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:324:6
| #21 0x1e66ccc in fuzzer::FuzzerDriver(int*, char***, int ()(unsigned char const, unsigned long)) /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerDriver.cpp:860:9
| #22 0x1e90e52 in main /src/llvm-project/compiler-rt/lib/fuzzer/FuzzerMain.cpp:20:10
| #23 0x79a5d4abd082 in __libc_start_main /build/glibc-SzIz7B/glibc-2.31/csu/libc-start.c:308:16
| #24 0x1e575ed in _start
|
| AddressSanitizer can not provide additional info.
| SUMMARY: AddressSanitizer: ABRT (/lib/x86_64-linux-gnu/libc.so.6+0x4300b) (BuildId: 1878e6b475720c7c51969e69ab2d276fae6d1dee)
| ==27194==ABORTING
Commit Message:
Additional Description:
Risk Level:
Testing:
Docs Changes:
Release Notes:
Platform Specific Features:
[Optional Runtime guard:]
[Optional Fixes #Issue]
[Optional Fixes commit #PR or SHA]
[Optional Deprecated:]
[Optional API Considerations:]