k/fetch_request: limit max fetch_request version to 6 #24
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In redpanda we do not yet support incremental fetch requests. The incremental fetches are there to minimize overhead of sending request/responses for partitions that didn't change since previous fetch. For more details: https://cwiki.apache.org/confluence/display/KAFKA/KIP-227%3A+Introduce+Incremental+FetchRequests+to+Increase+Partition+Scalability Signed-off-by: Michal Maslanka <michal@vectorized.io>
|
Have you done a sanity check with some different clients and benchmarks? My worry is that some client requires higher level of support, but doesn't itself need incremental fetch support, which would put us in a pickle? It's true that we did simply start off with the highest number version that worked, but we haven't (unfortunately) kept track of the requirements matrix. |
|
the java kafka clients seem to be fairly good in this respect. i think we'd want to try sarama, python, librdkafka, etc... |
|
I assumed that supporting older versions is fine. This way the clients wouldn't work with older kafka brokers |
not necessarily. i think sarama has some messed up versioning ont he client side. |
|
question is this critical for the update? i think split brain makes sense to push for release + tag |
|
We had a chat with @dotnwat and decided to leave it for now and try to implement incremental fetching. |
|
makes sense, i closed this |
mmaslankaprv
pushed a commit
to mmaslankaprv/redpanda
that referenced
this pull request
Sep 28, 2021
Issue redpanda-data#24 showed that the range balancer was broken. On first glance, the issue appears to be because the consumerIdx wraps past the end of potentialConsumers, and the obvious fix is to just inc and mod. This would be an incorrect fix. The real problem lies above in determining div and rem. The whole point is that (div*(potentialConsumers) + rem) == num partitions, thus if we drain div and one from rem every loop, then we will never increment past the end of potential consumers, because partitions will be completely drained on the last consumer. The prior logic accidentally made div smaller and rem larger, thus giving too few partitions to each consumer, and then in some scenarios, would cause us to exhaust consumers while still draining partitions. I will be adding unit tests later.
mmaslankaprv
pushed a commit
to mmaslankaprv/redpanda
that referenced
this pull request
Sep 28, 2021
ATOMIC STORES ============= This commit switches the consumer type to be stored in an atomic value, rather than a uint8 type that specifies which pointer to use guarded by a mutex. This switch fundamentally arises from trying to unblock metadata updates while a group consumer is leaving the group. Previously, we had to grab the consumer lock to check the consumer type to check if we were consuming with regex. We avoid that now. This actually makes a bunch of other areas simpler as well -- many places needed the group consumer to do some logic on the group consumer directly. Previously, we had to grab the consumer lock, and for simplicity we held it through the function. Holding it was unnecessary, and now we avoid grabbing the lock at all. Anything that sets the consumer value grabs a new dedicated assignMu. The dedicated assignMu allows us to unblock a clean group leave, which may (in a shortly incoming commit) grab the consumer mu to assign partitions on revoke. We do not have to worry about TOCTOU: the guarantee is things work in order. If a person concurrently modifies something, they may change the outcome of stuff that was set into sequence by original events, but the outcome is still sound according to our client. Particularly, a later metadata update will trigger the right sequence for the new assignment. Same type of logic with offset setting, but people should not be doing that concurrently with assigning and whatnot. UPDATES & LOCK ORDERING FIXES ============================= This is the bulk of this commit that mostly fixes some lock orderings and missing locks. This should fix the panic in redpanda-data#24 by at least logging on when it would be detected and continuing, however the bug itself is still a mystery. The debug logs about what the balance results were should help, though, if this crops up again. There are a few lock ordering fixes in here which are now documented extensively. Notably, PollFetches needs the consumer mu, and there is a huge reason as to why. The prerevoke and revoke logic, and how we ensure things are done before returning sometimes, is all more extensively documented. Lastly, all instances of assignPartitions is now properly guarded by the consumer mutex. Prior, some instances were not.
andrewhsu
pushed a commit
to andrewhsu/redpanda
that referenced
this pull request
Jan 6, 2023
dotnwat
pushed a commit
that referenced
this pull request
Feb 2, 2023
Without this change there is a data race that can result in a
heap-use-after-free.
```
==3190349==ERROR: AddressSanitizer: heap-use-after-free on address 0x60b000257420 at pc 0x14995faf26e4 bp 0x7ffd3e04b710 sp 0x7ffd3e04b708
READ of size 8 at 0x60b000257420 thread T0
#0 0x14995faf26e3 in profile_flush_file_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:468:24
#1 0x14995faf0cef in profile_close_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:559:14
#2 0x14995fb0c086 in profile_release /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_init.c:514:13
#3 0x14995fa924c6 in k5_os_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:506:9
#4 0x14995f938b87 in krb5_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:294:5
#5 0x14995ffc41c3 in krb5_gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/delete_sec_context.c:87:9
#6 0x14995ff399a6 in gssint_delete_internal_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_glue.c:606:15
#7 0x14995ff25450 in gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_delete_sec_context.c:91:11
#8 0x55e258b9c641 in security::gss::ctx_id::reset() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:170:13
#9 0x55e258b9c408 in security::gss::ctx_id::~ctx_id() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:165:17
#10 0x55e258b9c37c in security::gssapi_authenticator::impl::~impl() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:116:29
#11 0x55e258b9c23c in std::__1::default_delete<security::gssapi_authenticator::impl>::operator()(security::gssapi_authenticator::impl*) const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:54:5
#12 0x55e258b65324 in std::__1::unique_ptr<security::gssapi_authenticator::impl, std::__1::default_delete<security::gssapi_authenticator::impl> >::reset(security::gssapi_authenticator::impl*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:315:7
#13 0x55e258b5f302 in security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>) (.resume) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:202:15
#14 0x55e258b11830 in std::__1::coroutine_handle<seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type>::resume() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__coroutine/coroutine_handle.h:168:9
#15 0x55e258b11315 in seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type::run_and_dispose() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/rp_deps_install/include/seastar/core/coroutine.hh:78:20
#16 0x55e2594a1ead in seastar::reactor::run_tasks(seastar::reactor::task_queue&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2330:14
#17 0x55e2594a7d0e in seastar::reactor::run_some_tasks() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2737:9
#18 0x55e2594ac86f in seastar::reactor::do_run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2906:9
#19 0x55e2594aa3f8 in seastar::reactor::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2789:16
#20 0x55e2591c0a7c in seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:265:31
#21 0x55e2591be135 in seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:156:12
#22 0x55e2470c46e4 in application::run(int, char**) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/application.cc:323:16
#23 0x55e247081d16 in main /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/main.cc:22:16
#24 0x14995cc29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
#25 0x14995cc29e3f in __libc_start_main csu/../csu/libc-start.c:392:3
#26 0x55e246fc1844 in _start (/home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/bin/redpanda+0x3a678844) (BuildId: 0a81be2927716d279ef7fc96d20cc5d5dfbd1cb2)
0x60b000257420 is located 0 bytes inside of 104-byte region [0x60b000257420,0x60b000257488)
freed by thread T0 here:
#0 0x55e2470443e2 in free /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/src/compiler-rt/lib/asan/asan_malloc_linux.cpp:52:3
#1 0x14995faf3ed4 in profile_free_file_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:552:5
#2 0x14995faf3860 in profile_dereference_data_locked /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:509:9
#3 0x14995faf0b74 in profile_dereference_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:502:5
#4 0x14995faf3fc1 in profile_free_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:523:5
#5 0x14995faf0d38 in profile_close_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:562:5
#6 0x14995fb0c086 in profile_release /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_init.c:514:13
#7 0x14995fa924c6 in k5_os_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:506:9
#8 0x14995f938b87 in krb5_free_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:294:5
#9 0x14995ffc41c3 in krb5_gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/delete_sec_context.c:87:9
#10 0x14995ff399a6 in gssint_delete_internal_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_glue.c:606:15
#11 0x14995ff25450 in gss_delete_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_delete_sec_context.c:91:11
#12 0x55e258b9c641 in security::gss::ctx_id::reset() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:170:13
#13 0x55e258b9c408 in security::gss::ctx_id::~ctx_id() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi.h:165:17
#14 0x55e258b9c37c in security::gssapi_authenticator::impl::~impl() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:116:29
#15 0x55e258b9c23c in std::__1::default_delete<security::gssapi_authenticator::impl>::operator()(security::gssapi_authenticator::impl*) const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:54:5
#16 0x55e258b65324 in std::__1::unique_ptr<security::gssapi_authenticator::impl, std::__1::default_delete<security::gssapi_authenticator::impl> >::reset(security::gssapi_authenticator::impl*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__memory/unique_ptr.h:315:7
#17 0x55e258b5f302 in security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>) (.resume) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:202:15
#18 0x55e258b11830 in std::__1::coroutine_handle<seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type>::resume() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__coroutine/coroutine_handle.h:168:9
#19 0x55e258b11315 in seastar::internal::coroutine_traits_base<boost::outcome_v2::basic_result<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, boost::outcome_v2::policy::error_code_throw_as_system_error<seastar::basic_sstring<unsigned char, unsigned int, 31u, false>, std::__1::error_code, void> > >::promise_type::run_and_dispose() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/rp_deps_install/include/seastar/core/coroutine.hh:78:20
#20 0x55e2594a1ead in seastar::reactor::run_tasks(seastar::reactor::task_queue&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2330:14
#21 0x55e2594a7d0e in seastar::reactor::run_some_tasks() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2737:9
#22 0x55e2594ac86f in seastar::reactor::do_run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2906:9
#23 0x55e2594aa3f8 in seastar::reactor::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2789:16
#24 0x55e2591c0a7c in seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:265:31
#25 0x55e2591be135 in seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:156:12
#26 0x55e2470c46e4 in application::run(int, char**) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/application.cc:323:16
#27 0x55e247081d16 in main /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/main.cc:22:16
#28 0x14995cc29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
previously allocated by thread T4 here:
#0 0x55e24704468e in malloc /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/src/compiler-rt/lib/asan/asan_malloc_linux.cpp:69:3
#1 0x14995faee48d in profile_make_prf_data /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:150:9
#2 0x14995faf0360 in profile_open_file /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:233:12
#3 0x14995fb0ab1e in profile_init_flags /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_init.c:190:22
#4 0x14995fa90bc0 in os_init_paths /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:387:18
#5 0x14995fa9090e in k5_os_init_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/os/init_os_ctx.c:438:18
#6 0x14995f936ece in krb5_init_context_profile /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:209:14
#7 0x14995f936871 in krb5_init_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/krb5/krb/init_ctx.c:139:12
#8 0x14995fffc70b in krb5_gss_init_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/init_sec_context.c:1047:12
#9 0x14995ffa0719 in kg_accept_krb5 /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/accept_sec_context.c:694:12
#10 0x14995ff9e686 in krb5_gss_accept_sec_context_ext /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/accept_sec_context.c:1311:12
#11 0x14995ffaaf9c in krb5_gss_accept_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/krb5/accept_sec_context.c:1340:12
#12 0x14995ff05dbd in gss_accept_sec_context /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/lib/gssapi/mechglue/g_accept_sec_context.c:266:15
#13 0x55e258b2b157 in security::gssapi_authenticator::impl::more(std::__1::basic_string_view<unsigned char, std::__1::char_traits<unsigned char> >) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:292:25
#14 0x55e258b2946c in security::gssapi_authenticator::impl::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:218:16
#15 0x55e258b3b562 in security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>)::$_0::operator()() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/security/gssapi_authenticator.cc:195:25
#16 0x55e258b3a96a in ssx::impl::worker_task<security::gssapi_authenticator::authenticate(seastar::basic_sstring<unsigned char, unsigned int, 31u, false>)::$_0>::process(seastar::alien::instance&, unsigned int) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:67:37
#17 0x55e24764d1bc in ssx::impl::thread_worker::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:159:20
#18 0x55e24764ca9f in ssx::impl::thread_worker::start()::'lambda'()::operator()() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:104:49
#19 0x55e24764c910 in decltype(static_cast<ssx::impl::thread_worker::start()::'lambda'()>(fp)()) std::__1::__invoke<ssx::impl::thread_worker::start()::'lambda'()>(ssx::impl::thread_worker::start()::'lambda'()&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/type_traits:3640:23
#20 0x55e24764c808 in void std::__1::__thread_execute<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ssx::impl::thread_worker::start()::'lambda'()>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ssx::impl::thread_worker::start()::'lambda'()>&, std::__1::__tuple_indices<>) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/thread:282:5
#21 0x55e24764ad39 in void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, ssx::impl::thread_worker::start()::'lambda'()> >(void*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/thread:293:5
#22 0x14995cc94b42 in start_thread nptl/./nptl/pthread_create.c:442:8
Thread T4 created by T0 here:
#0 0x55e24702db0c in pthread_create /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/src/compiler-rt/lib/asan/asan_interceptors.cpp:208:3
#1 0x55e24764ab0c in std::__1::__libcpp_thread_create(unsigned long*, void* (*)(void*), void*) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__threading_support:375:10
#2 0x55e24764a106 in std::__1::thread::thread<ssx::impl::thread_worker::start()::'lambda'(), void>(ssx::impl::thread_worker::start()::'lambda'()&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/thread:309:16
#3 0x55e247606d40 in ssx::impl::thread_worker::start() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:104:19
#4 0x55e247305329 in ssx::thread_worker::start() (.resume) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/ssx/thread_worker.h:199:15
#5 0x55e24756b680 in std::__1::coroutine_handle<seastar::internal::coroutine_traits_base<void>::promise_type>::resume() const /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/llvm/install/bin/../include/c++/v1/__coroutine/coroutine_handle.h:168:9
#6 0x55e24756b1a5 in seastar::internal::coroutine_traits_base<void>::promise_type::run_and_dispose() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/rp_deps_install/include/seastar/core/coroutine.hh:120:20
#7 0x55e2594a1ead in seastar::reactor::run_tasks(seastar::reactor::task_queue&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2330:14
#8 0x55e2594a7d0e in seastar::reactor::run_some_tasks() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2737:9
#9 0x55e2594ac86f in seastar::reactor::do_run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2906:9
#10 0x55e2594aa3f8 in seastar::reactor::run() /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/reactor.cc:2789:16
#11 0x55e2591c0a7c in seastar::app_template::run_deprecated(int, char**, std::__1::function<void ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:265:31
#12 0x55e2591be135 in seastar::app_template::run(int, char**, std::__1::function<seastar::future<int> ()>&&) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/seastar-prefix/src/seastar/src/core/app-template.cc:156:12
#13 0x55e2470c46e4 in application::run(int, char**) /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/application.cc:323:16
#14 0x55e247081d16 in main /home/ben/development/src/github.com/BenPope/redpanda-clang-13/src/v/redpanda/main.cc:22:16
#15 0x14995cc29d8f in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
SUMMARY: AddressSanitizer: heap-use-after-free /home/ben/development/src/github.com/BenPope/redpanda-clang-13/vbuild/debug/clang/v_deps_build/krb5-prefix/src/krb5/src/util/profile/prof_file.c:468:24 in profile_flush_file_data
Shadow bytes around the buggy address:
0x0c1680042e30: fd fd fd fd fd fd fd fd fd fd fa fa fa fa fa fa
0x0c1680042e40: fa fa fd fd fd fd fd fd fd fd fd fd fd fd fd fa
0x0c1680042e50: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x0c1680042e60: fd fd fd fd fd fd fa fa fa fa fa fa fa fa fd fd
0x0c1680042e70: fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa fa
=>0x0c1680042e80: fa fa fa fa[fd]fd fd fd fd fd fd fd fd fd fd fd
0x0c1680042e90: fd fa fa fa fa fa fa fa fa fa fd fd fd fd fd fd
0x0c1680042ea0: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
0x0c1680042eb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fa fa fa
0x0c1680042ec0: fa fa fa fa fa fa fd fd fd fd fd fd fd fd fd fd
0x0c1680042ed0: fd fd fd fd fa fa fa fa fa fa fa fa fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
==3190349==ABORTING
```
Signed-off-by: Ben Pope <ben@redpanda.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In redpanda we do not yet support incremental fetch requests. The
incremental fetches are there to minimize overhead of sending
request/responses for partitions that didn't change since previous
fetch.
For more details:
KIP-227
Signed-off-by: Michal Maslanka michal@vectorized.io