Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

repair/row_level.cc:2141:7: runtime error: load of value 48, which is not a valid value for type 'bool' #5531

Closed
bhalevy opened this issue Dec 30, 2019 · 5 comments
Closed

repair/row_level.cc:2141:7: runtime error: load of value 48, which is not a valid value for type 'bool' #5531

bhalevy opened this issue Dec 30, 2019 · 5 comments
Assignees
@asias

Comments

@bhalevy
Copy link
Member

bhalevy commented Dec 30, 2019

Seen in https://jenkins.scylladb.com/view/master/job/scylla-master/job/dtest-debug/353/artifact/logs-debug.2/orphaned/1577482909_bootstrap_test.TestBootstrap.manual_bootstrap_test/node3.log
Scylla version 60a851d
Unfortunately no core dump was generated so we have only the backtrace.

INFO  2019-12-27 21:44:37,278 [shard 0] init - Scylla version 666.development-0.20191227.60a851d3a55 initialization completed.
INFO  2019-12-27 21:44:41,912 [shard 0] repair - starting user-requested repair for keyspace system_traces, repair id 1, options {{ trace -> false}, { incremental -> false}, { jobThreads -> 1}, { primaryRange -> false}, { parallelism -> parallel}}
INFO  2019-12-27 21:44:41,989 [shard 0] repair - Repair 1 out of 519 ranges, id=1, shard=0, keyspace=system_traces, table={node_slow_log, node_slow_log_time_idx, sessions_time_idx, sessions, events}, range=(-9186584531599809051, -9159805761446887102]
INFO  2019-12-27 21:44:41,990 [shard 0] repair - Repair 2 out of 519 ranges, id=1, shard=0, keyspace=system_traces, table={node_slow_log, node_slow_log_time_idx, sessions_time_idx, sessions, events}, range=(-inf, -9215376216204975485]
INFO  2019-12-27 21:44:41,994 [shard 0] repair - Repair 3 out of 519 ranges, id=1, shard=0, keyspace=system_traces, table={node_slow_log, node_slow_log_time_idx, sessions_time_idx, sessions, events}, range=(-9215376216204975485, -9186584531599809051]
INFO  2019-12-27 21:44:42,007 [shard 1] repair - Repair 1 out of 519 ranges, id=1, shard=1, keyspace=system_traces, table={node_slow_log, node_slow_log_time_idx, sessions_time_idx, sessions, events}, range=(-inf, -9215376216204975485]
INFO  2019-12-27 21:44:42,009 [shard 1] repair - Repair 2 out of 519 ranges, id=1, shard=1, keyspace=system_traces, table={node_slow_log, node_slow_log_time_idx, sessions_time_idx, sessions, events}, range=(-9186584531599809051, -9159805761446887102]
INFO  2019-12-27 21:44:42,010 [shard 1] repair - Repair 3 out of 519 ranges, id=1, shard=1, keyspace=system_traces, table={node_slow_log, node_slow_log_time_idx, sessions_time_idx, sessions, events}, range=(-9215376216204975485, -9186584531599809051]
repair/row_level.cc:2141:7: runtime error: load of value 48, which is not a valid value for type 'bool'
Aborting on shard 1.
Backtrace:
  /jenkins/workspace/scylla-master/dtest-debug/scylla/.ccm/scylla-repository/60a851d3a55f973e65ee71af61516f2750fe4aeb/scylla//opt/scylladb/libreloc/libasan.so.5+0x000000000006ce7f
  0x000000000a119d08
  0x000000000a0f5dbc
  0x0000000009f9d0fb
  0x0000000009f9d241
  0x0000000009ff3b08
  0x000000000a02fe0d
  0x000000000a02fe8e
  0x00007f0f8e761b1f
  /jenkins/workspace/scylla-master/dtest-debug/scylla/.ccm/scylla-repository/60a851d3a55f973e65ee71af61516f2750fe4aeb/scylla//opt/scylladb/libreloc/libc.so.6+0x000000000003c624
  /jenkins/workspace/scylla-master/dtest-debug/scylla/.ccm/scylla-repository/60a851d3a55f973e65ee71af61516f2750fe4aeb/scylla//opt/scylladb/libreloc/libc.so.6+0x00000000000258d8
  /jenkins/workspace/scylla-master/dtest-debug/scylla/.ccm/scylla-repository/60a851d3a55f973e65ee71af61516f2750fe4aeb/scylla//opt/scylladb/libreloc/libubsan.so.1+0x000000000001c1e1
  /jenkins/workspace/scylla-master/dtest-debug/scylla/.ccm/scylla-repository/60a851d3a55f973e65ee71af61516f2750fe4aeb/scylla//opt/scylladb/libreloc/libubsan.so.1+0x0000000000026d8b
  /jenkins/workspace/scylla-master/dtest-debug/scylla/.ccm/scylla-repository/60a851d3a55f973e65ee71af61516f2750fe4aeb/scylla//opt/scylladb/libreloc/libubsan.so.1+0x0000000000007e98
  /jenkins/workspace/scylla-master/dtest-debug/scylla/.ccm/scylla-repository/60a851d3a55f973e65ee71af61516f2750fe4aeb/scylla//opt/scylladb/libreloc/libubsan.so.1+0x00000000000092ae
  /jenkins/workspace/scylla-master/dtest-debug/scylla/.ccm/scylla-repository/60a851d3a55f973e65ee71af61516f2750fe4aeb/scylla//opt/scylladb/libreloc/libubsan.so.1+0x000000000000c504
  0x0000000007a21d69
  0x0000000007a222aa
  0x00000000079871c0
  0x00000000077daf61
  0x00000000077dbcdc
  0x0000000007619af1
  0x000000000765edbf
  0x000000000764aa8f
  0x00000000076380b0
  0x000000000761a439
  0x000000000764b26d
  0x000000000764b34c
  0x000000000764b4a4
  0x0000000007638f1f
  0x000000000765f62c
  0x000000000072afd1
  0x00000000006915c8
  0x00000000006916a7
  0x0000000000691800
  0x0000000000612887
  0x0000000000ad2cfe
  0x0000000009fd99b2
  0x0000000009fddb1b
  0x0000000009fe2cc8
  0x0000000009ff6565
  0x000000000a04c555
  0x0000000002a95706
  0x000000000ad0037e
  /jenkins/workspace/scylla-master/dtest-debug/scylla/.ccm/scylla-repository/60a851d3a55f973e65ee71af61516f2750fe4aeb/scylla//opt/scylladb/libreloc/libpthread.so.0+0x00000000000094e1
  /jenkins/workspace/scylla-master/dtest-debug/scylla/.ccm/scylla-repository/60a851d3a55f973e65ee71af61516f2750fe4aeb/scylla//opt/scylladb/libreloc/libc.so.6+0x0000000000101692

Decoded:

row_level_repair::row_level_repair(row_level_repair&&) at /jenkins/workspace/scylla-master/build/scylla/repair/row_level.cc:2141
seastar::internal::do_with_state<row_level_repair, seastar::future<> >::do_with_state(row_level_repair&&) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/do_with.hh:52
std::_MakeUniq<seastar::internal::do_with_state<row_level_repair, seastar::future<> > >::__single_object std::make_unique<seastar::internal::do_with_state<row_level_repair, seastar::future<> >, row_level_repair>(row_level_repair&&) at /usr/include/c++/9/bits/unique_ptr.h:849
auto seastar::do_with<row_level_repair, repair_cf_range_row_level(repair_info&, seastar::basic_sstring<char, unsigned int, 15u, true>, nonwrapping_range<dht::token>, std::vector<gms::inet_address, std::allocator<gms::inet_address> > const&)::{lambda(row_level_repair&)#1}>(row_level_repair&&, repair_cf_range_row_level(repair_info&, seastar::basic_sstring<char, unsigned int, 15u, true>, nonwrapping_range<dht::token>, std::vector<gms::inet_address, std::allocator<gms::inet_address> > const&)::{lambda(row_level_repair&)#1}&&) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/do_with.hh:93
repair_cf_range_row_level(repair_info&, seastar::basic_sstring<char, unsigned int, 15u, true>, nonwrapping_range<dht::token>, std::vector<gms::inet_address, std::allocator<gms::inet_address> > const&) at /jenkins/workspace/scylla-master/build/scylla/repair/row_level.cc:2502 (discriminator 3)
auto repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}::operator()() const::{lambda(auto:1&&)#1}::operator()<seastar::basic_sstring<char, unsigned int, 15u, true>&>({lambda()#2}) const at /jenkins/workspace/scylla-master/build/scylla/repair/repair.cc:978 (discriminator 2)
seastar::future<> seastar::do_void_futurize_helper<seastar::future<> >::apply<repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}::operator()() const::{lambda(auto:1&&)#1}&, seastar::basic_sstring<char, unsigned int, 15u, true>&>({lambda()#2}, seastar::basic_sstring<char, unsigned int, 15u, true>&) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:1530
seastar::future<> seastar::futurize<void>::apply<repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}::operator()() const::{lambda(auto:1&&)#1}&, seastar::basic_sstring<char, unsigned int, 15u, true>&>({lambda()#2}, seastar::basic_sstring<char, unsigned int, 15u, true>&) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:1556
seastar::future<> seastar::do_for_each<__gnu_cxx::__normal_iterator<seastar::basic_sstring<char, unsigned int, 15u, true>*, std::vector<seastar::basic_sstring<char, unsigned int, 15u, true>, std::allocator<seastar::basic_sstring<char, unsigned int, 15u, true> > > >, repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}::operator()() const::{lambda(auto:1&&)#1}>({lambda()#2}, {lambda()#2}, repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}::operator()() const::{lambda(auto:1&&)#1}) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future-util.hh:618
repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}::operator()() const at /jenkins/workspace/scylla-master/build/scylla/repair/repair.cc:982
seastar::apply_helper<repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}&, std::tuple<>&&, std::integer_sequence<unsigned long> >::apply({lambda()#2}, std::tuple) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/apply.hh:36
auto seastar::apply<repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}&>(repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}&, std::tuple<>&&) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/apply.hh:44
seastar::future<> seastar::futurize<seastar::future<> >::apply<repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}&>(repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}&, std::tuple<>&&) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:1563
seastar::future<>::then<repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}, seastar::future<> >(repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}&&)::{lambda()#1}::operator()() at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:1094
seastar::noncopyable_function<seastar::future<> ()>::indirect_vtable_for<seastar::future<>::then<repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}, seastar::future<> >(repair_range(repair_info&, nonwrapping_range<dht::token> const&)::{lambda(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&)#1}::operator()(std::vector<gms::inet_address, std::allocator<gms::inet_address> >&) const::{lambda()#2}&&)::{lambda()#1}>::call(seastar::noncopyable_function<seastar::future<> ()> const*) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/util/noncopyable_function.hh:133
seastar::noncopyable_function<seastar::future<> ()>::operator()() const at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/util/noncopyable_function.hh:187
seastar::apply_helper<seastar::noncopyable_function<seastar::future<> ()>, std::tuple<>&&, std::integer_sequence<unsigned long> >::apply(seastar::noncopyable_function<seastar::future<> ()>&&, std::tuple<>&&) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/apply.hh:36
auto seastar::apply<seastar::noncopyable_function<seastar::future<> ()>>(seastar::noncopyable_function<seastar::future<> ()>&&, std::tuple<>&&) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/apply.hh:44
seastar::future<> seastar::futurize<seastar::future<> >::apply<seastar::noncopyable_function<seastar::future<> ()>>(seastar::noncopyable_function<seastar::future<> ()>&&, std::tuple<>&&) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:1563
seastar::future<>::then_impl<seastar::noncopyable_function<seastar::future<> ()>, seastar::future<> >(seastar::noncopyable_function<seastar::future<> ()>&&)::{lambda()#1}::operator()() const::{lambda(seastar::future_state<>&&)#1}::operator()(seastar::future_state<>) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:1122
seastar::continuation<seastar::future<>::then_impl<seastar::noncopyable_function<seastar::future<> ()>, seastar::future<> >(seastar::noncopyable_function<seastar::future<> ()>&&)::{lambda()#1}::operator()() const::{lambda(seastar::future_state<>&&)#1}>::run_and_dispose() at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:450
seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /jenkins/workspace/scylla-master/build/scylla/seastar/src/core/reactor.cc:2106
seastar::reactor::run_some_tasks() at /jenkins/workspace/scylla-master/build/scylla/seastar/src/core/reactor.cc:2530 (discriminator 2)
seastar::reactor::run() at /jenkins/workspace/scylla-master/build/scylla/seastar/src/core/reactor.cc:2678
seastar::smp::configure(boost::program_options::variables_map, seastar::reactor_config)::{lambda()#3}::operator()() const at /jenkins/workspace/scylla-master/build/scylla/seastar/src/core/reactor.cc:3814 (discriminator 3)
std::_Function_handler<void (), seastar::smp::configure(boost::program_options::variables_map, seastar::reactor_config)::{lambda()#3}>::_M_invoke(std::_Any_data const&) at /usr/include/c++/9/bits/std_function.h:300
std::function<void ()>::operator()() const at /usr/include/c++/9/bits/std_function.h:690
seastar::posix_thread::start_routine(void*) at /jenkins/workspace/scylla-master/build/scylla/seastar/src/core/posix.cc:52

Pretty-printed:

row_level_repair::row_level_repair(row_level_repair&&) at /jenkins/workspace/scylla-master/build/scylla/repair/row_level.cc:2141

seastar::internal::do_with_state<
  row_level_repair,
  seastar::future<>
>::do_with_state(row_level_repair&&) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/do_with.hh:52

std::_MakeUniq<
  seastar::internal::do_with_state<
    row_level_repair,
    seastar::future<>
  >
>::__single_object std::make_unique<
  seastar::internal::do_with_state<
    row_level_repair,
    seastar::future<>
  >,
  row_level_repair
>(row_level_repair&&) at /usr/include/c++/9/bits/unique_ptr.h:849

auto seastar::do_with<
  row_level_repair,
  repair_cf_range_row_level(
    repair_info&,
    seastar::basic_sstring<char, unsigned int, 15u, true>,
    nonwrapping_range<dht::token>,
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    > const&
  )::{lambda(row_level_repair&)#1}
>(
  row_level_repair&&,
  repair_cf_range_row_level(
    repair_info&,
    seastar::basic_sstring<char, unsigned int, 15u, true>,
    nonwrapping_range<dht::token>,
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    > const&
  )::{lambda(row_level_repair&)#1}&&
) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/do_with.hh:93

repair_cf_range_row_level(
  repair_info&,
  seastar::basic_sstring<char, unsigned int, 15u, true>,
  nonwrapping_range<dht::token>,
  std::vector<
    gms::inet_address,
    std::allocator<gms::inet_address>
  > const&
) at /jenkins/workspace/scylla-master/build/scylla/repair/row_level.cc:2502 (discriminator 3)

auto repair_range(
  repair_info&,
  nonwrapping_range<dht::token> const&
)::{
  lambda(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  )#1
}::operator()(
  std::vector<
    gms::inet_address,
    std::allocator<gms::inet_address>
  >&
) const::{lambda()#2}::operator()() const::{lambda(auto:1&&)#1}::operator()<
  seastar::basic_sstring<char, unsigned int, 15u, true>&
>({lambda()#2}) const at /jenkins/workspace/scylla-master/build/scylla/repair/repair.cc:978 (discriminator 2)

seastar::future<> seastar::do_void_futurize_helper<seastar::future<>>::apply<
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2}::operator()() const::{lambda(auto:1&&)#1}&,
  seastar::basic_sstring<char, unsigned int, 15u, true>&
>(
  {lambda()#2},
  seastar::basic_sstring<char, unsigned int, 15u, true>&
) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:1530

seastar::future<> seastar::futurize<void>::apply<
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2}::operator()() const::{lambda(auto:1&&)#1}&,
  seastar::basic_sstring<char, unsigned int, 15u, true>&
>(
  {lambda()#2},
  seastar::basic_sstring<char, unsigned int, 15u, true>&
) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:1556

seastar::future<> seastar::do_for_each<
  __gnu_cxx::__normal_iterator<
    seastar::basic_sstring<char, unsigned int, 15u, true>*,
    std::vector<
      seastar::basic_sstring<char, unsigned int, 15u, true>,
      std::allocator<
        seastar::basic_sstring<char, unsigned int, 15u, true>
      >
    >
  >,
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2}::operator()() const::{lambda(auto:1&&)#1}
>(
  {lambda()#2},
  {lambda()#2},
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2}::operator()() const::{lambda(auto:1&&)#1}
) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future-util.hh:618

repair_range(
  repair_info&,
  nonwrapping_range<dht::token> const&
)::{
  lambda(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  )#1
}::operator()(
  std::vector<
    gms::inet_address,
    std::allocator<gms::inet_address>
  >&
) const::{lambda()#2}::operator()() const at /jenkins/workspace/scylla-master/build/scylla/repair/repair.cc:982

seastar::apply_helper<
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2}&,
  std::tuple<>&&,
  std::integer_sequence<unsigned long>
>::apply({lambda()#2}, std::tuple) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/apply.hh:36

auto seastar::apply<
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2}&
>(
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2}&,
  std::tuple<>&&
) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/apply.hh:44

seastar::future<> seastar::futurize<seastar::future<>>::apply<
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2}&
>(
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2}&,
  std::tuple<>&&
) at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:1563

seastar::future<>::then<
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2},
  seastar::future<>
>(
  repair_range(
    repair_info&,
    nonwrapping_range<dht::token> const&
  )::{
    lambda(
      std::vector<
        gms::inet_address,
        std::allocator<gms::inet_address>
      >&
    )#1
  }::operator()(
    std::vector<
      gms::inet_address,
      std::allocator<gms::inet_address>
    >&
  ) const::{lambda()#2}&&
)::{lambda()#1}::operator()() at /jenkins/workspace/scylla-master/build/scylla/seastar/include/seastar/core/future.hh:1094
@asias
Copy link
Contributor

asias commented Dec 30, 2019

I tried to figure out why could throw: repair/row_level.cc:2141:7: runtime error: load of value 48, which is not a valid value for type 'bool', but failed so far.

Does this reproduce all the time?

@gleb-cloudius
Copy link
Contributor

gleb-cloudius commented Dec 30, 2019 via email

@bhalevy
Copy link
Member Author

bhalevy commented Dec 30, 2019

Does this reproduce all the time?

@asias I see it also in the previous run: https://jenkins.scylladb.com/view/master/job/scylla-master/job/dtest-debug/352/artifact/logs-debug.2/orphaned/1577414952_bootstrap_test.TestBootstrap.manual_bootstrap_test/node3.log

But it's possible/likely this was pre-existing and was just exposed by passing the flags to UBSAN/ASAN to abort on error (scylladb/scylla-pkg@1b3709725dc2c446ca2f8f5c5d3fe4f6ac425e41)

It reproduces for me locally with the following command:

PRINT_DEBUG=true KEEP_TEST_DIR=true CASSANDRA_DIR=<your-path-to>/scylla/build/debug NOSE_PROCESSES=1 SCYLLA_EXT_OPTS="--smp 2 --memory 1024M" ASAN_OPTIONS=disable_coredump=0:abort_on_error=1 UBSAN_OPTIONS=halt_on_error=1:abort_on_error=1 BOOST_TEST_CATCH_SYSTEM_ERRORS=no nosetests -s -v --process-timeout=1800 bootstrap_test:TestBootstrap.manual_bootstrap_test

@asias
Copy link
Contributor

asias commented Dec 30, 2019

Thanks @bhalevy

asias added a commit to asias/scylla that referenced this issue Dec 30, 2019
@asias
repair: Fix use after free for repair_info
b6b36f2 
Fixes: scylladb#5531
@bhalevy bhalevy mentioned this issue Dec 30, 2019
Merged
nyh added a commit that referenced this issue Dec 31, 2019
@nyh
Fix uninitialized members
48a914c 
Merged pull request #5532 from
Benny Halevy:

Initialize bool members in row_level_repair and _storage_service causing
ubsan errors.

Fixes #5531
avikivity pushed a commit that referenced this issue Feb 16, 2020
@bhalevy @avikivity
repair: initialize row_level_repair: _zero_rows
fad143a 
Avoid following UBSAN error:
repair/row_level.cc:2141:7: runtime error: load of value 240, which is not a valid value for type 'bool'

Fixes #5531

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
(cherry picked from commit 474ffb6)
avikivity pushed a commit that referenced this issue Feb 16, 2020
@bhalevy @avikivity
repair: initialize row_level_repair: _zero_rows
b042e27 
Avoid following UBSAN error:
repair/row_level.cc:2141:7: runtime error: load of value 240, which is not a valid value for type 'bool'

Fixes #5531

Signed-off-by: Benny Halevy <bhalevy@scylladb.com>
(cherry picked from commit 474ffb6)
@avikivity
Copy link
Member

Backported to 3.2, 3.1 (3.3 already had it).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@asias asias

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@asias @avikivity @gleb-cloudius @bhalevy @scylladb-promoter