HTTP update to use sharded reference counting - assure a clean shutdown

[slivne]

No description provided.

[dorlaor]

Is this a must for AnglerFish? Why can't we just kill it as a starter

[slivne]

If we will not fix it when people will kill the process they will get

copy paste from Pekka's email ...

.
.
I have reported that a local_is_initialized() assertion triggers once in a
while upon shutdown. I did quick and dirty abort stack trace support with
backtrace() and addr2line and the problem seems to come from Seastar's http
code.

.
.

scylla: /home/penberg/urchin/seastar/core/sharded.hh:434: Service&
seastar::sharded::local() [with Service = database]: Assertion
`local_is_initialized()' failed.
Caught signal 6 (SIGABRT)
stack trace:
#0 0x7fac5a55fae9 in ?? ??:0
#1 0x114cd12 in print_stack_trace(unsigned long)
/home/penberg/urchin/abort.cc:38 (discriminator 2)
#2 0x114d4d2 in abort_handler(int, siginfo_t_, void_)
/home/penberg/urchin/abort.cc:82
#3 0x7fac590820d0 in ?? ??:0
#4 0x7fac58ce98d7 in ?? ??:0
#5 0x7fac58ceb53a in ?? ??:0
#6 0x7fac58ce247d in ?? ??:0
#7 0x7fac58ce2532 in ?? ??:0
#8 0xb5d6d8 in seastar::sharded::local()
/home/penberg/urchin/seastar/core/sharded.hh:435
#9 0x3488338 in operator() /home/penberg/urchin/api/column_family.cc:152
#10 0x34b2009 in _M_invoke /usr/include/c++/4.9.2/functional:2025
#11 0xa0801c in std::function<json::json_return_type (httpd::request
const&)>::operator()(httpd::request const&) const
/usr/include/c++/4.9.2/functional:2440
#12 0xa057f1 in
ZZN5httpd16function_handlerC4ERKSt8functionIFN4json16json_return_typeERKNS_7requestEEEENKUlSt10unique_ptrIS4_St14default_deleteIS4_EESB_INS_5replyESC_ISF_EEE_clESE_SH
/home/penberg/urchin/seastar/http/function_handlers.hh:84
#13 0xa0eb1d in
ZNSt17_Function_handlerIF6futureIISt10unique_ptrIN5httpd5replyESt14default_deleteIS3_EEEES1_INS2_7requestES4_IS8_EES6_EZNS2_16function_handlerC4ERKSt8functionIFN4json16json_return_typeERKS8_EEEUlSA_S6_E_E9_M_invokeERKSt9_Any_dataSA_S6
/usr/include/c++/4.9.2/functional:2025
#14 0xa0ae09 in std::function<future<std::unique_ptr<httpd::reply,
std::default_deletehttpd::reply > > (std::unique_ptr<httpd::request,
std::default_deletehttpd::request >, std::unique_ptr<httpd::reply,
std::default_deletehttpd::reply >)>::operator()(std::unique_p
r<httpd::request, std::default_deletehttpd::request >,
std::unique_ptr<httpd::reply, std::default_deletehttpd::reply >) const
#15 0xa065bd in httpd::function_handler::handle(basic_sstring<char,
unsigned int, 15u> const&, std::unique_ptr<httpd::request,
std::default_deletehttpd::request >, std::unique_ptr<httpd::reply,
std::default_deletehttpd::reply >)
/home/penberg/urchin/seastar/http/function_handlers.hh:106 (discriminator 2)
#16 0xa42e88 in httpd::routes::handle(basic_sstring<char, unsigned int,
15u> const&, std::unique_ptr<httpd::request,
std::default_deletehttpd::request >, std::unique_ptr<httpd::reply,
std::default_deletehttpd::reply >)
/home/penberg/urchin/seastar/http/routes.cc:76
#17 0xb4641f in
httpd::http_server::connection::generate_reply(std::unique_ptr<httpd::request,
std::default_deletehttpd::request >)
/home/penberg/urchin/seastar/http/httpd.hh:359 (discriminator 2)
#18 0xb41cff in
httpd::http_server::connection::read_one()::{lambda()#1}::operator()()::{lambda()#1}::operator()()
/home/penberg/urchin/seastar/http/httpd.hh:189
#19 0xb869d1 in
apply_helper<httpd::http_server::connection::read_one()::{lambda()#1}::operator()()::{lambda()#1},
std::tuple<>&&, std::integer_sequence

::apply({lambda()#1}&&, std::tuple)
/home/penberg/urchin/seastar/core/apply.hh:35
#20 0xb86aae in auto
applyhttpd::http_server::connection::read_one()::{lambda()#1}::operator()()::{lambda()#1}(httpd::http_server::connection::read_one()::{lambda()#1}::operator()()::{lambda()#1}&&,
std::tuple<>&&) /home/penberg/urchin/seastar/core/apply.hh:43
#21 0xb86b93 in future futurize<future
::applyhttpd::http_server::connection::read_one()::{lambda()#1}::operator()()::{lambda()#1}(httpd::http_server::connection::read_one()::{lambda()#1}::operator()()::{lambda()#1}&&,
std::tuple<>&&) /home/penberg/urchin/seastar/core/future.hh:1082
#22 0xe1e31c in
ZZN6futureIIEE4thenIZZN5httpd11http_server10connection8read_oneEvENUlvE_clEvEUlvE_S_IIbEEEET0_OT_ENUlOT_E_clI12future_stateIIEEEEDaSA
/home/penberg/urchin/seastar/core/future.hh:778
#23 0xe1e597 in
_ZN12continuationIZN6futureIIEE4thenIZZN5httpd11http_server10connection8read_oneEvENUlvE_clEvEUlvE_S0_IIbEEEET0_OT_EUlOT_E_IEE3runEv
/home/penberg/urchin/seastar/core/future.hh:360
#24 0x4270c9 in
reactor::run_tasks(circular_buffer<std::unique_ptr<task,
std::default_delete >, std::allocator<std::unique_ptr<task,
std::default_delete > > >&, unsigned long)
/home/penberg/urchin/seastar/core/reactor.cc:1094
#25 0x429f4e in reactor::run()
/home/penberg/urchin/seastar/core/reactor.cc:1191
#26 0x774120 in app_template::run_deprecated(int, char**,
std::function<void ()>&&)
/home/penberg/urchin/seastar/core/app-template.cc:122
#27 0xa6bd17 in main /home/penberg/urchin/main.cc:266 (discriminator 1)
#28 0x7fac58cd4fe0 in ?? ??:0
#29 0x4122e9 in _start ??:?

On Mon, Sep 7, 2015 at 10:32 AM, Dor Laor notifications@github.com wrote:

Is this a must for AnglerFish? Why can't we just kill it as a starter

—
Reply to this email directly or view it on GitHub
#292 (comment)
.

[penberg]

@dorlaor It's a shutdown issue so yes, we need to fix it for first beta.

[bhalevy]

@xemul are you working on this in the context of another issue?

[nyh]

This is a seven year old issue, we already solved many such unclean shutdown issues and probably solved this one too without noticing this issue. Closing.