Assert on null t_state.transact_return_point

[shinrich]

Seeing this crash now after putting in work around for issue #1401.

#0  0x00002b20159b7625 in raise () from /lib64/libc.so.6
#1  0x00002b20159b8d8d in abort () from /lib64/libc.so.6
#2  0x00002b20132c8a02 in ink_abort (message_format=0x2b20132de4f0 "%s:%d: failed assertion `%s`") at ink_error.cc:99
#3  0x00002b20132c6106 in _ink_assert (expression=0x7d8b20 "t_state.transact_return_point != nullptr", file=0x7d5e2c "HttpSM.cc", 
    line=7197) at ink_assert.cc:37
#4  0x00000000005fbbeb in HttpSM::call_transact_and_set_next_state (this=0x2b20c8096c00, f=0) at HttpSM.cc:7197
#5  0x00000000005fbefe in HttpSM::set_next_state (this=0x2b20c8096c00) at HttpSM.cc:7253
#6  0x00000000005fbd43 in HttpSM::call_transact_and_set_next_state (this=0x2b20c8096c00, f=0) at HttpSM.cc:7206
#7  0x00000000005e7623 in HttpSM::handle_api_return (this=0x2b20c8096c00) at HttpSM.cc:1606
#8  0x00000000005e7454 in HttpSM::state_api_callout (this=0x2b20c8096c00, event=60000, data=0x0) at HttpSM.cc:1544
#9  0x00000000005e6a22 in HttpSM::state_api_callback (this=0x2b20c8096c00, event=60000, data=0x0) at HttpSM.cc:1340
#10 0x0000000000536236 in TSHttpTxnReenable (txnp=0x2b20c8096c00, event=60000) at InkAPI.cc:5883
#11 0x00002aaaae401263 in http_hook (contp=0x2204f40, event=60016, edata=0x2b20c8096c00) at INKPluginInit.cc:408
#12 0x000000000052a4d3 in INKContInternal::handle_event (this=0x2204f40, event=60016, edata=0x2b20c8096c00) at InkAPI.cc:1048
#13 0x000000000051592e in Continuation::handleEvent (this=0x2204f40, event=60016, data=0x2b20c8096c00)
    at /home/shinrich/yats_build/trafficserver/iocore/eventsystem/I_Continuation.h:153
#14 0x000000000052acd3 in APIHook::invoke (this=0x2206f80, event=60016, edata=0x2b20c8096c00) at InkAPI.cc:1267
#15 0x00000000005e7134 in HttpSM::state_api_callout (this=0x2b20c8096c00, event=0, data=0x0) at HttpSM.cc:1462
#16 0x00000000005f4d91 in HttpSM::do_api_callout_internal (this=0x2b20c8096c00) at HttpSM.cc:5168
#17 0x0000000000602fa3 in HttpSM::do_api_callout (this=0x2b20c8096c00) at HttpSM.cc:438
#18 0x00000000005fbdb0 in HttpSM::set_next_state (this=0x2b20c8096c00) at HttpSM.cc:7239
#19 0x00000000005fbd43 in HttpSM::call_transact_and_set_next_state (this=0x2b20c8096c00, f=0) at HttpSM.cc:7206
#20 0x00000000005e7623 in HttpSM::handle_api_return (this=0x2b20c8096c00) at HttpSM.cc:1606
#21 0x00000000005e7454 in HttpSM::state_api_callout (this=0x2b20c8096c00, event=60000, data=0x0) at HttpSM.cc:1544
#22 0x00000000005e6a22 in HttpSM::state_api_callback (this=0x2b20c8096c00, event=60000, data=0x0) at HttpSM.cc:1340
#23 0x0000000000536236 in TSHttpTxnReenable (txnp=0x2b20c8096c00, event=60000) at InkAPI.cc:5883
#24 0x00002aaaafd35999 in carpPluginHook (contp=0x2204dc0, event=60002, edata=0x2b20c8096c00) at carp.cc:542
#25 0x000000000052a4d3 in INKContInternal::handle_event (this=0x2204dc0, event=60002, edata=0x2b20c8096c00) at InkAPI.cc:1048
#26 0x000000000051592e in Continuation::handleEvent (this=0x2204dc0, event=60002, data=0x2b20c8096c00)
    at /home/shinrich/yats_build/trafficserver/iocore/eventsystem/I_Continuation.h:153
#27 0x000000000052acd3 in APIHook::invoke (this=0x2206f00, event=60002, edata=0x2b20c8096c00) at InkAPI.cc:1267
#28 0x00000000005e7134 in HttpSM::state_api_callout (this=0x2b20c8096c00, event=60000, data=0x0) at HttpSM.cc:1462
#29 0x00000000005e6a22 in HttpSM::state_api_callback (this=0x2b20c8096c00, event=60000, data=0x0) at HttpSM.cc:1340
#30 0x0000000000536236 in TSHttpTxnReenable (txnp=0x2b20c8096c00, event=60000) at InkAPI.cc:5883
#31 0x00002aaaae401263 in http_hook (contp=0x2204f40, event=60002, edata=0x2b20c8096c00) at INKPluginInit.cc:408
#32 0x000000000052a4d3 in INKContInternal::handle_event (this=0x2204f40, event=60002, edata=0x2b20c8096c00) at InkAPI.cc:1048
#33 0x000000000051592e in Continuation::handleEvent (this=0x2204f40, event=60002, data=0x2b20c8096c00)
    at /home/shinrich/yats_build/trafficserver/iocore/eventsystem/I_Continuation.h:153
#34 0x000000000052acd3 in APIHook::invoke (this=0x2206fa0, event=60002, edata=0x2b20c8096c00) at InkAPI.cc:1267
#35 0x00000000005e7134 in HttpSM::state_api_callout (this=0x2b20c8096c00, event=0, data=0x0) at HttpSM.cc:1462
#36 0x00000000005f4d91 in HttpSM::do_api_callout_internal (this=0x2b20c8096c00) at HttpSM.cc:5168
#37 0x0000000000602fa3 in HttpSM::do_api_callout (this=0x2b20c8096c00) at HttpSM.cc:438
#38 0x00000000005fbdb0 in HttpSM::set_next_state (this=0x2b20c8096c00) at HttpSM.cc:7239
#39 0x00000000005fbd43 in HttpSM::call_transact_and_set_next_state (this=0x2b20c8096c00, f=
    0x60e8f0 <HttpTransact::ModifyRequest(HttpTransact::State*)>) at HttpSM.cc:7206
#40 0x00000000005e51f1 in HttpSM::state_read_client_request_header (this=0x2b20c8096c00, event=100, data=0x2aad2028f158)
    at HttpSM.cc:821
#41 0x00000000005eb68b in HttpSM::main_handler (this=0x2b20c8096c00, event=100, data=0x2aad2028f158) at HttpSM.cc:2662
#42 0x000000000051592e in Continuation::handleEvent (this=0x2b20c8096c00, event=100, data=0x2aad2028f158)
    at /home/shinrich/yats_build/trafficserver/iocore/eventsystem/I_Continuation.h:153
#43 0x000000000078749d in read_signal_and_update (event=100, vc=0x2aad2028f030) at UnixNetVConnection.cc:145
#44 0x000000000078a630 in UnixNetVConnection::readSignalAndUpdate (this=0x2aad2028f030, event=100) at UnixNetVConnection.cc:1125
#45 0x000000000076cc40 in SSLNetVConnection::net_read_io (this=0x2aad2028f030, nh=0x2b201796ce60, lthread=0x2b2017969010)
    at SSLNetVConnection.cc:586
#46 0x000000000077fdff in NetHandler::mainNetEvent (this=0x2b201796ce60, event=5, e=0x1c46760) at UnixNet.cc:509
#47 0x000000000051592e in Continuation::handleEvent (this=0x2b201796ce60, event=5, data=0x1c46760)
    at /home/shinrich/yats_build/trafficserver/iocore/eventsystem/I_Continuation.h:153
#48 0x00000000007a985d in EThread::process_event (this=0x2b2017969010, e=0x1c46760, calling_code=5) at UnixEThread.cc:143
#49 0x00000000007a9d5e in EThread::execute (this=0x2b2017969010) at UnixEThread.cc:270
#50 0x00000000007a8f13 in spawn_thread_internal (a=0x1b33a50) at Thread.cc:84
#51 0x00002b2015d20aa1 in start_thread () from /lib64/libpthread.so.0
#52 0x00002b2015a6d93d in clone () from /lib64/libc.so.6

This could be an issue unique to our plugins. This occurs after a successful remap (as shown in frame 5, t_state.url_remap_success is true). As the assert says t_state.transact_return_point is NULL. t_state.next_action is HttpTransact::SM_ACTION_SEND_ERROR_CACHE_NOOP. However if we have set the next action to be an error, it is odd that the url_remap was reported to be a success.

[shinrich]

Tracked it down some more. In this case, new remap rule is not found, and build_error_response is called to set the status of 404 with the reason of "Not Found". But somehow the t_state.transaction_return_point is not updated.

[bryancall]

Looks like this should be closed since the PR for it was merged. Please reopen if need be.