[21.11.5.33] Crash when using wait_end_of_query=1

[Algunenano]

SQL query with TCP client works fine:

    SELECT cluster, host_address, port FROM system.clusters FORMAT JSON

HTTP request works fine if it doesn't have wait_end_of_query=1:

curl 'http://localhost:8123/?database=default&query_id=52445679-1e58-415d-92e5-1aa714e691da&query=SELECT+cluster%2C+host_address%2C+port+FROM+system.clusters+FORMAT+JSON&max_result_bytes=104857600&log_queries=1&optimize_throw_if_noop=1&output_format_json_quote_64bit_integers=0&lock_acquire_timeout=10&max_execution_time=10'

HTTP request crashes if we add wait_end_of_query=1:

curl 'http://localhost:8123/?database=default&query_id=52445679-1e58-415d-92e5-1aa714e691da&query=SELECT+cluster%2C+host_address%2C+port+FROM+system.clusters+FORMAT+JSON&max_result_bytes=104857600&log_queries=1&optimize_throw_if_noop=1&output_format_json_quote_64bit_integers=0&lock_acquire_timeout=10&max_execution_time=10&wait_end_of_query=1'

Traceback:

2021.12.03 12:24:00.449871 [ 420960 ] {} <Trace> HTTPHandler-factory: HTTP Request for HTTPHandler-factory. Method: GET, Address: [::ffff:127.0.0.1]:45752, User-Agent: curl/7.80.0, Content Type: , Transfer Encoding: identity, X-Forwarded-For: (none)
2021.12.03 12:24:00.449964 [ 420960 ] {} <Trace> DynamicQueryHandler: Request URI: /?database=default&query_id=52445679-1e58-415d-92e5-1aa714e691da&query=SELECT+cluster%2C+host_address%2C+port+FROM+system.clusters+FORMAT+JSON&max_result_bytes=104857600&log_queries=1&optimize_throw_if_noop=1&output_format_json_quote_64bit_integers=0&lock_acquire_timeout=10&max_execution_time=10&wait_end_of_query=1
2021.12.03 12:24:00.449984 [ 420960 ] {} <Debug> HTTP-Session: 9216de00-6481-4c96-9216-de006481fc96 Authenticating user 'default' from [::ffff:127.0.0.1]:45752
2021.12.03 12:24:00.449996 [ 420960 ] {} <Debug> HTTP-Session: 9216de00-6481-4c96-9216-de006481fc96 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db
2021.12.03 12:24:00.450011 [ 420960 ] {} <Debug> HTTP-Session: 9216de00-6481-4c96-9216-de006481fc96 Creating query context from global context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: <NOT SET>
2021.12.03 12:24:00.450062 [ 420960 ] {} <Trace> ContextAccess (default): Settings: readonly=0, allow_ddl=true, allow_introspection_functions=false
2021.12.03 12:24:00.450072 [ 420960 ] {} <Trace> ContextAccess (default): List of all grants: GRANT ALL ON *.* WITH GRANT OPTION
2021.12.03 12:24:00.450078 [ 420960 ] {} <Trace> ContextAccess (default): List of all grants including implicit: GRANT ALL ON *.* WITH GRANT OPTION
2021.12.03 12:24:00.450198 [ 420960 ] {} <Trace> ContextAccess (default): Settings: readonly=2, allow_ddl=true, allow_introspection_functions=false
2021.12.03 12:24:00.450210 [ 420960 ] {} <Trace> ContextAccess (default): List of all grants: GRANT ALL ON *.* WITH GRANT OPTION
2021.12.03 12:24:00.450217 [ 420960 ] {} <Trace> ContextAccess (default): List of all grants including implicit: GRANT ALL ON *.* WITH GRANT OPTION
2021.12.03 12:24:00.450350 [ 420960 ] {52445679-1e58-415d-92e5-1aa714e691da} <Debug> executeQuery: (from [::ffff:127.0.0.1]:45752) SELECT cluster, host_address, port FROM system.clusters FORMAT JSON 
2021.12.03 12:24:00.450497 [ 420960 ] {52445679-1e58-415d-92e5-1aa714e691da} <Trace> ContextAccess (default): Access granted: SELECT(cluster, host_address, port) ON system.clusters
2021.12.03 12:24:00.450587 [ 420960 ] {52445679-1e58-415d-92e5-1aa714e691da} <Trace> InterpreterSelectQuery: FetchColumns -> Complete
2021.12.03 12:24:00.450765 [ 420960 ] {52445679-1e58-415d-92e5-1aa714e691da} <Information> executeQuery: Read 18 rows, 2.26 KiB in 0.000376347 sec., 47828 rows/sec., 5.86 MiB/sec.
2021.12.03 12:24:00.450911 [ 420913 ] {} <Trace> BaseDaemon: Received signal 11
2021.12.03 12:24:00.451002 [ 421175 ] {} <Fatal> BaseDaemon: ########################################
2021.12.03 12:24:00.451032 [ 421175 ] {} <Fatal> BaseDaemon: (version 21.11.5.33 (official build), build id: 76A10A4F605EF849249F2E8673661F7254B779DA) (from thread 420960) (query_id: 52445679-1e58-415d-92e5-1aa714e691da) Received signal Segmentation fault (11)
2021.12.03 12:24:00.451057 [ 421175 ] {} <Fatal> BaseDaemon: Address: 0x8 Access: write. Address not mapped to object.
2021.12.03 12:24:00.451069 [ 421175 ] {} <Fatal> BaseDaemon: Stack trace: 0x130e5bd3 0x9bdfe58 0x130d34ad 0x130d7ac0 0x1313c948 0x15d6e96f 0x15d70d61 0x15e85709 0x15e82e40 0x7fb9d208a259 0x7fb9d1f875e3
2021.12.03 12:24:00.451096 [ 421175 ] {} <Fatal> BaseDaemon: 2. DB::CascadeWriteBuffer::nextImpl() @ 0x130e5bd3 in /mnt/ch/official_binaries/clickhouse-common-static-21.11.5.33/usr/bin/clickhouse
2021.12.03 12:24:00.451114 [ 421175 ] {} <Fatal> BaseDaemon: 3. DB::WriteBuffer::finalize() @ 0x9bdfe58 in /mnt/ch/official_binaries/clickhouse-common-static-21.11.5.33/usr/bin/clickhouse
2021.12.03 12:24:00.451129 [ 421175 ] {} <Fatal> BaseDaemon: 4. DB::HTTPHandler::processQuery(DB::HTTPServerRequest&, DB::HTMLForm&, DB::HTTPServerResponse&, DB::HTTPHandler::Output&, std::__1::optional<DB::CurrentThread::QueryScope>&) @ 0x130d34ad in /mnt/ch/official_binaries/clickhouse-common-static-21.11.5.33/usr/bin/clickhouse
2021.12.03 12:24:00.451142 [ 421175 ] {} <Fatal> BaseDaemon: 5. DB::HTTPHandler::handleRequest(DB::HTTPServerRequest&, DB::HTTPServerResponse&) @ 0x130d7ac0 in /mnt/ch/official_binaries/clickhouse-common-static-21.11.5.33/usr/bin/clickhouse
2021.12.03 12:24:00.451153 [ 421175 ] {} <Fatal> BaseDaemon: 6. DB::HTTPServerConnection::run() @ 0x1313c948 in /mnt/ch/official_binaries/clickhouse-common-static-21.11.5.33/usr/bin/clickhouse
2021.12.03 12:24:00.451164 [ 421175 ] {} <Fatal> BaseDaemon: 7. Poco::Net::TCPServerConnection::start() @ 0x15d6e96f in /mnt/ch/official_binaries/clickhouse-common-static-21.11.5.33/usr/bin/clickhouse
2021.12.03 12:24:00.451172 [ 421175 ] {} <Fatal> BaseDaemon: 8. Poco::Net::TCPServerDispatcher::run() @ 0x15d70d61 in /mnt/ch/official_binaries/clickhouse-common-static-21.11.5.33/usr/bin/clickhouse
2021.12.03 12:24:00.451183 [ 421175 ] {} <Fatal> BaseDaemon: 9. Poco::PooledThread::run() @ 0x15e85709 in /mnt/ch/official_binaries/clickhouse-common-static-21.11.5.33/usr/bin/clickhouse
2021.12.03 12:24:00.451193 [ 421175 ] {} <Fatal> BaseDaemon: 10. Poco::ThreadImpl::runnableEntry(void*) @ 0x15e82e40 in /mnt/ch/official_binaries/clickhouse-common-static-21.11.5.33/usr/bin/clickhouse
2021.12.03 12:24:00.451203 [ 421175 ] {} <Fatal> BaseDaemon: 11. start_thread @ 0x9259 in /usr/lib/libpthread-2.33.so
2021.12.03 12:24:00.451213 [ 421175 ] {} <Fatal> BaseDaemon: 12. clone @ 0xfe5e3 in /usr/lib/libc-2.33.so

Tests per version (with clean environments, that is no config):

• 21.10.4.26: OK
• 21.11.4.14: OK
• 21.11.5.33: CRASH
• 21.12.1.9879 (master from today): Ok

Will look into bisecting the issue between 21.11.4.14 and 21.11.5.33

[Algunenano]

Bisected to 3612f5a. Reverting 3612f5a on top of v21.11.5.33-stable makes it stop crashing.

This commit was added in #31673 and backported to other branches but it's not causing issues there (just tested with the latest 21.10 branch build and it works ok).

[Algunenano]

Seems this was fixed in 21.12 / master with 3958857. Backporting that into 21.11 fixes the issue too