WriteBufferFromOStream(): Code: 24, e.displayText() = DB::Exception: Cannot write to ostream at offset 1071166

[xiashuijun]

2019.01.03 15:31:07.561639 [ 22565132 ] {} <Error> virtual DB::WriteBufferFromOStream::~WriteBufferFromOStream(): Code: 24, e.displayText() = DB::Exception: Cannot write to ostream at offset 1071166, e.what() = DB::Exception, Stack trace:

0. clickhouse-server(StackTrace::StackTrace()+0x16) [0x564b1f6]
1. clickhouse-server(DB::Exception::Exception(std::string const&, int)+0x1f) [0x297af4f]
2. clickhouse-server(DB::WriteBufferFromOStream::nextImpl()+0xba) [0x567493a]
3. clickhouse-server(DB::WriteBufferFromOStream::~WriteBufferFromOStream()+0x2e) [0x56749fe]
4. clickhouse-server(DB::WriteBufferFromHTTPServerResponse::~WriteBufferFromHTTPServerResponse()+0x71) [0x56744b1]
5. clickhouse-server(std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0xa2) [0x297bd22]
6. clickhouse-server(DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&)+0x50a) [0x299587a]
7. clickhouse-server(Poco::Net::HTTPServerConnection::run()+0x23e) [0x57437ce]
8. clickhouse-server(Poco::Net::TCPServerConnection::start()+0xf) [0x573f76f]
9. clickhouse-server(Poco::Net::TCPServerDispatcher::run()+0x110) [0x573fdd0]
10. clickhouse-server(Poco::PooledThread::run()+0x77) [0x57f5717]
11. clickhouse-server(Poco::ThreadImpl::runnableEntry(void*)+0x38) [0x57f2a78]
12. clickhouse-server() [0x5c8becf]
13. /lib64/libpthread.so.0() [0x34096079d1]
14. /lib64/libc.so.6(clone+0x6d) [0x3408ee88fd]

this exception happen Occasionally when graphite data write to clickhouse.

clickhosue version is 18.14.13,engine is ReplicatedGraphiteMergeTree

thank you very much !!!!!!!

[doom369]

Same issue here:

2019.08.16 09:27:41.097746 [ 45 ] {} <Trace> HTTPHandler-factory: HTTP Request for HTTPHandler-factory. Method: POST, Address: [::ffff:xxx.xxx.xxx.xxx]:50230, User-Agent: Apache-HttpClient/4.5.2 (Java/11.0.4), Content Type: , Transfer Encoding: chunked
2019.08.16 09:27:41.097865 [ 45 ] {} <Trace> HTTPHandler: Request URI: /?compress=1&database=qa_reporting
2019.08.16 09:27:41.098032 [ 45 ] {8c319ce6-2e20-4772-8965-2a9436d9393b} <Debug> executeQuery: (from [::ffff:xxx.xxx.xxx.xxx]:50230) INSERT INTO reporting_app_stat_minute (region, ts, active, active_week, active_month, minute_rate, minute_incoming_rate, minute_outgoing_rate, connected, online_apps, online_hards, total_online_apps, total_online_hards, registrations)  FORMAT TabSeparated 
2019.08.16 09:27:41.098781 [ 45 ] {8c319ce6-2e20-4772-8965-2a9436d9393b} <Debug> executeQuery: Query pipeline:
NullAndDoCopy
 InputStreamFromASTInsertQuery

2019.08.16 09:27:41.169481 [ 52 ] {48478327-c758-469d-a7d1-51149cd3df17} <Error> virtual DB::WriteBufferFromHTTPServerResponse::~WriteBufferFromHTTPServerResponse(): Code: 24, e.displayText() = DB::Exception: Cannot write to ostream at offset 72, Stack trace:

0. /usr/bin/clickhouse-server(StackTrace::StackTrace()+0x30) [0x7f58f30]
1. /usr/bin/clickhouse-server(DB::Exception::Exception(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int)+0x25) [0x3c427b5]
2. /usr/bin/clickhouse-server() [0x3ab7759]
3. /usr/bin/clickhouse-server(DB::WriteBufferFromHTTPServerResponse::nextImpl()+0xa1) [0x78e0231]
4. /usr/bin/clickhouse-server(DB::WriteBufferFromHTTPServerResponse::~WriteBufferFromHTTPServerResponse()+0x4d) [0x78e057d]
5. /usr/bin/clickhouse-server(std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()+0x48) [0x3c428c8]
6. /usr/bin/clickhouse-server(DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&)+0x88f) [0x3c923ef]
7. /usr/bin/clickhouse-server(Poco::Net::HTTPServerConnection::run()+0x2a9) [0x7932009]
8. /usr/bin/clickhouse-server(Poco::Net::TCPServerConnection::start()+0x10) [0x792cde0]
9. /usr/bin/clickhouse-server(Poco::Net::TCPServerDispatcher::run()+0xed) [0x792d4fd]
10. /usr/bin/clickhouse-server(Poco::PooledThread::run()+0x81) [0x8057e21]
11. /usr/bin/clickhouse-server(Poco::ThreadImpl::runnableEntry(void*)+0x3c) [0x8055bcc]
12. /usr/bin/clickhouse-server() [0xba2baa0]
13. /lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f33f58bb6db]
14. /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f33f504288f]
 (version 19.13.1.11 (official build))

[filimonov]

That is caused by client side disconnect on http protocol, for example due to read timeout on client side. So server can't write anymore to the socket after client close that.

Test case:

wget 'http://localhost:8123/?query=SELECT%201,sleepEachRow(0.01)%20FROM%20numbers(200)' -qO- --read-timeout=1

That situation should be handled / reported a bit more intelligent way.

[xiashuijun]

That is caused by client side disconnect on http protocol, for example due to read timeout on client side. So server can't write anymore to the socket after client close that.

Test case:

wget 'http://localhost:8123/?query=SELECT%201,sleepEachRow(0.01)%20FROM%20numbers(200)' -qO- --read-timeout=1

That situation should be handled / reported a bit more intelligent way.

ok thank you for you suggestion!!

[T-M-L-C]

Is it solved now? Are there any corresponding problems or solutions?

[doom369]

I haven't seen it in the new versions 21+

[T-M-L-C]

I haven't seen it in the new versions 21+

ClickHouse/clickhouse-java#1550 @doom369

[doom369]

@T-M-L-C most likely driver bug, we use another one.