remote bootstrap stuck for a table with "Command is greater than read buffer" error #467
Comments
yugabyte-ci
pushed a commit
that referenced
this issue
Sep 10, 2018
Summary: The following issues were fixed: 1) nsessions was not passed to ctor of RemoteBootstrapSession, so rate limiter was disabled in service side. 2) When rate limiter is inactive in remote bootstrap service, it ignores max chunk size that was sent by client. 3) Since we allocate read buffer by chunks and don't move, to pass data of size N we should set buffer limit to N+chunk size. Test Plan: ybd --cxx-test ql-transaction-test --gtest_filter QLTransactionTest.RemoteBootstrap Reviewers: mikhail, kannan, hector Reviewed By: hector Subscribers: ybase Differential Revision: https://phabricator.dev.yugabyte.com/D5450
mbautin
pushed a commit
that referenced
this issue
Jul 11, 2019
…ed to the earlier commit 7e4108a Original commit message: #467: Fix several issues with remote bootstrap Summary: The following issues were fixed: 1) nsessions was not passed to ctor of RemoteBootstrapSession, so rate limiter was disabled in service side. 2) When rate limiter is inactive in remote bootstrap service, it ignores max chunk size that was sent by client. 3) Since we allocate read buffer by chunks and don't move, to pass data of size N we should set buffer limit to N+chunk size. Test Plan: ybd --cxx-test ql-transaction-test --gtest_filter QLTransactionTest.RemoteBootstrap Reviewers: mikhail, kannan, hector Reviewed By: hector Subscribers: ybase Differential Revision: https://phabricator.dev.yugabyte.com/D5450
mbautin
pushed a commit
to mbautin/yugabyte-db
that referenced
this issue
Jul 16, 2019
Summary: The following issues were fixed: 1) nsessions was not passed to ctor of RemoteBootstrapSession, so rate limiter was disabled in service side. 2) When rate limiter is inactive in remote bootstrap service, it ignores max chunk size that was sent by client. 3) Since we allocate read buffer by chunks and don't move, to pass data of size N we should set buffer limit to N+chunk size. Test Plan: ybd --cxx-test ql-transaction-test --gtest_filter QLTransactionTest.RemoteBootstrap Reviewers: mikhail, kannan, hector Reviewed By: hector Subscribers: ybase Differential Revision: https://phabricator.dev.yugabyte.com/D5450 Note: This commit provides additional functionality that is logically related to the earlier commit yugabyte@7e4108a and supersedes the commit yugabyte@9c8e1d6
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
With the recent change to allow a rate limit on remote bootstraps (#428), seeing this error on remote bootstrap on a cluster with large tablets, and one tablet was stuck in TABLET_DATA_COPYING mode.
{code}
W0909 06:36:08.415642 105404 tcp_stream.cc:91] { local: 172.151.26.103:36173 remote: 172.151.17.230:9100 }: Shutting down with pending inbound data (size: 267133381, limit: 267386880, status = Invalid argument (yb/rpc/connection.cc:242): Command is greater than read buffer)
{code}
Increasing the
--rpc_max_message_sizeflag on the TServer receiving the tablet helped as a temporary workaround.The text was updated successfully, but these errors were encountered: