Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

golang grpc client 请求 brpc服务,在response非常大的时候报错 #1087

Closed
hpsdy opened this issue Apr 15, 2020 · 10 comments
Closed

Comments

@hpsdy
Copy link

hpsdy commented Apr 15, 2020

Describe the bug (描述bug)
rpc error: code = ResourceExhausted desc = stream terminated by RST_STREAM with error code: FLOW_CONTROL_ERROR

To Reproduce (复现方法)
请求获取的数据量很大的时候复现。请求的数据比较小的时候是正常的。

Expected behavior (期望行为)
正常处理

Versions (各种版本)
OS:
Compiler:
brpc:
protobuf:

Additional context/screenshots (更多上下文/截图)

@zyearn
Copy link
Member

zyearn commented Apr 16, 2020

grpc client默认的window size是65K,如果请求的response超过这个值,需要设置一下grpc client的stream_size

@hpsdy
Copy link
Author

hpsdy commented Apr 16, 2020

嗯,发现单独设置stream_size无效,加上connect size,maxmessagesize 已经OK了。

@hpsdy hpsdy closed this as completed Apr 16, 2020
@EAHITechnology
Copy link

加上connect size,maxmessagesize 已

麻烦问一下是具体哪个参数,我也遇到了

@searKing
Copy link

加上connect size,maxmessagesize 已

麻烦问一下是具体哪个参数,我也遇到了

grpc.DialContext的opts增加如下选项 grpc/grpc-go#1210 即可,大小可以根据需要调整:

grpc.WithDefaultCallOptions(grpc.MaxCallRecvMsgSize(math.MaxInt32), 
    grpc.MaxCallSendMsgSize(math.MaxInt32)),
    grpc.WithInitialConnWindowSize(math.MaxInt32),
    grpc.WithInitialWindowSize(math.MaxInt32)

@azhe198827
Copy link

c++ 或者python 用grpc 的时候也会报同样的错,但是没有window size 可以设置

@pengybhnu
Copy link

c++ 或者python 用grpc 的时候也会报同样的错,但是没有window size 可以设置

c++ 找到设置方法了么?参考窗口大小可以通过 grpc.http2.lookahead_bytes 设置,还未找到手动设置ConnWindowSize。response过大时 grpc 客户端访问brpc 还是报FLOW_CONTROL_ERROR。

@azhe198827
Copy link

c++ 或者python 用grpc 的时候也会报同样的错,但是没有window size 可以设置

c++ 找到设置方法了么?参考窗口大小可以通过 grpc.http2.lookahead_bytes 设置,还未找到手动设置ConnWindowSize。response过大时 grpc 客户端访问brpc 还是报FLOW_CONTROL_ERROR。

问题困扰两年多了,一直没有找到如何解决。

Cyber-SiKu added a commit to Cyber-SiKu/curve that referenced this issue Jun 9, 2023
when run `curve fs status copyset`, while get error:
```bahs
rpc error: code = ResourceExhausted desc = stream terminated by
RST_STREAM with error code: FLOW_CONTROL_ERROR
```

Refer to [issue](apache/brpc#1087).

Signed-off-by: Cyber-SiKu <Cyber-SiKu@outlook.com>
Cyber-SiKu added a commit to opencurve/curve that referenced this issue Jun 12, 2023
when run `curve fs status copyset`, while get error:
```bahs
rpc error: code = ResourceExhausted desc = stream terminated by
RST_STREAM with error code: FLOW_CONTROL_ERROR
```

Refer to [issue](apache/brpc#1087).

Signed-off-by: Cyber-SiKu <Cyber-SiKu@outlook.com>
@tailai01
Copy link

遇到了同样的问题,还没解决。有解决的大佬请给回复下,谢谢。

@ehds
Copy link
Contributor

ehds commented Nov 7, 2023

// flow control
// NOTE: Currently the stream context is definitely removed and updating
// window size is useless, however it's not true when progressive request
// is supported.
// TODO(zhujiashun): Instead of just returning error to client, a better
// solution to handle not enough window size is to wait until WINDOW_UPDATE
// is received, and then retry those failed response again.
if (!MinusWindowSize(&ctx->_remote_window_left, _data.size())) {
char rstbuf[FRAME_HEAD_SIZE + 4];
SerializeFrameHead(rstbuf, 4, H2_FRAME_RST_STREAM, 0, _stream_id);
SaveUint32(rstbuf + FRAME_HEAD_SIZE, H2_FLOW_CONTROL_ERROR);
out->append(rstbuf, sizeof(rstbuf));
return butil::Status::OK();
}

看样子可能是因为这里引起的,如果当前传输数据大于 remote_window_left 就直接 ResetStream 了,如 TODO 描述应该可以等到 remote_window_left 有空余的时候再继续发送。

@zyearn @wwbmmm 有计划支持吗?

@zyearn
Copy link
Member

zyearn commented Nov 8, 2023

目前的workaround是设置grpc client的stream_size和conn_size。如果有兴趣的话可以给我们提一个PR @ehds

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants