Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

主要修复 SocketChannel 默认缓存大小(1MB)不够用时需自动扩充,否则将因缓存空间不足而造成I/O超时假象 #536

Merged
merged 5 commits into from
Mar 12, 2018

Conversation

Wu-Jianqiang
Copy link
Contributor

通常复制包含 CLOB/BLOB 字段类型数据时会遭遇这个问题。1.0.24版开始表现为没有任何反应,也没有任何错误日志。升级到1.0.25版之后即报"socket read timeout occured !"错误,给人误以为网络或者数据库问题(复制是在经过公网SSH转发的两个私网之间)。后经调试跟踪发现实质是固定1MB缓存大小问题。实际运行测试发现一次性readSize达到15MB及以上。截止目前的 1.0.26-SNAPSHOT 编译发布,已稳定运行。

@CLAassistant
Copy link

CLAassistant commented Feb 26, 2018

CLA assistant check
All committers have signed the CLA.

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Collaborator

@lcybo lcybo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

用change master to语法单位才是秒。canal不是真正的mysql,无法使用change master to语句,所以用set @@MASTER_HEARTBEAT_PERIOD单位是纳秒。

Copy link
Collaborator

@lcybo lcybo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

另外,mysql的maxThreeBytes最大就是将近16MB,没有必要反复改,缓存开始直接设为16M就可以啦。

@Wu-Jianqiang
Copy link
Contributor Author

Wu-Jianqiang commented Feb 27, 2018

lcybo,谢谢你的指正。

根据你提供的线索,我搜索了下 MySQL jdbc 中 com.mysql.jdbc.MysqlIO 源码,部分摘录如下:

if (versionMeetsMinimum(4, 0, 8)) { this.maxThreeBytes = (256 * 256 * 256) - 1; this.useNewLargePackets = true; } else { this.maxThreeBytes = 255 * 255 * 255; this.useNewLargePackets = false; }
看起来确实可以简单的设置为: 16MB = 256 * 256 * 256

不过我还是希望代码中做必要的检查和详细日志,不然太烧脑了。

@lcybo
Copy link
Collaborator

lcybo commented Feb 27, 2018

嗯,其实个人觉得dump IO这里没有用netty NIO的必要,割裂了读写线程,引入了不必要的复杂度。
用BIO通过设置较小的SO_TIMEOUT进行捕捉可以规避闪断问题和处理线程中断场景,也更加简洁。

@agapple
Copy link
Member

agapple commented Mar 12, 2018

tks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants