Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

分片下载大文件200多MB,出现 “i/o timeout” #10

Closed
xunchangguo opened this issue Aug 19, 2016 · 4 comments
Closed

分片下载大文件200多MB,出现 “i/o timeout” #10

xunchangguo opened this issue Aug 19, 2016 · 4 comments

Comments

@xunchangguo
Copy link

参数设置connectTimeoutSec=30,readWriteTimeout=120,分片大小:5M,Checkpoint=false
错误信息如下:
read tcp 198.9.4.207:53567->101.201.182.67:80: i/o timeout

另:通过“OSS控制台客户端(Windows)”下载正常

@xunchangguo
Copy link
Author

补充下:
通过“OSS控制台客户端(Windows)”下载完成后的文件和用SDK下载出现 i/o timeout后的已经下载的文件对比发现,大小是一样的,用2进制比较工具比较,内容也是一样的。
怀疑:实际文件大小和通过接口获取的不一致,通过接口获取的文件大小大

@xunchangguo
Copy link
Author

有时候,出现“unexpected EOF”,只要出现了错误,协程不退出,导致文件被占用,下载最新版本sdk,说是解决了协程不退出问题,试了,问题依旧啊,没有解决啊!

@baiyubin
Copy link
Contributor

read ... i/o timeout 是网络超时,分片下载出错后,下载的文件是无效,这点需要优化。请问什么情况下出现 unexpected EOF ?

@xunchangguo
Copy link
Author

这么久了,总算有个回复的了。不过我也是只能“呵呵”了,最终还是都得靠自己啊。自己修改代码,解决了。
1、协程不退出,使用sync.WaitGroup解决,错误代码:
for completed < len(parts) {
.............................
select {
case <-results:
completed++
ps[part.Index] = part
case err = <-failed:
close(die)
return err
}
2、unexpected EOF ,出错代码
_, err = fd.Seek(part.Start, os.SEEK_SET)
if err != nil {
failed <- err
break
}
_, err = io.Copy(fd, rd)
if err != nil {
failed <- err
break
}
解决方式,可参照AWS SDK的实现方式和思路

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants