上传出错中断 #211

jacwl · 2016-01-02T02:54:14Z

文件比较多(5000+), 主要是音频和文本, 考虑很多文件可以秒传，使用的rapidupload only模式, 所以没有事先打包

传的还算顺利, 传了3h的时候突然出错终止了，

RapidUpload: '/home/jack/11 - Yamaha.mp3' =R=> '/apps/bypy/jack/11 - Yamaha.mp3' OK.
Saving Hash Cache...
 [18:58:33] Failed to save Hash Cache. Exception:
  File "bypy.py", line 3773, in 
    main()
  File "bypy.py", line 3770, in main
    onexit(result)
  File "bypy.py", line 3546, in onexit
    cached.savecache()
  File "bypy.py", line 904, in savecache
    perr("Failed to save Hash Cache. Exception:\n{}".format(format_exc_str()))
  File "bypy.py", line 432, in format_exc_str
    return ''.join(format_exc(**kwargs))
  File "bypy.py", line 429, in format_exc
    return traceback.format_stack(**kwargs)

重试了一次结果传到同一个位置时再次出错了

 [02:25:38] Remote file '/apps/bypy/jack/11 - Yamaha.mp3' exists and is the same, skip uploading
Saving Hash Cache...
 [02:25:38] Failed to save Hash Cache. Exception:
  File "bypy.py", line 3773, in 
    main()
  File "bypy.py", line 3770, in main
    onexit(result)
  File "bypy.py", line 3546, in onexit
    cached.savecache()
  File "bypy.py", line 904, in savecache
    perr("Failed to save Hash Cache. Exception:\n{}".format(format_exc_str()))
  File "bypy.py", line 432, in format_exc_str
    return ''.join(format_exc(**kwargs))
  File "bypy.py", line 429, in format_exc
    return traceback.format_stack(**kwargs)

The text was updated successfully, but these errors were encountered:

houtianze · 2016-01-02T13:40:59Z

感谢反馈。

之前的code输出stack有点问题，看起来像是保存json的问题，但是不知道具体异常。麻烦用最新版git clone一下再试试，如果还有错误，再贴一下输出。我怀疑是json encoding的问题。

jacwl · 2016-01-02T16:26:43Z

这次比前2次出错更晚，但还是导致上传中断了

Remote file '/apps/bypy/jack/02 - Scene Two.flac' already exists, skip uploading
  [00:10:10] Remote file '/apps/bypy/jack/02 - Scene Two.flac' exists and is the same, skip uploading
 __get_file_info(): rdir : /apps/bypy/jack | rfile: 03 - Scene Three.flac
 GET https://pcs.baidu.com/rest/2.0/pcs/file
 actargs: /apps/bypy/jack/03 - Scene Three.flac
 Params: {u'path': u'/apps/jack', u'order': u'asc', u'method': u'list', u'by': u'name'}
 HTTP Status Code: 200
 200 OK, processing action
 List json: {u'error_code': 0, u'error_msg': u'no error', u'request_id': 46550032376647078}
  [00:10:12] Error accessing 'https://pcs.baidu.com/rest/2.0/pcs/file'
  [00:10:12] Exception:
u'list'
Stack:
['  File "bypy.py", line 3921, in \n    main()\n', '  File "bypy.py", line 3898, in main\n    result = getattr(by, args.command[0])(*uargs)\n', '  File "bypy.py", line 2385, in upload\n    return self.__upload_dir(lpath, rpath, ondup)\n', '  File "bypy.py", line 2298, in __upload_dir\n    thisresult = self.__walk_upload(localpath, remotepath, ondup, walk)\n', '  File "bypy.py", line 2269, in __walk_upload\n    subresult = self.__get_file_info(rfile, dumpex = False)\n', '  File "bypy.py", line 2002, in __get_file_info\n    return self.__get(pcsurl + \'file\', pars, self.__get_file_info_act, remotefile, **kwargs)\n', '  File "bypy.py", line 1651, in __get\n    return self.__request(url, pars, act, \'GET\', actargs, retry, addtoken, dumpex, **kwargs)\n', '  File "bypy.py", line 1630, in __request\n    result = self.__request_work(url, pars, act, method, actargs, addtoken, dumpex, **kwnew)\n', '  File "bypy.py", line 1608, in __request_work\n    self.__request_work_die(ex, url, pars, r, act)\n', '  File "bypy.py", line 1456, in __request_work_die\n    self.__dump_exception(ex, url, pars, r, act)\n', '  File "bypy.py", line 1428, in __dump_exception\n    perr("Exception:\\n{}\\nStack:\\n{}".format(ex, traceback.format_stack()))\n']
 [00:10:12] Function: __get_file_info_act
 [00:10:12] Website parameters: {u'path': u'/apps/bypy/jack', u'order': u'asc', u'method': u'list', u'by': u'name'}
 [00:10:12] HTTP Response Status Code: 200
 [00:10:12] Error JSON returned:{u'error_code': 0, u'error_msg': u'no error', u'request_id': 46550032376647078}
Error code: 0
Error Description: no error
 [00:10:12] Website returned: {"error_code":0,"error_msg":"no error","request_id":46550032376647078}
 [00:10:12] Fatal Exception, no way to continue.
Quitting...
 [00:10:12] If the error is reproducible, run the program with `-dv` arguments again to get more info.
Saving Hash Cache...
 [00:10:12] Failed to save Hash Cache. Exception:
must be unicode, not str
Stack:
['  File "bypy.py", line 3921, in \n    main()\n', '  File "bypy.py", line 3898, in main\n    result = getattr(by, args.command[0])(*uargs)\n', '  File "bypy.py", line 2385, in upload\n    return self.__upload_dir(lpath, rpath, ondup)\n', '  File "bypy.py", line 2298, in __upload_dir\n    thisresult = self.__walk_upload(localpath, remotepath, ondup, walk)\n', '  File "bypy.py", line 2269, in __walk_upload\n    subresult = self.__get_file_info(rfile, dumpex = False)\n', '  File "bypy.py", line 2002, in __get_file_info\n    return self.__get(pcsurl + \'file\', pars, self.__get_file_info_act, remotefile, **kwargs)\n', '  File "bypy.py", line 1651, in __get\n    return self.__request(url, pars, act, \'GET\', actargs, retry, addtoken, dumpex, **kwargs)\n', '  File "bypy.py", line 1630, in __request\n    result = self.__request_work(url, pars, act, method, actargs, addtoken, dumpex, **kwnew)\n', '  File "bypy.py", line 1608, in __request_work\n    self.__request_work_die(ex, url, pars, r, act)\n', '  File "bypy.py", line 1459, in __request_work_die\n    onexit(result)\n', '  File "bypy.py", line 3694, in onexit\n    cached.savecache()\n', '  File "bypy.py", line 893, in savecache\n    perr("Failed to save Hash Cache. Exception:\\n{}\\nStack:\\n{}".format(ex, traceback.format_stack()))\n']

houtianze · 2016-01-03T08:54:03Z

感谢再次试验。

试试最新版，应该可以了。中了Python 另外一个Unicode的坑（JSON保存）。。。

jacwl · 2016-01-03T12:27:58Z

依然使用的rapidupload only模式，上传过程中这样的错误好像出现了若干次，我以为影响不大的，结果在传了3个多小时之后还是导致上传中断了

Remote file '/apps/bypy/jack/11 - Yamaha.mp3' already exists, skip uploading
  [20:19:27] Remote file '/apps/bypy/jack/11 - Yamaha.mp3' exists and is the same, skip uploading
Saving Hash Cache...
  [20:19:27] Failed to save Hash Cache.
Exception:
 - 'ascii' codec can't decode byte 0xad in position 9: ordinal not in range(128)
Stack:  File "bypy.py", line 3938, in 
    main()
  File "bypy.py", line 3935, in main
    onexit(result)
  File "bypy.py", line 3713, in onexit
    cached.savecache()
  File "bypy.py", line 918, in savecache
    perr("Failed to save Hash Cache.\n{}".format(formatex(ex)))
  File "bypy.py", line 433, in formatex
    type(ex), ex, ''.join(traceback.format_stack()))

houtianze · 2016-01-03T15:40:08Z

这次的错误不一样了，是之前改哈希缓存格式引入的bug，这次没改错的话，应该可以了。

…yntax Error

houtianze · 2016-01-04T03:52:30Z

这是一个以前 xslidian 修正过的bug #162 ，可惜我之前重构的时候给弄丢了。。
现在clone / pip 最新版应该可以了。

jacwl · 2016-01-04T08:33:13Z

感谢作者的更新，rapidupload可以正常传完了，但分片上传还是有问题
这里我也感到很疑惑，因为我本地的win10+python2.7+requests2.9使用分片上传的问题上次已经解决，今天用最新版bypy传了几个都正常，但我2个vps都不正常，一个是ubuntu一个是archlinux，和win10一样用的python2.7+requests2.9

出错有以下几种，都是导致上传终止的 (我反复重试了若干次)

Exception:(110, 'ETIMEDOUT')

 [06:27:19] Error accessing 'https://c.pcs.baidu.com/rest/2.0/pcs/file'
 [06:27:19] Exception:
 - (110, 'ETIMEDOUT')
Stack:  File "bypy.py", line 4015, in 
    main()
  File "bypy.py", line 3994, in main
    result = getattr(by, args.command[0])(*uargs)
  File "bypy.py", line 2481, in upload
    return self.__upload_dir(lpath, rpath, ondup)
  File "bypy.py", line 2394, in __upload_dir
    thisresult = self.__walk_upload(localpath, remotepath, ondup, walk)
  File "bypy.py", line 2378, in __walk_upload
    fileresult = self.__upload_file(lfile, rfile, ondup)
  File "bypy.py", line 2428, in __upload_file
    result = self.__upload_file_slices(localpath, remotepath, ondup)
  File "bypy.py", line 2254, in __upload_file_slices
    ec = self.__upload_slice(remotepath)
  File "bypy.py", line 2223, in __upload_slice
    files = { 'file' : ('file', self.__current_slice) } )
  File "bypy.py", line 1751, in __post
    return self.__request(url, pars, act, 'POST', actargs, retry, addtoken, dumpex, **kwargs)
  File "bypy.py", line 1727, in __request
    result = self.__request_work(url, pars, act, method, actargs, addtoken, dumpex, **kwnew)
  File "bypy.py", line 1705, in __request_work
    self.__request_work_die(ex, url, pars, r, act)
  File "bypy.py", line 1540, in __request_work_die
    self.__dump_exception(ex, url, pars, r, act)
  File "bypy.py", line 1512, in __dump_exception
    perr(formatex(ex))
  File "bypy.py", line 433, in formatex
    type(ex), ex, ''.join(traceback.format_stack()))
 [06:27:19] Function: __upload_slice_act
 [06:27:19] Website parameters: {u'type': u'tmpfile', u'method': u'upload'}
 [06:27:19] Fatal Exception, no way to continue.
Quitting...
 [06:27:19] If the error is reproducible, run the program with `-dv` arguments again to get more info

.

 [02:23:16] Error accessing 'https://c.pcs.baidu.com/rest/2.0/pcs/file'
 [02:23:16] Exception:
(110, 'ETIMEDOUT')
Traceback (most recent call last):
  File "bypy.py", line 1352, in __request_work
    r = self.__requester.request(method, url, params = parsnew, timeout = self.__timeout, verify = self.__checkssl, **kwargs)
  File "bypy.py", line 1063, in request
    return requests.request(method, url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 53, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 376, in send
    timeout=timeout
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
    body=body, headers=headers)
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/connectionpool.py", line 353, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib/python2.7/httplib.py", line 1057, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1097, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 1053, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 897, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 873, in send
    self.sock.sendall(data)
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 218, in sendall
    sent = self._send_until_done(data[total_sent:total_sent + SSL_WRITE_BLOCKSIZE])
  File "/usr/lib/python2.7/site-packages/requests/packages/urllib3/contrib/pyopenssl.py", line 207, in _send_until_done
    return self.connection.send(data)
  File "/usr/lib/python2.7/site-packages/OpenSSL/SSL.py", line 1271, in send
    self._raise_ssl_error(self._ssl, result)
  File "/usr/lib/python2.7/site-packages/OpenSSL/SSL.py", line 1178, in _raise_ssl_error
    raise SysCallError(errno, errorcode.get(errno))
SysCallError: (110, 'ETIMEDOUT')
 [02:23:16] Function: __upload_slice_act
 [02:23:16] Website parameters: {u'type': u'tmpfile', u'method': u'upload'}
 [02:23:16] Fatal Exception, no way to continue.
Quitting...
 [02:23:16] If the error is reproducible, run the program with `-dv` arguments again to get more info.

houtianze · 2016-01-04T10:43:21Z

感谢再次反馈新问题。

加入了 (110, 'ETIMEDOUT') 处理。Linux下不好用应该是因为对应的Windows异常已经处理，而我不知道Linux下的对应异常是什么。

jacwl · 2016-01-05T00:28:46Z

linux的分片上传似乎还是有问题，我之前反馈的时候都习惯性把不导致上传终止的错误忽略了，但这个错误每次分片上传都会出现,，从分片上传一开始就出错

 Uploading MD5 slice: 0df65f8a6e7fc3ea6977db7750634dbf, #1 / 63
 POST https://c.pcs.baidu.com/rest/2.0/pcs/file
 actargs: /apps/bypy/jack/Blur.mkv
 Params: {u'type': u'tmpfile', u'method': u'upload'}
 [00:51:18] Error accessing 'https://c.pcs.baidu.com/rest/2.0/pcs/file'
 [00:51:18] Exception:
 - (104, 'ECONNRESET')
Stack:  File "bypy.py", line 4016, in 
    main()
  File "bypy.py", line 3995, in main
    result = getattr(by, args.command[0])(*uargs)
  File "bypy.py", line 2482, in upload
    return self.__upload_dir(lpath, rpath, ondup)
  File "bypy.py", line 2395, in __upload_dir
    thisresult = self.__walk_upload(localpath, remotepath, ondup, walk)
  File "bypy.py", line 2379, in __walk_upload
    fileresult = self.__upload_file(lfile, rfile, ondup)
  File "bypy.py", line 2429, in __upload_file
    result = self.__upload_file_slices(localpath, remotepath, ondup)
  File "bypy.py", line 2255, in __upload_file_slices
    ec = self.__upload_slice(remotepath)
  File "bypy.py", line 2224, in __upload_slice
    files = { 'file' : ('file', self.__current_slice) } )
  File "bypy.py", line 1752, in __post
    return self.__request(url, pars, act, 'POST', actargs, retry, addtoken, dumpex, **kwargs)
  File "bypy.py", line 1728, in __request
    result = self.__request_work(url, pars, act, method, actargs, addtoken, dumpex, **kwnew)
  File "bypy.py", line 1703, in __request_work
    self.__dump_exception(ex, url, pars, r, act)
  File "bypy.py", line 1512, in __dump_exception
    perr(formatex(ex))
  File "bypy.py", line 433, in formatex
    type(ex), ex, ''.join(traceback.format_stack()))
 [00:51:18] Function: __upload_slice_act
 [00:51:18] Website parameters: {u'type': u'tmpfile', u'method': u'upload'}
 [00:51:18] Waiting 10 seconds before retrying...
 [00:51:28] Request Try #2 / 100000000

之前是输出这个错误，然后就很长一段时间没有任何输出，直到出现110 etimedout，这次修正了110 etimedout之后就会反复重试直到上传开始，出错前兆是进度突然卡住，几分钟到几十分钟后出现104 econnreset错误，再次反复重试直到上传开始，我丧心病狂的把重试次数设为1亿次，目前传了2个多小时，虽然速度非常慢但不出意外的话至少有几率能传完了

所以我觉得问题可能出在(104, 'ECONNRESET')这里，之前的反馈及描述不准确，非常抱歉

update：
在断断续续传了5小时时候还是出错终止了，截止出错一共传了455MB的数据

 [03:55:05] Error accessing 'https://c.pcs.baidu.com/rest/2.0/pcs/file'
 [03:55:05] Exception:
 - (110, 'ETIMEDOUT')
Stack:  File "bypy.py", line 4016, in 
    main()
  File "bypy.py", line 3995, in main
    result = getattr(by, args.command[0])(*uargs)
  File "bypy.py", line 2482, in upload
    return self.__upload_dir(lpath, rpath, ondup)
  File "bypy.py", line 2395, in __upload_dir
    thisresult = self.__walk_upload(localpath, remotepath, ondup, walk)
  File "bypy.py", line 2379, in __walk_upload
    fileresult = self.__upload_file(lfile, rfile, ondup)
  File "bypy.py", line 2429, in __upload_file
    result = self.__upload_file_slices(localpath, remotepath, ondup)
  File "bypy.py", line 2255, in __upload_file_slices
    ec = self.__upload_slice(remotepath)
  File "bypy.py", line 2224, in __upload_slice
    files = { 'file' : ('file', self.__current_slice) } )
  File "bypy.py", line 1752, in __post
    return self.__request(url, pars, act, 'POST', actargs, retry, addtoken, dumpex, **kwargs)
  File "bypy.py", line 1728, in __request
    result = self.__request_work(url, pars, act, method, actargs, addtoken, dumpex, **kwnew)
  File "bypy.py", line 1703, in __request_work
    self.__dump_exception(ex, url, pars, r, act)
  File "bypy.py", line 1512, in __dump_exception
    perr(formatex(ex))
  File "bypy.py", line 433, in formatex
    type(ex), ex, ''.join(traceback.format_stack()))
 [03:55:05] Function: __upload_slice_act
 [03:55:05] Website parameters: {u'type': u'tmpfile', u'method': u'upload'}
 [03:55:05] Waiting 10 seconds before retrying...
 [03:55:15] Request Try #2 / 100000000
 POST https://c.pcs.baidu.com/rest/2.0/pcs/file
 actargs: /apps/bypy/jack/Blur.mkv
 Params: {u'type': u'tmpfile', u'method': u'upload'}
 [04:12:35] Error accessing 'https://c.pcs.baidu.com/rest/2.0/pcs/file'
 [04:12:35] Exception:
 - (32, 'EPIPE')
Stack:  File "bypy.py", line 4016, in 
    main()
  File "bypy.py", line 3995, in main
    result = getattr(by, args.command[0])(*uargs)
  File "bypy.py", line 2482, in upload
    return self.__upload_dir(lpath, rpath, ondup)
  File "bypy.py", line 2395, in __upload_dir
    thisresult = self.__walk_upload(localpath, remotepath, ondup, walk)
  File "bypy.py", line 2379, in __walk_upload
    fileresult = self.__upload_file(lfile, rfile, ondup)
  File "bypy.py", line 2429, in __upload_file
    result = self.__upload_file_slices(localpath, remotepath, ondup)
  File "bypy.py", line 2255, in __upload_file_slices
    ec = self.__upload_slice(remotepath)
  File "bypy.py", line 2224, in __upload_slice
    files = { 'file' : ('file', self.__current_slice) } )
  File "bypy.py", line 1752, in __post
    return self.__request(url, pars, act, 'POST', actargs, retry, addtoken, dumpex, **kwargs)
  File "bypy.py", line 1728, in __request
    result = self.__request_work(url, pars, act, method, actargs, addtoken, dumpex, **kwnew)
  File "bypy.py", line 1706, in __request_work
    self.__request_work_die(ex, url, pars, r, act)
  File "bypy.py", line 1540, in __request_work_die
    self.__dump_exception(ex, url, pars, r, act)
  File "bypy.py", line 1512, in __dump_exception
    perr(formatex(ex))
  File "bypy.py", line 433, in formatex
    type(ex), ex, ''.join(traceback.format_stack()))
 [04:12:35] Function: __upload_slice_act
 [04:12:35] Website parameters: {u'type': u'tmpfile', u'method': u'upload'}
 [04:12:35] Fatal Exception, no way to continue.
Quitting...
 [04:12:35] If the error is reproducible, run the program with `-dv` arguments again to get more info.

houtianze · 2016-01-06T13:38:56Z

hmm... 我现在怀疑问题是出在pyOpenSSL上面了——是不是这个库卡住的问题，以前只装requests貌似没见过这个问题。改了一下测试的代码，我现在跑一下看看能不能不卡。

另外，如果你有空，可以帮忙测试一下是否(sudo) pip uninstall pyasn1 pyOpenSSL卸载掉这两个库是不是就没这个问题了？

houtianze · 2016-01-06T15:50:21Z

不懂是不是这个问题（看后面后挂的一堆issue。如果是貌似就只能等urllib3 / requests来修正了），用个第三方的库也不容易。。。

urllib3/urllib3#367

不过拿掉pyopenssl之后travis测试还是会卡住。。。原因不明

jacwl · 2016-01-07T03:08:31Z

卸掉pyasn1和pyopenssl之后错误的确少了很多

我昨晚睡前跑的，早上6点起来一看还是昨晚那个文件，5个多小时传了不到3GB，观察了一下貌似每传一个分片就会停一段时间，从几分钟到几小时不等，实际上传速度没那么慢

20150108 update
之前测试时用的还是4号的版本，今天更新到6号的再试已经好很多了，感谢作者这么多天的跟进

houtianze added a commit that referenced this issue Jan 2, 2016

Fix stack printing #211

157c67a

houtianze added a commit that referenced this issue Jan 3, 2016

Fix Hash Cache JSON dumping (Unicode again) #211

8aa4d19

houtianze added a commit that referenced this issue Jan 3, 2016

Fix (use string for hash value) Hash Cache JSON saving #211

e7ab66b

houtianze added a commit that referenced this issue Jan 4, 2016

Handle {'error_code': 0, 'error_msg': 'no error'} #162 #211 ; Fix a S…

b6c7ded

…yntax Error

houtianze added a commit that referenced this issue Jan 4, 2016

Fix formatex() Syntax Error; Handle (110, 'ETIMEDOUT') #211 #212

acc62f4

jacwl changed the title ~~上传出错中断Failed to save Hash Cache~~ 上传出错中断 Jan 5, 2016

jacwl closed this as completed Jan 8, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

上传出错中断 #211

上传出错中断 #211

jacwl commented Jan 2, 2016

houtianze commented Jan 2, 2016

jacwl commented Jan 2, 2016

houtianze commented Jan 3, 2016

jacwl commented Jan 3, 2016

houtianze commented Jan 3, 2016

houtianze commented Jan 4, 2016

jacwl commented Jan 4, 2016

houtianze commented Jan 4, 2016

jacwl commented Jan 5, 2016

houtianze commented Jan 6, 2016

houtianze commented Jan 6, 2016

jacwl commented Jan 7, 2016

上传出错中断 #211

上传出错中断 #211

Comments

jacwl commented Jan 2, 2016

houtianze commented Jan 2, 2016

jacwl commented Jan 2, 2016

houtianze commented Jan 3, 2016

jacwl commented Jan 3, 2016

houtianze commented Jan 3, 2016

houtianze commented Jan 4, 2016

jacwl commented Jan 4, 2016

houtianze commented Jan 4, 2016

jacwl commented Jan 5, 2016

houtianze commented Jan 6, 2016

houtianze commented Jan 6, 2016

jacwl commented Jan 7, 2016