Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add retries to the Upload resource #147

Closed
jsh2134 opened this issue Apr 12, 2017 · 0 comments
Closed

Add retries to the Upload resource #147

jsh2134 opened this issue Apr 12, 2017 · 0 comments
Milestone

Comments

@jsh2134
Copy link
Contributor

jsh2134 commented Apr 12, 2017

When doing an import with 50 files, I used the import shortcut. On the 44th upload the connection to upload was aborted and I have to start again.

Having the upload endpoint retry multiple times would greatly improve this workflow.

Total time elapsed: 00:01:07
Successfuly uploaded file vt.exac.decomposed.normalized.vcf.gz.044.json.gz (id:432432806254611136 size:97631659 md5:761267ce7102cdf6bad111642af0dda0)
Uploading file: vt.exac.decomposed.normalized.vcf.gz.001.json.gz
[ 77 %] Time elapsed: 00:03:40 | ETA: 00:01:03

[100 %] Time elapsed: 00:07:39 | ETA: 00:00:00
Total time elapsed: 00:07:39
Traceback (most recent call last):
  File "/Users/ganesha/.virtualenvs/sbdp/bin/solvebio", line 11, in <module>
    load_entry_point('solvebio', 'console_scripts', 'solvebio')()
  File "/Users/ganesha/code/solve/solvebio-python/solvebio/cli/main.py", line 231, in main
    return args.func(args)
  File "/Users/ganesha/code/solve/solvebio-python/solvebio/cli/data.py", line 120, in import_file
    manifest.add(*args.file)
  File "/Users/ganesha/code/solve/solvebio-python/solvebio/resource/manifest.py", line 69, in add
    self.add_file(path)
  File "/Users/ganesha/code/solve/solvebio-python/solvebio/resource/manifest.py", line 29, in add_file
    upload = solvebio.Upload.create(path=path)
  File "/Users/ganesha/code/solve/solvebio-python/solvebio/resource/upload.py", line 112, in create
    headers=headers)
  File "/Users/ganesha/.virtualenvs/sbdp/lib/python2.7/site-packages/requests/api.py", line 124, in put
    return request('put', url, data=data, **kwargs)
  File "/Users/ganesha/.virtualenvs/sbdp/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Users/ganesha/.virtualenvs/sbdp/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/ganesha/.virtualenvs/sbdp/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
    r = adapter.send(request, **kwargs)
  File "/Users/ganesha/.virtualenvs/sbdp/lib/python2.7/site-packages/requests/adapters.py", line 473, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(32, 'Broken pipe'))
@davecap davecap modified the milestone: v1.16.0 Apr 17, 2017
davecap added a commit that referenced this issue Apr 19, 2017
Add retries to Upload creation (fixes #147)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants