Skip to content

Commit

Permalink
Don't try direct-to-S3 uploads on files >5GB. Closes #261.
Browse files Browse the repository at this point in the history
  • Loading branch information
polyatail committed Jun 5, 2019
1 parent cb71cdd commit 9d9f672
Showing 1 changed file with 9 additions and 2 deletions.
11 changes: 9 additions & 2 deletions onecodex/lib/upload.py
Original file line number Diff line number Diff line change
Expand Up @@ -655,11 +655,18 @@ def upload_sequence_fileobj(file_obj, file_name, fields, retry_fields, session,
-------
`string` containing sample ID of newly uploaded file.
"""

# First attempt to upload via our validating proxy
try:
_direct_upload(file_obj, file_name, fields, session, samples_resource)
sample_id = fields["sample_id"]

# Are we being directed to skip the proxy? If so, don't try to upload >5GB files
if (
"AWSAccessKeyId" in fields["additional_fields"]
and getattr(file_obj, "_fsize", 0) > 5 * 1024 ** 3
):
raise RetryableUploadException

_direct_upload(file_obj, file_name, fields, session, samples_resource)
except RetryableUploadException:
# upload failed -- retry direct upload to S3 intermediate; first try to cancel pending upload
try:
Expand Down

0 comments on commit 9d9f672

Please sign in to comment.