New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error uploading empty file: "seek() takes 2 positional arguments but 3 were given" #2403

Open
ajdavis opened this Issue Jan 21, 2017 · 17 comments

Comments

Projects
None yet
@ajdavis

ajdavis commented Jan 21, 2017

awscli version "1.11.13-1ubuntu1~16.04.0" was installed with the Ubuntu package manager and run like:

aws s3 cp doc/html s3://BUCKET/PATH/HERE --recursive --acl public-read --region us-east-1

It successfully uploads all the files in the source directory except for one zero-byte file. It fails there with:

seek() takes 2 positional arguments but 3 were given
@kyleknap

This comment has been minimized.

Member

kyleknap commented Jan 23, 2017

I was not able to reproduce the issue with that version of the CLI. I have a feeling it may be with how the CLI got packaged for Ubuntu. Unfortunately, we do not maintain those methods of installation. So I am not sure what may be causing the issue. These are the methods that we do maintain: http://docs.aws.amazon.com/cli/latest/userguide/installing.html. Do you get the same issue when you use pip or the bundled installer?

@ixodie

This comment has been minimized.

ixodie commented Jan 24, 2017

I am having a similar issue.

aws-cli/1.11.13 Python/3.5.2 Linux/4.4.0-59-generic botocore/1.4.70 (ubuntu 16.04)

I do:

aws s3 sync /mnt/ebs-volume/image/ s3://xxx-my-images

And I get a TON of messages like this:
upload failed: image/xxx/xxx/xxx.jpg to s3://xxx-my-images/xxx/xxx//xxx.jpg seek() takes 2 positional arguments but 3 were given

So I cannot really be sure that the images are being uploaded/synced properly.

@JordonPhillips

This comment has been minimized.

Member

JordonPhillips commented Jan 25, 2017

@kyleknap I wonder what version of requests is being used. This looks like the same kind of error that you'll get if you try to use the latest version of requests, and I know Ubuntu un-bundles our vendored version in favor of their version.

@ixodie

This comment has been minimized.

ixodie commented Jan 25, 2017

I can troubleshoot here if you tell me what to look at.

I should point out that I am getting that message on non-zero length files as well.

@ctrlrsf

This comment has been minimized.

ctrlrsf commented Jan 27, 2017

I was seeing these errors with awscli installed from Ubuntu 16.04 package. Worked around it by installing awscli with pip. See below for version differences. Note, awscli from Ubuntu is running on python 3.5 and awscli I installed with pip is actually python 2.7.

$ dpkg -l | grep aws
ii  awscli                              1.11.13-1ubuntu1~16.04.0            all          Universal Command Line Environment for AWS

# Ubuntu version:
$ aws --version
aws-cli/1.11.13 Python/3.5.2 Linux/4.4.0-45-generic botocore/1.4.70

# pip install awscli version:
$ ~/.local/bin/aws --version
aws-cli/1.11.44 Python/2.7.12 Linux/4.4.0-45-generic botocore/1.5.7
@ixodie

This comment has been minimized.

ixodie commented Jan 27, 2017

Worked like a charm. Thanks!

@omervk

This comment has been minimized.

omervk commented Mar 7, 2017

As long as the tool only supports Python 2, there should be an error when trying to use Python 3.

Partyschaum added a commit to Jimdo/reportbook that referenced this issue Jun 1, 2017

Use bundled aws CLI instead of packaged one
Right now the packaged version is broken in current Ubuntu releases: aws/aws-cli#2403
@liwo

This comment has been minimized.

liwo commented Jun 8, 2017

I just created a bug report with ubuntu to let the package maintainers know about the issue. Maybe they can sort this out?

https://bugs.launchpad.net/ubuntu/+source/awscli/+bug/1696800

kamalmostafa added a commit to kamalmostafa/s3transfer that referenced this issue Jun 12, 2017

Implement seek whence param for zero length files
Bug: aws/aws-cli#2403
Bug-Ubuntu: https://bugs.launchpad.net/bugs/1696800

As described in the bug reports referenced above, trying to copy a
zero-length file to S3 fails (with some version combinations of python3,
awscli, s3transfer, and requests) like this:

    $ aws s3 cp emptyfile s3://somebucket
    upload failed: ./emptyfile to s3://somebucket/emptyfile seek() takes 2 positional arguments but 3 were given

... due to the incomplete implementation of seek() -- missing the optional
'whence' argument -- in s3transfer's ReadFileChunk method.  This patch
adds the whence argument to all of s3transfer's seek implementations.

tomstich added a commit to Jimdo/reportbook that referenced this issue Sep 21, 2017

Use bundled aws CLI instead of packaged one (#489)
Right now the packaged version is broken in current Ubuntu releases: aws/aws-cli#2403
@nlebas

This comment has been minimized.

nlebas commented Oct 25, 2017

I've had the same problem when trying to upload an open file. The data had not been flushed to disk, yet.
Closing my file before uploading it solved the issue.

$ aws --version
aws-cli/1.11.13 Python/3.5.2 Linux/4.4.0-1013-aws botocore/1.4.70

$ cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.3 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.3 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

@Preen

This comment has been minimized.

Preen commented Jan 23, 2018

still an issue on Ubuntu 16.04 LTS.

dbnicholson added a commit to dbnicholson/s3transfer that referenced this issue Jan 24, 2018

Implement seek whence param for zero length files
Bug: aws/aws-cli#2403
Bug-Ubuntu: https://bugs.launchpad.net/bugs/1696800

As described in the bug reports referenced above, trying to copy a
zero-length file to S3 fails (with some version combinations of python3,
awscli, s3transfer, and requests) like this:

    $ aws s3 cp emptyfile s3://somebucket
    upload failed: ./emptyfile to s3://somebucket/emptyfile seek() takes 2 positional arguments but 3 were given

... due to the incomplete implementation of seek() -- missing the optional
'whence' argument -- in s3transfer's ReadFileChunk method.  This patch
adds the whence argument to all of s3transfer's seek implementations.
@ASayre

This comment has been minimized.

Contributor

ASayre commented Feb 6, 2018

Good Morning!

We're closing this issue here on GitHub, as part of our migration to UserVoice for feature requests involving the AWS CLI.

This will let us get the most important features to you, by making it easier to search for and show support for the features you care the most about, without diluting the conversation with bug reports.

As a quick UserVoice primer (if not already familiar): after an idea is posted, people can vote on the ideas, and the product team will be responding directly to the most popular suggestions.

We’ve imported existing feature requests from GitHub - Search for this issue there!

And don't worry, this issue will still exist on GitHub for posterity's sake. As it’s a text-only import of the original post into UserVoice, we’ll still be keeping in mind the comments and discussion that already exist here on the GitHub issue.

GitHub will remain the channel for reporting bugs.

Once again, this issue can now be found by searching for the title on: https://aws.uservoice.com/forums/598381-aws-command-line-interface

-The AWS SDKs & Tools Team

This entry can specifically be found on UserVoice at: https://aws.uservoice.com/forums/598381-aws-command-line-interface/suggestions/33168337-error-uploading-empty-file-seek-takes-2-positi

@ASayre ASayre closed this Feb 6, 2018

@jamesls

This comment has been minimized.

Member

jamesls commented Apr 6, 2018

Based on community feedback, we have decided to return feature requests to GitHub issues.

@jamesls jamesls reopened this Apr 6, 2018

@aerostitch

This comment has been minimized.

aerostitch commented May 15, 2018

So if you start debugging a minimum (because we're in 2018 and saying "use python2" is really lame) using:

$ aws --debug s3 cp /tmp/bla s3://my-bucket/bla

You get something that looks like:

efc2ebbbbc0c01db31c7f42e23d4a3c094f91f78bf1caf56710be7d436f4c559
2018-05-15 14:46:00,184 - ThreadPoolExecutor-0_0 - botocore.auth - DEBUG - Signature:
ed7af55076ae84ac2c26bab35bfc0a856113caa84577e7f7972d9ccf6ddf6a32
2018-05-15 14:46:00,184 - ThreadPoolExecutor-0_0 - botocore.hooks - DEBUG - Event request-created.s3.PutObject: calling handler <function signal_transferring at 0x7f2ab9123d90>
2018-05-15 14:46:00,184 - ThreadPoolExecutor-0_0 - s3transfer.tasks - DEBUG - Exception raised.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/usr/lib/python3/dist-packages/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/usr/lib/python3/dist-packages/s3transfer/upload.py", line 692, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 314, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/lib/python3/dist-packages/botocore/client.py", line 599, in _make_api_call
    operation_model, request_dict)
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 148, in make_request
    return self._send_request(request_dict, operation_model)
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 173, in _send_request
    request = self.create_request(request_dict, operation_model)
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 158, in create_request
    prepared_request = self.prepare_request(request)
  File "/usr/lib/python3/dist-packages/botocore/endpoint.py", line 169, in prepare_request
    return request.prepare()
  File "/usr/lib/python3/dist-packages/botocore/awsrequest.py", line 371, in prepare
    p.prepare_body(self.data, self.files)
  File "/usr/lib/python3/dist-packages/botocore/awsrequest.py", line 442, in prepare_body
    data.seek(0, 2)
TypeError: seek() takes 2 positional arguments but 3 were given

And there you see that the element you are trying to seek on is a s3transfer.utils.ReadFileChunk which does have only 1 argument (outside of self): https://github.com/boto/s3transfer/blob/master/s3transfer/utils.py#L470
So either botocore needs to be patched to change the logic https://github.com/boto/botocore/blob/develop/botocore/awsrequest.py#L441 or s3transfer needs to support seek with 3 args.

@aerostitch

This comment has been minimized.

aerostitch commented May 15, 2018

A patch is actually already proposed in s3transfert via: boto/s3transfer#88 and boto/s3transfer#102

@dhruveshsheladiya

This comment has been minimized.

dhruveshsheladiya commented Jul 24, 2018

Try installing awscli from pip which is the suggested/supported way of doing so: http://docs.aws.amazon.com/cli/latest/userguide/installing.html

@KES777

This comment has been minimized.

KES777 commented Nov 15, 2018

I have same issue. But I upload zero length files first time.

Second try get me errors

@dbnicholson

This comment has been minimized.

dbnicholson commented Nov 17, 2018

The issue occurs when using the s3transfer library with python3. I fixed this in boto/s3transfer#102 10 months ago. Please review that and merge it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment