Common known issues and their solutions

Roger Ertesvåg edited this page Sep 20, 2018 · 16 revisions

The purpose of this page is to list the most common known issues and their solutions.
Please check that your problem is not already listed here before reporting a bug.

As of today, the last release of s3cmd is 2.0.2.

1. How to update s3cmd to its last version:

The last release of s3cmd can always be downloaded from here:
http://s3tools.org/download

Or the current upstream version can be found in github:
github.com/s3tools/s3cmd (master branch).

Debian packages in Unstable and Experimental are kept up-to-date. Ubuntu Universe for very recent distro versions also is kept up-to-date.

Fedora and RHEL/CentOS/Amazon Linux packages are kept up-to-date in the Fedora and EPEL repositories, respectively.

The DEB and RPM repositories at s3tools.org are stale, and should not be used.

2. Check that your version of s3cmd is not too old

If s3cmd encounter an error, most of the time you will be able to see your s3cmd version in the generated report.
Example:

Problem: KeyErr: 'elapsed'
**S3cmd: 1.1.0-beta3**

Traceback (most recent call last):
File "/usr/bin/s3cmd", line 1800, in ?
main()

If your version is older than the last release, please update s3cmd before filling a bug report.
Your problem is certainly already fixed in a more recent version.

3. Problem: KeyError: 'elapsed'

Example:

Traceback (most recent call last):
  File "/usr/bin/s3cmd", line 1800, in <module>
    main()
  File "/usr/bin/s3cmd", line 1741, in main
    cmd_func(args)
  File "/usr/bin/s3cmd", line 309, in cmd_object_put
    (unicodise(full_name_orig), uri_final, response["size"], response["elapsed"],
KeyError: 'elapsed'

Solution:

Known bug in previous versions. Please upgrade s3cmd version to 1.5.2 or higher.

4. Problem: ParseError: syntax error: line 1, column 0

Example:

Problem: ParseError: no element found: line 1, column 0
S3cmd:   1.1.0-beta3

Traceback (most recent call last):
  File "/usr/bin/s3cmd", line 1800, in <module>
    main()
  File "/usr/bin/s3cmd", line 1741, in main
    cmd_func(args)
  File "/usr/bin/s3cmd", line 462, in cmd_object_del
    subcmd_object_del_uri(uri_str)
  File "/usr/bin/s3cmd", line 488, in subcmd_object_del_uri
    response = s3.object_delete(S3Uri(item['object_uri_str']))
  File "/usr/share/s3cmd/S3/S3.py", line 403, in object_delete
    response = self.send_request(request)
  File "/usr/share/s3cmd/S3/S3.py", line 613, in send_request
    e = S3Error(response)
  File "/usr/share/s3cmd/S3/Exceptions.py", line 48, in __init__
    tree = getTreeFromXml(response["data"])
  File "/usr/share/s3cmd/S3/Utils.py", line 69, in getTreeFromXml
    tree = ET.fromstring(xml)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1301, in XML
    return parser.close()
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1654, in close
    self._raiseerror(v)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
ParseError: no element found: line 1, column 0

Solution:

This error is probably coming from a bad response from the server you are using that doesn't return a xml formatted error response.
One of the possible causes of such an issue is when you have a firewall, proxy or load balancer between you and the s3 server that encountered the error and so doesn't return a S3 formatted xml response.

Versions 1.5.2 and higher of s3cmd treat these issues more gracefully and so updating could help you figuring out the real origin of your issue. Please try s3cmd v1.5.2 or higher which handles this a little more elegantly (e.g. w/o crashing).

5. Problem: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal

Solution: ensure your LANG= environment variable is set to a value with a Unicode locale (not None). A good value is LANG=en_US.UTF-8. You must have a unicode locale in order for the python string libraries to properly handle unicode.

Solution: upgrade to v1.5.2 or higher. Many Unicode bugs have been found and fixed recently.

6: Problem: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2022' in position 45: ordinal not in range(128)

Solution: see above UnicodeWarning solution.

7: Problem: s3cmd not working with a bucket in the "cn-north-1" region

Solution: For this location where buckets are located in china, you need to set specific domain parameters. Set the following parameters in your .s3cfg configuration file:

bucket_location = cn-north-1
host_base = s3.cn-north-1.amazonaws.com.cn
host_bucket = %(bucket)s.s3.cn-north-1.amazonaws.com.cn
website_endpoint = http://%(bucket)s.s3-website-%(location)s.amazonaws.com.cn/

(Thanks to wbchn for this solution)

8: Problem: Forbidden/access denied error when doing mv/cp operation

Solution: Check the policy of your user to ensure that you have all the required permissions.

The following policy should provide all the necessary permissions needed for all s3cmd operations:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Stmt123456",
            "Effect": "Allow",
            "Action": [
                "s3:ListAllMyBuckets"
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        },
        {
            "Sid": "Stmt123457",
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:ListBucketMultipartUploads",
                "s3:GetBucketLocation",
                "s3:AbortMultipartUpload",
                "s3:GetObjectAcl",
                "s3:GetObjectVersion",
                "s3:DeleteObject",
                "s3:DeleteObjectVersion",
                "s3:GetObject",
                "s3:PutObjectAcl",
                "s3:PutObject",
                "s3:GetObjectVersionAcl"
            ],
            "Resource": [
                "arn:aws:s3:::my_bucket",
                "arn:aws:s3:::my_bucket/*"
            ]
        }
    ]
}
where my_bucket is the bucket where I need to manage files though s3cmd

(_Thanks to rorra of SO for this solution: https://stackoverflow.com/a/30351767/1228040)

9. I still have a bug and can't find here a solution for it

Feel free to report it by email to: s3tools-bugs@lists.sourceforge.net

Please, don't forget to attach the error report generated by s3cmd to your message.

Also, please note that you can run s3cmd with the flag "--debug" to generate more complete log that could help you figure out the origin of your issue.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.