-
-
Notifications
You must be signed in to change notification settings - Fork 902
Common known issues and their solutions (FAQ)
The purpose of this page is to list the most common known issues and their solutions.
Please check that your problem is not already listed here before reporting a bug.
As of today, the last release of s3cmd is 2.3.0.
The last release of s3cmd can always be downloaded from here:
http://s3tools.org/download
Or the current upstream version can be found in github:
https://github.com/s3tools/s3cmd (master branch).
Debian packages in Unstable and Experimental are kept up-to-date. Ubuntu Universe for very recent distro versions also is kept up-to-date.
Fedora and RHEL/CentOS/Amazon Linux packages are kept up-to-date in the Fedora and EPEL repositories, respectively.
The DEB and RPM repositories at s3tools.org are stale, and should not be used.
If s3cmd encounter an error, most of the time you will be able to see your s3cmd version in the generated report.
Example:
Problem: KeyErr: 'elapsed'
**S3cmd: 1.1.0-beta3**
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 1800, in ?
main()
If your version is older than the last release, please update s3cmd before filling a bug report.
Your problem is certainly already fixed in a more recent version.
Example:
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 1800, in <module>
main()
File "/usr/bin/s3cmd", line 1741, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 309, in cmd_object_put
(unicodise(full_name_orig), uri_final, response["size"], response["elapsed"],
KeyError: 'elapsed'
Solution:
Known bug in previous versions. Please upgrade s3cmd version to 1.5.2 or higher.
Example:
Problem: ParseError: no element found: line 1, column 0
S3cmd: 1.1.0-beta3
Traceback (most recent call last):
File "/usr/bin/s3cmd", line 1800, in <module>
main()
File "/usr/bin/s3cmd", line 1741, in main
cmd_func(args)
File "/usr/bin/s3cmd", line 462, in cmd_object_del
subcmd_object_del_uri(uri_str)
File "/usr/bin/s3cmd", line 488, in subcmd_object_del_uri
response = s3.object_delete(S3Uri(item['object_uri_str']))
File "/usr/share/s3cmd/S3/S3.py", line 403, in object_delete
response = self.send_request(request)
File "/usr/share/s3cmd/S3/S3.py", line 613, in send_request
e = S3Error(response)
File "/usr/share/s3cmd/S3/Exceptions.py", line 48, in __init__
tree = getTreeFromXml(response["data"])
File "/usr/share/s3cmd/S3/Utils.py", line 69, in getTreeFromXml
tree = ET.fromstring(xml)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1301, in XML
return parser.close()
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1654, in close
self._raiseerror(v)
File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
raise err
ParseError: no element found: line 1, column 0
Solution:
This error is probably coming from a bad response from the server you are using that doesn't return a xml formatted error response.
One of the possible causes of such an issue is when you have a firewall, proxy or load balancer between you and the s3 server that encountered the error and so doesn't return a S3 formatted xml response.
Versions 1.5.2 and higher of s3cmd treat these issues more gracefully and so updating could help you figuring out the real origin of your issue. Please try s3cmd v1.5.2 or higher which handles this a little more elegantly (e.g. w/o crashing).
5. Problem: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
Solution: ensure your LANG= environment variable is set to a value with a Unicode locale (not None). A good value is LANG=en_US.UTF-8. You must have a unicode locale in order for the python string libraries to properly handle unicode.
Solution: upgrade to v1.5.2 or higher. Many Unicode bugs have been found and fixed recently.
6: Problem: UnicodeEncodeError: 'ascii' codec can't encode character u'\u2022' in position 45: ordinal not in range(128)
Solution: see above UnicodeWarning solution.
Solution: For this location where buckets are located in china, you need to set specific domain parameters. Set the following parameters in your .s3cfg configuration file:
bucket_location = cn-north-1
host_base = s3.cn-north-1.amazonaws.com.cn
host_bucket = %(bucket)s.s3.cn-north-1.amazonaws.com.cn
website_endpoint = http://%(bucket)s.s3-website-%(location)s.amazonaws.com.cn/
(Thanks to wbchn for this solution)
Solution: Check the policy of your user to ensure that you have all the required permissions.
The following policy should provide all the necessary permissions needed for all s3cmd operations:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt123456",
"Effect": "Allow",
"Action": [
"s3:ListAllMyBuckets"
],
"Resource": [
"arn:aws:s3:::*"
]
},
{
"Sid": "Stmt123457",
"Effect": "Allow",
"Action": [
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:GetBucketLocation",
"s3:AbortMultipartUpload",
"s3:GetObjectAcl",
"s3:GetObjectVersion",
"s3:DeleteObject",
"s3:DeleteObjectVersion",
"s3:GetObject",
"s3:PutObjectAcl",
"s3:PutObject",
"s3:GetObjectVersionAcl"
],
"Resource": [
"arn:aws:s3:::my_bucket",
"arn:aws:s3:::my_bucket/*"
]
}
]
}
where my_bucket is the bucket where I need to manage files though s3cmd
(_Thanks to rorra of SO for this solution: https://stackoverflow.com/a/30351767/1228040)
Help on this topic can be found in the following wiki page:
Understand Content Type, Mime type guessing and Magic
10. Problem: My uploaded file is displayed instead of being downloaded when I try to get it with my browser
Solution: Please ensure to set the "Content-Disposition" header to "attachment" when you upload the file.
And so, it will be "downloaded" and not "rendered" if you try to access it with your browser through a direct url link.
(Note: Trying to set a specific Content-Type to achieve this purpose is useless)
Feel free to report it by email to: s3tools-bugs@lists.sourceforge.net
Please, don't forget to attach the error report generated by s3cmd to your message.
Also, please note that you can run s3cmd with the flag "--debug" to generate more complete log that could help you figure out the origin of your issue.