Builds not being uploaded properly by s3cmd

[larsbergstrom]

It looks like s3cmd is installed by not being run properly. Maybe we need to inject a python in there to execute it properly? Possible that either we installed s3cmd to the Windows Native one or to the mingw one but are running it with the other. Such a mess :-/

Log is here: http://build.servo.org/builders/windows-nightly/builds/7/steps/compile_1/logs/stdio
Relevant bits:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  and have a look at the known issues list:
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions
  If the error persists, please report the
  following lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net

Error loading some components of s3cmd (Import Error)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Invoked as: /usr/bin/s3cmd --mime-type=application/octet-stream put target/2016-07-04T02-24-10-servo-tech-demo.tar.gz s3://servo-builds/nightly/windows/2016-07-04T02-24-10-servo-tech-demo.tar.gz
Problem: OSError: No such file or directory
S3cmd:   1.6.1
python:   2.7.11 (default, Mar  4 2016, 10:54:03) 
[GCC 4.9.2]
environment LANG=en_US.UTF-8

Traceback (most recent call last):
  File "/usr/bin/s3cmd", line 2900, in <module>
    from S3.S3 import S3
  File "/usr/lib/python2.7/site-packages/S3/S3.py", line 39, in <module>
    import magic
  File "/usr/lib/python2.7/site-packages/magic.py", line 152, in <module>
    libmagic = ctypes.CDLL(dll)
  File "/usr/lib/python2.7/ctypes/__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: No such file or directory

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
    An unexpected error has occurred.
  Please try reproducing the error using
  the latest s3cmd code from the git master
  branch found at:
    https://github.com/s3tools/s3cmd
  and have a look at the known issues list:
    https://github.com/s3tools/s3cmd/wiki/Common-known-issues-and-their-solutions
  If the error persists, please report the
  above lines (removing any private
  info as necessary) to:
   s3tools-bugs@lists.sourceforge.net
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

[highfive]

cc @wafflespeanut

[highfive]

cc @aneeshusa

[aneeshusa]

Don't know much about Windows so I don't have any ideas, but 2 comments:

• AFAIK s3cmd uses the magic lib just to guess the MIME type, but we're explicitly passing a MIME type so it doesn't guess. I'm not sure why it still tries to load/find the lib.
• This is the kind of bug where even a partial Salting is useful so that we don't have to deal with it again after it gets fixed.

This seems to be loading the (native) libmagic library - is libmagic available for Windows? If so, is it installed? (I'd prefer if we could get s3cmd to not try to load it in the first place, though.)

[pix64]

Magic for Windows: http://gnuwin32.sourceforge.net/packages/file.htm

[aneeshusa]

This was fixed with servo/saltfs#442.