S3 Uploader: Authorization mechanism ... is not supported. Please use AWS4-HMAC-SHA256.

[andygrunwald]

I run into the following issue:

./complainer -masters "http://192.168.99.100:5050" \
                     -reporters "slack" \
                     -slack.channel "#mesos" \
                     -slack.hook_url "https://hooks.slack.com/services/TOKEN" \
                     -slack.username "Mesos Cluster" \
                     -slack.icon_emoji ":mesos:" \
                     -uploader "s3" \
                     -s3.access_key "KEY" \
                     -s3.bucket "BUCKET" \
                     -s3.endpoint "https://s3.eu-central-1.amazonaws.com/" \
                     -s3.secret_key "SECRET"

2016/06/08 19:05:02 Reporting ChronosTask:my-failing-job (ct:1465405479849:0:my-failing-job:) from 192.168.99.100
2016/06/08 19:05:02 Error reporting failure of ct:1465405479849:0:my-failing-job:: cannot get stdout and stderr urls from uploader: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

This seems to be related to goamz/goamz#118

[andygrunwald]

An alternative would be to fire the S4 Put operation on our own and sign the request with v4Signer := aws.NewV4Signer(auth, "s3", region)
Or complainer will switch from goamz/goamz to the official Golang Client library from AWS at https://github.com/aws/aws-sdk-go

Here is the PutObject-Operation of the official golang sdk from AWS.

Are you open for such a change?
I`d like to hear your opinion about this.

[bobrik]

This is pretty messy.

• Official SDK does not support v2 signatures: aws/signer: Add S3 V2 request signature aws/aws-sdk-go#400
• goamz does not support v4: Fully implement AWS4 signing support goamz/goamz#118

Ceph and many other systems are still v2, AWS itself is v4.

I guess we'll have to split s3 uploader into two: official one + goamz for ceph and friends.

[andygrunwald]

Only 2 AWS regions right now rely on v4 signatures: cn-north-1 and eu-central-1 (+ all newer regions).
All others are able to work with v2 signature, but are able to work with v4 as well.

If you are talking about S3 on Ceph you are talking about http://docs.ceph.com/docs/master/radosgw/s3/ ?
If yes, according to http://tracker.ceph.com/issues/10333 and ceph/ceph#4943 this was implemented in Feb this year.

I like your idea about the two cases split.
Are you planning to work on this? If not, i try to find some time to do this, but i am only to able the eu-central-1 + v4 combination. But i guess you would be able to test the Ceph interface.

[bobrik]

I'll take a look, hopefully this week.

Thanks for bringing this up.