S3 Authentication issue: AWS demands updated SDK

[amks1]

I've installed Kobocat version 2.021.45. When trying to use Amazon S3 storage, the collected media gets stored in the bucket and I am able to browse it there.
However when trying to view the submission in the Kobotoolbox KPI interface, the image doesn't load and the following error is returned on clicking the broken image:

The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

Doing a quick search, I came across this post: https://stackoverflow.com/questions/26533245/the-authorization-mechanism-you-have-provided-is-not-supported-please-use-aws4, which essentially says that the SDK needs to be updated for S3 instances created in regions that were added after January 2014.

[amks1]

The solution to this is to add the following lines in Django settings. (Tested and working with Kobocat)

AWS_S3_SIGNATURE_VERSION = 's3v4'
AWS_S3_REGION_NAME = 'eu-central-1'

However, as the region would be different for different users, I'm thinking this should be added as a sub-question in the AWS section of the kobo-install script?

[jnm]

This isn't required on our EU instances, which have only these environment variables set inside the kobocat container:

root@kobocat:/srv/src/kobocat# env | grep AWS | cut -f 1 -d '='
KOBOCAT_AWS_STORAGE_BUCKET_NAME
KOBOCAT_AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
KPI_AWS_STORAGE_BUCKET_NAME
KOBOCAT_AWS_SECRET_ACCESS_KEY
AWS_ACCESS_KEY_ID

Inside the Django shell on a EU production server:

>>> settings.AWS_
settings.AWS_ACCESS_KEY_ID        settings.AWS_QUERYSTRING_EXPIRE   settings.AWS_S3_HOST              settings.AWS_SECRET_ACCESS_KEY    
settings.AWS_DEFAULT_ACL          settings.AWS_S3_FILE_BUFFER_SIZE  settings.AWS_S3_USE_SSL           settings.AWS_STORAGE_BUCKET_NAME 

Are you trying to use Amazon S3, or are you seeing an issue when using DigitalOcean as described in kobotoolbox/kpi#3750?

[amks1]

This isn't required on our EU instances, which have only these environment variables set inside the kobocat container:

It applies to S3 regions that were created after 2014, so it could be that your EU instance regions are older than that.

Are you trying to use Amazon S3, or are you seeing an issue when using DigitalOcean as described in kobotoolbox/kpi#3750?

This was while testing with Amazon S3 and is unrelated to the other issue.

[jnm]

Thanks. You're right: we have an EC2 instance in eu-central-1 but it's using S3 storage in an older region.

@noliveleger do you think we could attempt to connect to S3 using the new s3v4 signature version by default and fall back to attempting the old one if it fails? At first glance, it feels nicer than burdening people with another manual setting, but if it's more straightforward to add an environment variable and configure it with kobo-install, so be it. It's up to your judgment.