Add support for ca-central1 region

[animageofmine]

Elasticsearch version:
5.1.1
Plugins installed: []
repository-s3
JVM version:
1.8
OS version:
debian
Description of the problem including expected versus actual behavior:
Linux elasticsearch-datanode1 3.10.0-514.2.2.el7.x86_64 #1 SMP Tue Dec 6 23:06:41 UTC 2016 x86_64 GNU/Linux

Steps to reproduce:

1. Register the repository from elasticsearch
2. get HTTP 500

I looked at the code, I don't see a switch case for ca-central-1 region. Is that something that can be added quickly? The endpoints are located here: http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region

Provide logs (if relevant):

{
  "error": {
    "root_cause": [
      {
        "type": "repository_exception",
        "reason": "[essnapshots] failed to create repository"
      }
    ],
    "type": "repository_exception",
    "reason": "[essnapshots] failed to create repository",
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "No automatic endpoint could be derived from region [ca-central-1]"
    }
  },
  "status": 500
}

[abeyad]

It should be straightforward, the relevant code is in InternalAwsS3Service (and the corresponding EC2 plugin code in AwsEc2ServiceImpl). @animageofmine Is this something you might be interested in contributing a pull request to?

[dadoonet]

You can as a work around define cloud.aws.ec2.endpoint or cloud.aws.s3.endpoint to ec2.ca-central-1.amazonaws.com and s3.ca-central-1.amazonaws.com in the meantime.

[dadoonet]

@animageofmine In case you'd like so send a PR, here is a similar one for another region: #22308

If you don't I'll add it in the next days.

[animageofmine]

@abeyad @dadoonet I can send a PR. I haven't contributed or sent a PR before, let me know if there is a guideline or documentation for the same.

I am following this documentation for now: http://hisham.hm/2016/01/01/how-to-make-a-pull-request-on-github-a-quick-tutorial/

[abeyad]

@animageofmine we'll be happy to help, that looks like a good guide. Once you have issued the PR, you will need to sign the contributor's agreement, which is straightforward. Thanks for contributing!

[nsaiisasidhar]

Hi, I am new contributor to this project, made PR for the issue above. Please review it.

[animageofmine]

@abeyad Thank you. I have the change ready and waiting for approval from management. Should be out in a bit.

[animageofmine]

@abeyad Quick question. I didn't find any tests related to this particular class. I was wondering if there is a way to test this plugin from local box. Also, let me know if this does not require explicit testing on local box.

[animageofmine]

@dadoonet

Stupid question for the workaround.

You can as a work around define cloud.aws.ec2.endpoint or cloud.aws.s3.endpoint to ec2.ca-central-1.amazonaws.com and s3.ca-central-1.amazonaws.com in the meantime.

How can you define cloud.aws.s3.endpoint to s3.ca-central-1.amazonaws.com? Is there a configuration somewhere? Would something like this work?

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "access_key" : "my_access_key",
    "secret_key" : "my_secret_key",
    "bucket": "my_bucket_name",
    "region": "ca-central-1",
    "base_path": "my_base_path",
    "endpoint": "s3.ca-central-1.amazonaws.com"
  }
}

[dadoonet]

Just remove region.

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "access_key" : "my_access_key",
    "secret_key" : "my_secret_key",
    "bucket": "my_bucket_name",
    "base_path": "my_base_path",
    "endpoint": "s3.ca-central-1.amazonaws.com"
  }
}

Or you can set in elasticsearch.yml:

cloud.aws.s3.endpoint: s3.ca-central-1.amazonaws.com

And then:

PUT _snapshot/my_s3_repository
{
  "type": "s3",
  "settings": {
    "access_key" : "my_access_key",
    "secret_key" : "my_secret_key",
    "bucket": "my_bucket_name",
    "base_path": "my_base_path"
  }
}

[animageofmine]

got it! I prefer 1st approach since we have multiple dockerized clusters in different regions using the same .yml

Thanks so much, really appreciate your quick response and help!

UPDATE: Confirmed, that the workaround works as expected.