Support for eu-west-2 (London) cloud-aws plugin

[nicpalmer]

Elasticsearch version: All

Plugins installed: [discovery-ec2]

JVM version: JDK8

OS version: Docker (Alpine)

Description of the problem including expected versus actual behavior:

It is a reasonable expectation that all AWS regions be usable with the cloud-aws plugins.

Currently the new eu-west-2 region is not supported and errors with the following:

[2016-12-21 10:48:34,342][DEBUG][http.netty               ] [Hydro-Man] using max_chunk_size[8kb], max_header_size[8kb], max_initial_line_length[4kb], max_content_length[100mb], receive_predictor[512kb->512kb], pipelining[true], pipelining_max_events[10000]
[2016-12-21 10:48:34,370][DEBUG][indices.recovery         ] [Hydro-Man] using max_bytes_per_sec[40mb], concurrent_streams [3], file_chunk_size [512kb], translog_size [512kb], translog_ops [1000], and compress [true]
[2016-12-21 10:48:34,378][DEBUG][indices.store            ] [Hydro-Man] using indices.store.throttle.type [NONE], with index.store.throttle.max_bytes_per_sec [10gb]
[2016-12-21 10:48:34,379][DEBUG][indices.memory           ] [Hydro-Man] using indexing buffer size [100.7mb], with indices.memory.min_shard_index_buffer_size [4mb], indices.memory.max_shard_index_buffer_size [512mb], indices.memory.shard_inactive_time [5m], indices.memory.interval [30s]
[2016-12-21 10:48:34,385][DEBUG][indices.cache.query      ] [Hydro-Man] using [node] query cache with size [10%], actual_size [100.7mb], max filter count [1000]
[2016-12-21 10:48:34,393][DEBUG][indices.fielddata.cache  ] [Hydro-Man] using size [-1] [-1b]
Exception in thread "main" java.lang.IllegalArgumentException: No automatic endpoint could be derived from region [eu-west-2]
	at org.elasticsearch.cloud.aws.AwsEc2ServiceImpl.client(AwsEc2ServiceImpl.java:198)
	at org.elasticsearch.discovery.ec2.AwsEc2UnicastHostsProvider.<init>(AwsEc2UnicastHostsProvider.java:75)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at <<<guice>>>
	at org.elasticsearch.node.Node.<init>(Node.java:213)
	at org.elasticsearch.node.Node.<init>(Node.java:140)
	at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)
	at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:178)
	at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
	at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)

Steps to reproduce:

1. Set the cloud.aws.region value in the elasticsearch.yml config file to eu-west-2.
2. Start the cluster
3. Follow log and see error

Relevant files:
elasticsearch.yml

#Network host 
network.host: 0.0.0.0
network.publish_host: _ec2_

# AWS Discovery
cloud.aws.region: eu-west-2
cloud.aws.access_key: <ACCESS KEY>
cloud.aws.secret_key: <SECRET KEY>
discovery.type: ec2
discovery.zen.minimum_master_nodes: 2

From my limited understanding of Java and the elasticsearch code base; I believe that the following change should fix the issues I am experiencing and am happy to raise a PR.
I have ran the test suite against my local repo with this change and did not receive any failures. With some guidance I'd like to compile everything and do some proper testing.

    protected static String findEndpoint(Logger logger, Settings settings) {
        String endpoint = null;
        if (CLOUD_EC2.ENDPOINT_SETTING.exists(settings)) {
            endpoint = CLOUD_EC2.ENDPOINT_SETTING.get(settings);
            logger.debug("using explicit ec2 endpoint [{}]", endpoint);
        } else if (REGION_SETTING.exists(settings) || CLOUD_EC2.REGION_SETTING.exists(settings)) {
            final String region = CLOUD_EC2.REGION_SETTING.get(settings);
            switch (region) {
                case "us-east-1":
                case "us-east":
                    endpoint = "ec2.us-east-1.amazonaws.com";
                    break;
                case "us-east-2":
                    endpoint = "ec2.us-east-2.amazonaws.com";
                    break;
                case "us-west":
                case "us-west-1":
                    endpoint = "ec2.us-west-1.amazonaws.com";
                    break;
                case "us-west-2":
                    endpoint = "ec2.us-west-2.amazonaws.com";
                    break;
                case "ap-southeast":
                case "ap-southeast-1":
                    endpoint = "ec2.ap-southeast-1.amazonaws.com";
                    break;
                case "ap-south":
                case "ap-south-1":
                    endpoint = "ec2.ap-south-1.amazonaws.com";
                    break;
                case "us-gov-west":
                case "us-gov-west-1":
                    endpoint = "ec2.us-gov-west-1.amazonaws.com";
                    break;
                case "ap-southeast-2":
                    endpoint = "ec2.ap-southeast-2.amazonaws.com";
                    break;
                case "ap-northeast":
                case "ap-northeast-1":
                    endpoint = "ec2.ap-northeast-1.amazonaws.com";
                    break;
                case "ap-northeast-2":
                    endpoint = "ec2.ap-northeast-2.amazonaws.com";
                    break;
                case "eu-west":
                case "eu-west-1":
                    endpoint = "ec2.eu-west-1.amazonaws.com";
                    break;
                case "eu-west-2":
                    endpoint = "ec2.eu-west-2.amazonaws.com";
                    break;
                case "eu-central":
                case "eu-central-1":
                    endpoint = "ec2.eu-central-1.amazonaws.com";
                    break;
                case "sa-east":
                case "sa-east-1":
                    endpoint = "ec2.sa-east-1.amazonaws.com";
                    break;
                case "cn-north":
                case "cn-north-1":
                    endpoint = "ec2.cn-north-1.amazonaws.com.cn";
                    break;
                default:
                    throw new IllegalArgumentException("No automatic endpoint could be derived from region [" + region + "]");
            }
            logger.debug("using ec2 region [{}], with endpoint [{}]", region, endpoint);
        }
        return endpoint;
    }

[dadoonet]

Hey @nicpalmer! Thanks for raising this.

You are warmly welcomed to open a PR and I'll review it.

Take a look at #21961 as an example. You can modify both repository-s3 and discovery-ec2 plugins at the same time.

[dadoonet]

Also note that as a workaround you can set for now cloud.aws.ec2.endpoint: ec2.eu-west-2.amazonaws.com in your elasticsearch.yml file.

[nicpalmer]

@dadoonet thanks for your workaround, that's done the trick for me!

I'll get the PR raised and sent over hopefully by friday.

[dadoonet]

Closed by #22308