'us-east-1` is not a valid region when creating s3 bucket

[yandooo]

Elasticsearch version: 5.2.0

  "version" : {
    "number" : "5.2.0",
    "build_hash" : "24e05b9",
    "build_date" : "2017-01-24T19:52:35.800Z",
    "build_snapshot" : false,
    "lucene_version" : "6.4.0"
  }

JVM version: 1.8

OS version: Alpine Linux

Describe the feature:

When we use us-east-1 s3 plugin does not work.

PUT _snapshot/test
{
  "type": "s3",
  "settings": {
    "bucket": "repo-bucket-1",
    "region": "us-east-1"
  }
}

Response:

{"error":{
"root_cause":[{"type":"amazon_s3_exception","reason":"amazon_s3_exception: The specified location-constraint is not valid (Service: Amazon S3; Status Code: 400; Error Code: InvalidLocationConstraint; Request ID: F44633A6F3AD5323)"}],
"type":"repository_exception","reason":"[s3_backup_repository] failed to create repository","caused_by":
{"type":"amazon_s3_exception","reason":"amazon_s3_exception: The specified location-constraint is not valid (Service: Amazon S3; Status Code: 400; Error Code: InvalidLocationConstraint; Request ID: F44633A6F3AD5323)"}},
"status":500}

removing region does not work as well. Any ideas?

[dadoonet]

I believe it's a duplicate of #16978.

Closing.

Note that we are removing region setting as we prefer people use endpoint instead.

See https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-repository.html#repository-s3-endpoint

See #22848 and #22853.

[yandooo]

@dadoonet I posted in description

removing region does not work as well. Any ideas?

[dadoonet]

Did you try with "endpoint": "s3.amazonaws.com" instead?

[yandooo]

@dadoonet same issue. Trying dualstack seems like goes further but not match:

so request

PUT _snapshot/test
{
  "type": "s3",
  "settings": {
   "endpoint": "s3.dualstack.us-east-1.amazonaws.com/test"
  }
}

error code is different (need to check what kind of key expected)

{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[s3_backup_repository] path  is not accessible on master node"}],"type":"repository_verification_exception","reason":"[s3_backup_repository] path  is not accessible on master node","caused_by":{"type":"i_o_exception","reason":"Unable to upload object tests-e0_8EQz6QQKclw7Tx13L_g/master.dat-temp","caused_by":{"type":"amazon_s3_exception","reason":"amazon_s3_exception: The request signature we calculated does not match the signature you provided. Check your key and signing method. (Service: Amazon S3; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: 46692488122C6B99)"}}},"status":500}

[yandooo]

@dadoonet just double checked it and it doesn't work:

{
  "type": "s3",
  "settings": {
    "bucket": "test",
	"endpoint": "s3.amazonaws.com"
  }
}

error response:

"amazon_s3_exception: The specified location-constraint is not valid (Service: Amazon S3; Status Code: 400; Error Code: InvalidLocationConstraint;

[dadoonet]

I'm surprised by what you pasted before you edited the response.

I just ran a test:

curl -XDELETE 127.0.0.1:9200/_snapshot/s3repo?pretty
curl -XPUT 127.0.0.1:9200/_snapshot/s3repo?pretty -d '{
  "type": "s3",
  "settings": {
	  "bucket": "david-test-to-remove.my.domain.com"
  }
}'

And in elasticsearch.yml, I defined my credentials:

cloud:
  aws:
    access_key: "ACCESS"
    secret_key: "SECRET"

Everything is fine.

Then:

curl -XDELETE 127.0.0.1:9200/_snapshot/s3repo?pretty
curl -XPUT 127.0.0.1:9200/_snapshot/s3repo?pretty -d '{
  "type": "s3",
  "settings": {
	  "bucket": "david-test-to-remove.my.domain.com"
	  "endpoint": "s3.amazonaws.com"
  }
}'

Worked perfectly as well. So you may be doing something wrong or you don't tell all the steps/details.

[dadoonet]

BTW I tested on 5.2.1.

[dadoonet]

@yandooo Have a look at #16978 (comment)
It actually might explain what you are seeing.

[yandooo]

@dadoonet I pass AWS key and value as env variable as per plugin documentation (ES is running in docker container). Shouldn't be any difference though.

Buckets exist in AWS (I'll double check again later) so #16978 (comment) doen't shed light on what's going on.

[dadoonet]

I pass AWS key and value as env variable

Note that this will be removed in 6.0 (master). See #22842.

Can you try with elasticsearch.yml config file in case it makes any difference?

[yandooo]

@dadoonet I think I sorted this issue. It was wrong SECRET_ID passed to container.

Now I have ES specific issue:

{"error":{"root_cause":[{"type":"repository_verification_exception","reason":"[s3_backup] [[3HYa7OTwQZyvcbWUwlfj5Q, 'RemoteTransportException[[3HYa7OT][10.51.96.5:9300][internal:admin/repository/verify]]; nested: RepositoryMissingException[[s3_backup] missing];']]"}],"type":"repository_verification_exception","reason":"[s3_backup] [[3HYa7OTwQZyvcbWUwlfj5Q, 'RemoteTransportException[[3HYa7OT][10.51.96.5:9300][internal:admin/repository/verify]]; nested: RepositoryMissingException[[s3_backup] missing];']]"},"status":500}

any ideas?

[yandooo]

Running with ?verify=false returns true.

[yandooo]

Problem is solved.

Thanks everyone.