Azure multiple repositories uses same container (elasticsearch-snapshots)

[abhishekvaid]

Elasticsearch version: 2.4.0

Plugins installed: [azure-cloud plugin, head, marvel]

JVM version: 1.8.0_101

OS version: Ubuntu 16.04

Description of the problem including expected versus actual behavior: Expected behavior will be that each container will store its own snapshots. Currently that is not happening.

Steps to reproduce:

1. create multiple cloud->azure->store entries in elasticsearch.yaml. use same storage account but different containers in different entries.
2. register multiple repos and commit to any one of them.
3. go to portal.azure.com and observe containers in storage account. Only elasticsearch-snapshots (a non pre-existing container) will be having repos and not others.

Provide logs (if relevant):

Describe the feature:

• I have defined multiple Azure repos with different containers but same storage account. The elasticsearch.yaml file has different URIs registered under reach repo with same storage account name and key. I then went on to register different repos with ES by changing "account" field in "settings". All in all I have defined three repos. instagram_archival, facebook_archival and twitter_archival. These all use same storage account and key, but different containers. I'm not sure how to specify a container. Via a URI or just the name. In this session I tried both. They both didn't work.

• One of the account is "instagram_archival" has following .yaml setting:

  instagram_archival:
  account: abcd
  key: xxx
  container: https://abcd.blob.core.windows.net/instagramarchival
  compress: true

• The problem is that when I created few snapshots in this repo (instagram_archival), it all went to default container ("elasticsearch-snaphots"). I had already created all containers from azure portal, although I didn't create this container "elasticsearch-snapshots" and elasticsearch somehow ignored those already existing and went ahead and created one of its own.

[dadoonet]

Can you provide a full elasticsearch.yml file as a gist? You can replace all key/secret values obviously but please keep the format untouched.

[abhishekvaid]

https://gist.github.com/abhishekvaid/c0e1eb2b80e5ebfefe7b709a1d1fcc68

Here's the gist

[abhishekvaid]

Can you please look at it for 10 minutes. Maybe I missed here something that you might advice me with.

[dadoonet]

@abhishekvaid Sorry for the late reply.

I'm seeing in your elasticsearch.yml file that you defined this on a client node:

node.master: false
node.client: true
node.data: false

Any reason you are doing that? I meant that AFAIK snapshot/restore runs on master/data nodes not on client nodes.
Did you also define this on master/data nodes?

[dadoonet]

No further feedback.

Closing but feel free to reopen if you have more details.