New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Azure repository creation Issue in Elasticsearch 2.4.0 #20633
Comments
@GaneshbabuRamamoorthy I edited your message to remove sensitive information. I advise you to change your credentials as soon as possible. |
You are missing one level in your settings.
See https://www.elastic.co/guide/en/elasticsearch/plugins/current/cloud-azure-repository.html |
@tlrx Thanks sure will changes the credentials. @dadoonet [ERROR][cloud.azure.storage] [localhost_RD_NDX_DATA] can not create azure storage client: Primary and secondary location URIs in a StorageUri must point to the same resource. Please find the attached yml file and error log and let me know your response. Thanks, |
@dadoonet Can you please double check this again? WIth the following looks like failing as well
And getting the following error:
|
@gmoskovicz I can't reproduce it so I probably need more details. Here is what I did so far: On a fresh new and clean 2.4.0 installation: bin/plugin install cloud-azure vi config/elasticsearch.yml cloud:
azure:
storage:
my_account:
account: MY_ACCOUNT
key: MY_KEY bin/elasticsearch Then: curl -XDELETE 127.0.0.1:9200/_snapshot/azure?pretty
curl -XPUT '127.0.0.1:9200/_snapshot/azure?pretty' -d '{
"type": "azure"
}'
curl -XDELETE 127.0.0.1:9200/foo?pretty
curl -XPUT 127.0.0.1:9200/foo/doc/1?pretty -d '{ "foo": "bar" }'
curl -XPUT "127.0.0.1:9200/_snapshot/azure/snapshot_1?wait_for_completion=true&pretty" Everything went well. Can you tell me exactly what is the command used to create the repository? |
Hi @dadoonet, I modified the below changes in all yml files,
then I started the ES and below is the error msg I am getting in logs, when I tried to create the repository using sense,
I am getting the following error msg,
Please confirm me is the cloud azure yml configuration is correct? Let me know if you need further informations. Thanks |
Ok. So I can reproduce it actually. It sounds like to happen when your key is incorrect. So to me here, we are swallowing the original exception which is far from being ideal. I believe we should raise a better exception. So for now, you have to check in the azure console that your credentials are correct. I'm using here for example the |
BTW I'm pretty sure you are seeing a similar trace as this one, right?
|
If you want to print the detailed logs of what is happening, you can also change your logging level with: curl -XPUT '127.0.0.1:9200/_cluster/settings?pretty' -d '{
"transient": {
"logger.cloud.azure": "TRACE"
}
}' But sadly, there is a bug in the TRACE log and we are not printing the exception message! :( logger.trace("fails creating container [{}]", container, e.getMessage()); Should be something like: logger.trace("fails creating container [{}]", e, container); |
Yes, Its the same trace I also got in the logs.. When I asked my infrastructure team to create repository in Azure cloud machine and they replied back with the following account and key details.
Seems the issue is based on Account name, I have given the details as such in the yml on all the nodes, when I modified the yml of account name as "devesrepo" on all nodes and then I restarted the Elasticsearch, I couldn't see any error msg's, now I am able to create repository. Regards, |
Oh I see. The account is in fact PLEASE ask your team to regenerate all the credentials. You exposed here sensitive information. |
I'm going to leave your issue opened as I'd like to fix at least the logs. |
This commit fixes 2 things. First, we were swallowing the original exception when creating a client with bad credentials. So even in `TRACE` log level, nothing useful were coming out of it. With this commit, it nows prints: ``` [2016-09-27 15:54:13,118][ERROR][cloud.azure.storage ] [node_s0] can not create azure storage client: Storage Key is not a valid base64 encoded string. ``` Closes elastic#20633. It also fixes running integration tests manually which were not working and were failing with a NPE when starting.
We were swallowing the original exception when creating a client with bad credentials. So even in `TRACE` log level, nothing useful were coming out of it. With this commit, it now prints: ``` [2016-09-27 15:54:13,118][ERROR][cloud.azure.storage ] [node_s0] can not create azure storage client: Storage Key is not a valid base64 encoded string. ``` Closes elastic#20633. Backport of elastic#20669 for master branch (6.0)
We were swallowing the original exception when creating a client with bad credentials. So even in `TRACE` log level, nothing useful were coming out of it. With this commit, it now prints: ``` [2016-09-27 15:54:13,118][ERROR][cloud.azure.storage ] [node_s0] can not create azure storage client: Storage Key is not a valid base64 encoded string. ``` Closes #20633. Backport of #20669 for 5.x branch (5.1) (cherry picked from commit 14af343)
We were swallowing the original exception when creating a client with bad credentials. So even in `TRACE` log level, nothing useful were coming out of it. With this commit, it now prints: ``` [2016-09-27 15:54:13,118][ERROR][cloud.azure.storage ] [node_s0] can not create azure storage client: Storage Key is not a valid base64 encoded string. ``` Closes #20633. Backport of #20669 for 5.0 branch (cherry picked from commit 14af343) (cherry picked from commit 6345640)
dear @dadoonet, Right now, I am getting the following error: Thanks in advance. |
Hi All,
I want to take snapshot and restore using azure repository in Linux, I have configured these below setting in yml,
with the above setting, service is started, but i am m getting below error when creating repository in Elasticsearch 2.4.0,
Error msg:-
When I looked into the ES logs I could see cloud storage account not getting created at the start of elasticsearch
Please let me know your suggestions to resolve this.
Thanks,
Ganeshbabu R
The text was updated successfully, but these errors were encountered: