Another 6.x support pull request

[shribigb]

This is another 6.x pull request. It covers x-pack support and plugin support. Tested installing both 5.6.3 and 6.1.0 version with the same role. I also added adding bootstrap user in case es_api_basic_auth_username is elastic.

[gregdurham]

This section causes issues on my setup, when xpack is set to disabled due to the keystore not existing. Should this block of code ever be hit when xpack is not installed? Should we block it with a when: (es_enable_xpack and '"security" in es_xpack_features')

[ghost]

@gregdurham just his this issue too. Your suggestion will work and think its the way to go.

[shribigb]

Yes I am making those changes, I encountered the same in further testing

[gregdurham]

Thank you!

[ghost]

Thanks!

[gregdurham]

Had issues with this on RHEL 7, the binary java8 does not exist

[gregdurham]

i.e. it is already just java

[shribigb]

Amazon AMIs have different behaviour than RedHat. Made the change and tested on both Amazon AMI and redhat 7.4. Thanks @gregdurham for reviewing

[ghost]

https://github.com/shribigb/ansible-elasticsearch/blob/3076ed920eacca4f0aa932bea9f9d0b9207a5437/tasks/xpack/security/elasticsearch-security-native.yml#L16

This task fails on the second run if the {{es_api_basic_auth_password}} has been changed from the that of the original bootstrap password.

[shribigb]

@Hokeycokey I will give it a try, from the documentation it seems this should be fine(https://www.elastic.co/guide/en/x-pack/6.1/setting-up-authentication.html#set-built-in-user-passwords).. if you change the password for elastic after setting bootstrap.password it will take the new password. I have a check to find out if bootstrap.password is already set.. in the second run I dont change/set bootstrap password. As long as you are sending new password as part of your playbook run in theory it should work. But I will test it anyways. So to reproduce in first run say you set the password to changeme for elastic user and then changed it to changeme1 through API and in next run of the playbook you are setting es_api_basic_auth_password to changeme1 and it fails there?

[ghost]

Hi @shribigb and thanks.

In defaults I have the following:

es_api_basic_auth_username: elastic
es_api_basic_auth_password: changeme

Within playbook:

vars:
   es_users:
     native:
       elastic: 
         password: changednow

Error on second run:

ASK [elasticsearch : List Native Users] ***************************************
fatal: [192.168.2.5]: FAILED! => {"changed": false, "content": "{\"error\":{\"root_cause\":[{\"type\":\"security_exception\",\"reason\":\"failed to authenticate user [elastic]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"}}],\"type\":\"security_exception\",\"reason\":\"failed to authenticate user [elastic]\",\"header\":{\"WWW-Authenticate\":\"Basic realm=\\\"security\\\" charset=\\\"UTF-8\\\"\"}},\"status\":401}", "content_length": "345", "content_type": "application/json; charset=UTF-8", "json": {"error": {"header": {"WWW-Authenticate": "Basic realm=\"security\" charset=\"UTF-8\""}, "reason": "failed to authenticate user [elastic]", "root_cause": [{"header": {"WWW-Authenticate": "Basic realm=\"security\" charset=\"UTF-8\""}, "reason": "failed to authenticate user [elastic]", "type": "security_exception"}], "type": "security_exception"}, "status": 401}, "msg": "Status code was not [200]: HTTP Error 401: Unauthorized", "redirected": false, "status": 401, "url": "http://localhost:9200/_xpack/security/user", "www_authenticate": "Basic realm=\"security\" charset=\"UTF-8\""}

It seems as though the task uses es_api_basic_auth_password: changeme, on every run. It works on the first run of course. By the 2nd run the password has changed.

[shribigb]

Hi @Hokeycokey that's expected behaviour. Role doen't know anything about you have changed password of es_api_basic_auth_username in setting es_users.native.elastic.password to changednow. Rather in first run itself its setting bootstrap.password to changeme and then changing the password of elastic to changednow when its trying to change passwords of native users. In second run in your playbook, you should be considering setting es_api_basic_auth_password: changednow.

[Crazybus]

Great work on this @shribigb! Seems to be just a few small things that need to be fixed up before this can be merged into master. I think we should also be changing the default version to be 6.x once this has been fully tested and merged.

I'm currently setting up a jenkins job on https://devops-ci.elastic.co/ to get the 5.x and 6.x branches automatically tested for each pull request. The next thing I'll be working on is making sure that all tests pass properly for 5.x and 6.x. There are currently a few tests which have hardcoded versions in them.

[Crazybus]

Should this shell have failed_when: False? Looking at the below task correct java version selected which has ansible_os_family == 'RedHat' and java_full_path is defined makes it sound like this task should only run for RedHat families but also allow failures in the event that 1.8.0 isn't available.

When running the tests for Ubuntu 16.04 this fails as the only available path is /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java.

I would suggest something like:

- name: Get the installed java path
  shell: "update-alternatives --display java | grep '^/' | awk '{print $1}' | grep 1.8.0"
  register: java_full_path
  failed_when: False
  when: ansible_os_family == 'RedHat'

- name: correct java version selected
  alternatives:
    name: java
    path: "{{ java_full_path.stdout }}"
    link: /usr/bin/java
  when: ansible_os_family == 'RedHat' and java_full_path is defined

[shribigb]

This is fixed! I somehow missed a check for RedHat.

[cvladicgit]

I tried this on CentOS 7 to install version 6.1.2 and I am getting an error in elasticsearch-security.yml "Ensure x-pack conf directory exists (file)" task step. There are some areas where I have to include Become: yes and some with Become_User.

TASK [elasticsearch : Check if bootstrap password is set] ****************************************************************************************************
fatal: [esnode1]: FAILED! => {"changed": true, "cmd": ["/usr/share/elasticsearch/bin/elasticsearch-keystore", "list"], "delta": "0:00:01.361441", "end": "2018-01-23 09:13:37.358360", "failed": true, "msg": "non-zero return code", "rc": 65, "start": "2018-01-23 09:13:35.996919", "stderr": "", "stderr_lines": [], "stdout": "ERROR: Elasticsearch keystore not found. Use 'create' command to create one.", "stdout_lines": ["ERROR: Elasticsearch keystore not found. Use 'create' command to create one."]}

[Crazybus]

Hi @shribigb

Thanks for the great work here! I have created a new pull request #412 based on your work. I rebased your pull request against master (with the updated .kitchen.yml for testing 5.x and 6.x) and I'm now fixing up all of the tests.

Cheers,

Michael