Add support for MySQL plugin authentication

[tomaszkiewicz]

SUMMARY

The change adds support for MySQL plugin authentication, eg. AWS RDS IAM Authentication plugin.

ISSUE TYPE

• Feature Pull Request

COMPONENT NAME

MySQL

ANSIBLE VERSION

ansible 2.6.2
  config file = /home/luktom/.ansible.cfg
  configured module search path = [u'/home/luktom/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python2.7/dist-packages/ansible
  executable location = /usr/local/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]

ADDITIONAL INFORMATION

[ansibot]

cc @Jmainguy
click here for bot help

[ansibot]

The test ansible-test sanity --test pep8 [explain] failed with 2 errors:

lib/ansible/modules/database/mysql/mysql_user.py:677:25: E128 continuation line under-indented for visual indent
lib/ansible/modules/database/mysql/mysql_user.py:680:25: E128 continuation line under-indented for visual indent

click here for bot help

[Andersson007]

I asked aws community

[Andersson007]

@tomaszkiewicz folks said it's possible, see test/integration/targets/rds_instance.
If you have any questions you can ask in #ansible-aws irc channel.

[abohne]

@Andersson007 do you mean tests similar to https://github.com/ansible/ansible/blob/devel/test/integration/targets/mysql_user/tasks/create_user.yml ?

Is the environment the CI executes tests capable of using mysql plugin auth? (as e.g. AWS RDS requires to enable a specific setting for IAM auth)

Maybe I'm mistaken, but I think you could use one of the standard auth plugins (even mysql_native_password) for the integration tests?

[abohne]

Any updates here?

[daraul]

I've been pulling my hair out wondering why I can do sudo mysql -u root, but not ansible -K -m mysql_user -a "user=root" all. Hopefully this gets merged in soon...

[gundalow]

@daraul @abohne have you both tested this PR locally, does it work for you?

[daraul]

I've been wondering how I could do that, @gundalow, but couldn't find anything in the docs. Point me in the right direction and I'll test this out ASAP

[gundalow]

@daraul Since this PR only modifies a single file download https://raw.githubusercontent.com/ansible/ansible/50a6ec33f44989b980c9e0563ea4278667f481b0/lib/ansible/modules/database/mysql/mysql_user.py into library/ directory which you will need to create

ie

site.yml
roles/
library/mysql_user.py

[daraul]

@gundalow just ran this in my playbook:

         - name: Setup database user
           become: yes
           mysql_user:
             user: "{{ database_user }}"
             password: "{{ database_password }}"
             priv: "{{ database_name }}.*:ALL"

and was given this error:

TASK [Setup database user] *******************************************************************************************************************************************************************
task path: /home/daraul/dev/gwi-core-laravel/site.yml:108
<3.226.47.77> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<3.226.47.77> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o ControlPath=/home/daraul/.ansible/cp/a4d893a731 3.226.47.77 '/bin/sh -c '"'"'echo ~ubuntu && sleep 0'"'"''
<3.226.47.77> (0, b'/home/ubuntu\n', b'')
<3.226.47.77> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<3.226.47.77> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o ControlPath=/home/daraul/.ansible/cp/a4d893a731 3.226.47.77 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1574103853.575425-175696853591014 `" && echo ansible-tmp-1574103853.575425-175696853591014="` echo /home/ubuntu/.ansible/tmp/ansible-tmp-1574103853.575425-175696853591014 `" ) && sleep 0'"'"''
<3.226.47.77> (0, b'ansible-tmp-1574103853.575425-175696853591014=/home/ubuntu/.ansible/tmp/ansible-tmp-1574103853.575425-175696853591014\n', b'')
Using module file /home/daraul/dev/gwi-core-laravel/library/mysql_user.py
<3.226.47.77> PUT /home/daraul/.ansible/tmp/ansible-local-15306qt9fl977/tmp_rq_1n0k TO /home/ubuntu/.ansible/tmp/ansible-tmp-1574103853.575425-175696853591014/AnsiballZ_mysql_user.py
<3.226.47.77> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o ControlPath=/home/daraul/.ansible/cp/a4d893a731 '[3.226.47.77]'
<3.226.47.77> (0, b'sftp> put /home/daraul/.ansible/tmp/ansible-local-15306qt9fl977/tmp_rq_1n0k /home/ubuntu/.ansible/tmp/ansible-tmp-1574103853.575425-175696853591014/AnsiballZ_mysql_user.py\n', b'')
<3.226.47.77> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<3.226.47.77> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o ControlPath=/home/daraul/.ansible/cp/a4d893a731 3.226.47.77 '/bin/sh -c '"'"'chmod u+x /home/ubuntu/.ansible/tmp/ansible-tmp-1574103853.575425-175696853591014/ /home/ubuntu/.ansible/tmp/ansible-tmp-1574103853.575425-175696853591014/AnsiballZ_mysql_user.py && sleep 0'"'"''
<3.226.47.77> (0, b'', b'')
<3.226.47.77> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<3.226.47.77> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o ControlPath=/home/daraul/.ansible/cp/a4d893a731 -tt 3.226.47.77 '/bin/sh -c '"'"'sudo -H -S -n  -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-hxvcwzybqegfmasbftpojwwinhabguxl ; /usr/bin/python3 /home/ubuntu/.ansible/tmp/ansible-tmp-1574103853.575425-175696853591014/AnsiballZ_mysql_user.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation succeeded
<3.226.47.77> (1, b'\r\n{"msg": "unable to connect to database, check login_user and login_password are correct or /root/.my.cnf has the credentials. Exception message: (1698, \\"Access denied for user \'root\'@\'localhost\'\\")", "failed": true, "exception": "  File \\"/tmp/ansible_mysql_user_payload_oc31z0d7/__main__.py\\", line 691, in main\\n    connect_timeout=connect_timeout)\\n  File \\"/tmp/ansible_mysql_user_payload_oc31z0d7/ansible_mysql_user_payload.zip/ansible/module_utils/mysql.py\\", line 79, in mysql_connect\\n    db_connection = mysql_driver.connect(**config)\\n  File \\"/usr/local/lib/python3.6/dist-packages/pymysql/__init__.py\\", line 94, in Connect\\n    return Connection(*args, **kwargs)\\n  File \\"/usr/local/lib/python3.6/dist-packages/pymysql/connections.py\\", line 325, in __init__\\n    self.connect()\\n  File \\"/usr/local/lib/python3.6/dist-packages/pymysql/connections.py\\", line 599, in connect\\n    self._request_authentication()\\n  File \\"/usr/local/lib/python3.6/dist-packages/pymysql/connections.py\\", line 861, in _request_authentication\\n    auth_packet = self._read_packet()\\n  File \\"/usr/local/lib/python3.6/dist-packages/pymysql/connections.py\\", line 684, in _read_packet\\n    packet.check_error()\\n  File \\"/usr/local/lib/python3.6/dist-packages/pymysql/protocol.py\\", line 220, in check_error\\n    err.raise_mysql_exception(self._data)\\n  File \\"/usr/local/lib/python3.6/dist-packages/pymysql/err.py\\", line 109, in raise_mysql_exception\\n    raise errorclass(errno, errval)\\n", "invocation": {"module_args": {"user": "laravel", "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER", "priv": "laravel.*:ALL", "login_host": "localhost", "login_port": 3306, "encrypted": false, "host": "localhost", "host_all": false, "state": "present", "append_privs": false, "check_implicit_admin": false, "update_password": "always", "connect_timeout": 30, "config_file": "/root/.my.cnf", "sql_log_bin": true, "login_user": null, "login_password": null, "login_unix_socket": null, "plugin": null, "plugin_hash_string": null, "plugin_auth_string": null, "client_cert": null, "client_key": null, "ca_cert": null}}}\r\n', b'Shared connection to 3.226.47.77 closed.\r\n')
<3.226.47.77> Failed to connect to the host via ssh: Shared connection to 3.226.47.77 closed.
<3.226.47.77> ESTABLISH SSH CONNECTION FOR USER: ubuntu
<3.226.47.77> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="ubuntu"' -o ConnectTimeout=10 -o StrictHostKeyChecking=no -o ControlPath=/home/daraul/.ansible/cp/a4d893a731 3.226.47.77 '/bin/sh -c '"'"'rm -f -r /home/ubuntu/.ansible/tmp/ansible-tmp-1574103853.575425-175696853591014/ > /dev/null 2>&1 && sleep 0'"'"''
<3.226.47.77> (0, b'', b'')
The full traceback is:
  File "/tmp/ansible_mysql_user_payload_oc31z0d7/__main__.py", line 691, in main
    connect_timeout=connect_timeout)
  File "/tmp/ansible_mysql_user_payload_oc31z0d7/ansible_mysql_user_payload.zip/ansible/module_utils/mysql.py", line 79, in mysql_connect
    db_connection = mysql_driver.connect(**config)
  File "/usr/local/lib/python3.6/dist-packages/pymysql/__init__.py", line 94, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/pymysql/connections.py", line 325, in __init__
    self.connect()
  File "/usr/local/lib/python3.6/dist-packages/pymysql/connections.py", line 599, in connect
    self._request_authentication()
  File "/usr/local/lib/python3.6/dist-packages/pymysql/connections.py", line 861, in _request_authentication
    auth_packet = self._read_packet()
  File "/usr/local/lib/python3.6/dist-packages/pymysql/connections.py", line 684, in _read_packet
    packet.check_error()
  File "/usr/local/lib/python3.6/dist-packages/pymysql/protocol.py", line 220, in check_error
    err.raise_mysql_exception(self._data)
  File "/usr/local/lib/python3.6/dist-packages/pymysql/err.py", line 109, in raise_mysql_exception
    raise errorclass(errno, errval)

fatal: [3.226.47.77]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "append_privs": false,
            "ca_cert": null,
            "check_implicit_admin": false,
            "client_cert": null,
            "client_key": null,
            "config_file": "/root/.my.cnf",
            "connect_timeout": 30,
            "encrypted": false,
            "host": "localhost",
            "host_all": false,
            "login_host": "localhost",
            "login_password": null,
            "login_port": 3306,
            "login_unix_socket": null,
            "login_user": null,
            "password": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "plugin": null,
            "plugin_auth_string": null,
            "plugin_hash_string": null,
            "priv": "laravel.*:ALL",
            "sql_log_bin": true,
            "state": "present",
            "update_password": "always",
            "user": "laravel"
        }
    },
    "msg": "unable to connect to database, check login_user and login_password are correct or /root/.my.cnf has the credentials. Exception message: (1698, \"Access denied for user 'root'@'localhost'\")"
}

I can confirm that sudo mysql -u root ... works, because a previous task creates the database like so:

    - name: Setup database
      become: yes
      shell: >
          mysql -u {{ mysql_root_user }} -e 'CREATE DATABASE IF NOT EXISTS {{ database_name }};'

I'm not sure what the problem is. Maybe this custom module is conflicting with the official one? Is there any way to confirm that the custom module is being used? Can I just rename it to something like mysql_user_custom.py and use that instead?

---

Update: trying the same module, with plugin: auth_socket gave me the same error.

[gundalow]

From your debug
Using module file /home/daraul/dev/gwi-core-laravel/library/mysql_user.py So that means it's using the correct version.

@Andersson007 Any ideas?

[bmalynovytch]

@daraul I know this is annoying sometimes, but some client libraries convert implicitly localhost into use the unix socket instead of localhost over TCP.
Could you try configuring the path of the unix socket (login_unix_socket) in your ansible task instead of the login_host?

[daraul]

Setting login_unix_socket: /var/run/mysqld/mysqld.sock worked like a charm using both the existing and updated versions of this module, @bmalynovytch.

[Andersson007]

could you please put the empty lines between examples back

1. it is not related to the changes
2. it was much more better (when not one bunch of text)

[gundalow]

Could you please keep the blank lines between tasks, it helps readability.

[Andersson007]

considering that CI tests can be difficult to implement in this case, if there are a couple of people who tried it manually, IMO it might be merged after @gundalow 's requests are done

[abohne]

@gundalow @Andersson007 I've been using this for a while to manage users with the AWSAuthenticationPlugin on AWS RDS.

[Andersson007]

@abohne , thanks for the feedback!
will wait for the author (until the next week, then i rewrite it if there's no feedback)

[ansibot]

The test ansible-test sanity --test validate-modules [explain] failed with 4 errors:

lib/ansible/modules/database/mysql/mysql_user.py:0:0: parameter-type-not-in-doc: Argument 'plugin' in argument_spec defines type as 'str' but documentation doesn't define type
lib/ansible/modules/database/mysql/mysql_user.py:0:0: parameter-type-not-in-doc: Argument 'plugin_auth_string' in argument_spec defines type as 'str' but documentation doesn't define type
lib/ansible/modules/database/mysql/mysql_user.py:0:0: parameter-type-not-in-doc: Argument 'plugin_hash_string' in argument_spec defines type as 'str' but documentation doesn't define type
lib/ansible/modules/database/mysql/mysql_user.py:152:13: invalid-examples: EXAMPLES is not valid YAML

The test ansible-test sanity --test yamllint [explain] failed with 1 error:

lib/ansible/modules/database/mysql/mysql_user.py:152:13: error: EXAMPLES: syntax error: mapping values are not allowed here

click here for bot help

[20ton]

tested (with typo fixed):

- mysql_user:
    name: prometheus
    plugin: unix_socket
    priv: '*.*:PROCESS,REPLICATION CLIENT,SELECT'
    login_unix_socket: /run/mysqld/mysqld.sock

result:

MariaDB [(none)]> select Host,User,Password,Plugin from mysql.user;
+-----------+------------+-------------------------------------------+-------------+
| Host      | User       | Password                                  | Plugin      |
+-----------+------------+-------------------------------------------+-------------+
| localhost | root       |                                           | unix_socket |
| localhost | prometheus |                                           | unix_socket |
+-----------+------------+-------------------------------------------+-------------+
2 rows in set (0.001 sec)

[20ton]

plugin, not plugin_auth_string

[Andersson007]

Hey all!
I completed this one for you :) #65789
The code was copied. I only changed the doc strings a bit and functions argument strings formatting.
@20ton i fixed the typo there
@tomaszkiewicz @abohne @bmalynovytch @gundalow @20ton and others please look

[Andersson007]

@gundalow , could you please close this via via #65789