add support for proxysql

[bmildren]

ISSUE TYPE

• New Module Pull Request

COMPONENT NAME

database/proxysql/proxysql_backend_servers.py
database/proxysql/proxysql_global_variables.py
database/proxysql/proxysql_manage_config.py
database/proxysql/proxysql_mysql_users.py
database/proxysql/proxysql_query_rules.py
database/proxysql/proxysql_replication_hostgroups.py
database/proxysql/proxysql_scheduler.py

ANSIBLE VERSION

ansible 2.1.0.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = Default w/o overrides

SUMMARY

This PR adds support for proxysql, a popular proxy solution when using MySQL.   More detail about proxysql can be found here: https://github.com/sysown/proxysql.

[gregdek]

Thanks @bmildren for this new module. When this module receives 'shipit' comments from two community members and any 'needs_revision' comments have been resolved, we will mark for inclusion.

[This message brought to you by your friendly Ansibull-bot.]

[renecannao]

shipit

Thanks

[roman-vynar]

🐑 it!

[bmildren]

I've produced a basic example below of how the modules can be used based on the "Mini HOW TO on ProxySQL Configuration" in the proxysql docs. The example supplies the default admin:admin creds, however a config file containing the username and password can also be used.

- name: proxysql | config | add server
  proxysql_backend_servers:
    login_user: "admin"
    login_password: "admin"
    hostgroup_id: 1
    hostname: "127.0.0.1"
    port: "{{ item }}"
    load_to_runtime: False
    state: present
  register: new_servers
  with_items:
    - 21891
    - 21892
    - 21893

- name: proxysql | config | manage monitor user
  proxysql_global_variables:
    login_user: "admin"
    login_password: "admin"
    variable: "mysql-monitor_username"
    value: "monitor"

- name: proxysql | config | manage monitor user password
  proxysql_global_variables:
    login_user: "admin"
    login_password: "admin"
    variable: "mysql-monitor_password"
    value: "monitor"

- name: proxysql | config | update monitor global variables
  proxysql_global_variables:
    login_user: "admin"
    login_password: "admin"
    variable: "{{ item }}"
    value: 2000
  with_items:
    - "mysql-monitor_connect_interval"
    - "mysql-monitor_ping_interval"
    - "mysql-monitor_read_only_interval"

- name: proxysql | config | load servers to runtime
  proxysql_manage_config:
    login_user: "admin"
    login_password: "admin"
    action: LOAD
    config_settings: "MYSQL SERVERS"
    direction: TO
    config_layer: RUNTIME
  when: new_servers.changed

- name: proxysql | config | add replication hostgroups
  proxysql_replication_hostgroups:
    login_user: "admin"
    login_password: "admin"
    writer_hostgroup: 1
    reader_hostgroup: 2
    state: present

- name: proxysql | config | add users
  proxysql_mysql_users:
    login_user: "admin"
    login_password: "admin"
    username: "{{ item.usr }}"
    password: "{{ item.pwd }}"
    default_hostgroup: 1
    state: present
  with_items:
   - { usr: "root", pwd: "" }
   - { usr: "msandbox", pwd: "msandbox" }

- name: proxysql | config | manage rules
  proxysql_query_rules:
    login_user: "admin"
    login_password: "admin"
    rule_id: 10
    active: True
    username: 'msandbox'
    match_digest: '^SELECT c FROM sbtest1 WHERE id=\?$'
    destination_hostgroup: 2
    cache_ttl: 5000
    apply: True
    load_to_runtime: False
    state: present
  register: rule1

- name: proxysql | config | manage rules
  proxysql_query_rules:
    login_user: "admin"
    login_password: "admin"
    rule_id: 20
    active: True
    username: 'msandbox'
    match_digest: 'DISTINCT c FROM sbtest1'
    destination_hostgroup: 2
    cache_ttl: 5000
    apply: True
    load_to_runtime: False
    state: present
  register: rule2

- name: proxysql | config | load servers to runtime
  proxysql_manage_config:
    login_user: "admin"
    login_password: "admin"
    action: LOAD
    config_settings: "MYSQL QUERY RULES"
    direction: TO
    config_layer: RUNTIME
  when: rule1.changed or rule2.changed

- name: proxysql | config | manage rules
  proxysql_query_rules:
    login_user: "admin"
    login_password: "admin"
    rule_id: 30
    active: True
    username: 'msandbox'
    match_pattern: 'DISTINCT(.*)ORDER BY c'
    replace_pattern: 'DISTINCT\1'
    apply: True
    state: present

- name: proxysql | config | fix rule
  proxysql_query_rules:
    login_user: "admin"
    login_password: "admin"
    rule_id: 20
    apply: False
    state: present

- name: proxysql | config | manage rules
  proxysql_query_rules:
    login_user: "admin"
    login_password: "admin"
    rule_id: 5
    active: True
    username: 'msandbox'
    match_pattern: '^SELECT (c) FROM sbtest(1) WHERE id=(1|2|3)(...)$'
    replace_pattern: 'SELECT c FROM sbtest2 WHERE id=\3\4'
    apply: True
    state: present

[akshaysuryawanshi]

Nice Ben. Ship it

[egon1024]

shipit

This is great - it will be handy.

[dveeden]

+1

[abadger]

Some comments.. they'll make better code but they don't strictly fix any errors:

• Would like to see the module_utils imports pushed up with the other imports and changed to not be wildcard imports (This has only been possible since 2.1 so older modules haven't caught up yet).
• We have a function that does the sys.exec_info dance.. I kinda want to use it instead of sys.exc_info directly so that it's easier to remove the compat code once we stop supporting python-2.4. using that is like:

from ansible.module_utils.pycompat24 import get_exception
[...]
except MySQLdb.Error:
    e = get_exception()

and one that may be more serious:

• Some of the sql strings are coming from the user. We may need to quote them to avoid sql injection. There's some code in module_utils that might help with that here: https://github.com/ansible/ansible/blob/devel/lib/ansible/module_utils/database.py

You can see it in action in code like this: https://github.com/ansible/ansible-modules-core/blob/devel/database/mysql/mysql_db.py#L116

[abadger]

needs_revision

[gregdek]

Thanks @bmildren for this PR. This PR requires revisions, either because it fails to build or by reviewer request. Please make the suggested revisions. When you are done, please comment with text 'ready_for_review' and we will put this PR back into review.

[This message brought to you by your friendly Ansibull-bot.]

[bmildren]

Thanks for the previous review! As suggested I've moved the module_utils imports up with the other imports, changed them to not be wildcard imports, and I've switched to using the get_exception function.

With regards to sql injection, I've reviewed the sql calls, and the use of cursor.execute(query_string, query_data) syntax escapes the strings so would mitigate against this, the only places this syntax isn't used is in proxysql_manage_config, however here all the user inputs all come from predefined choices.

Thanks!
Ben

[bmildren]

ready_for_review

[abadger]

@bmildren This was breaking documentation building so we had to revert it. Take a look at 6e4a182 for an example of how to clean up the module docs. You can run ansible-doc MODULE_NAME on each of the proxy_sql modules to check that the docs are parsing okay.

Once that's done resubmit and we can try again. (Feel free to ping me on irc abadger1999 in #ansible-devel on ic.freenode.net) if you need to get it looked at once you've resubmitted.