Add postgresql_query module to database collection

[wrouesnel]

Note: resubmitted due to github not letting the branch be updated.

Issue Type:

• New Module Pull Request

Plugin Name:

postgresql_query

Ansible Version:

ansible 2.1.0 (devel 11f081cb99) last updated 2016/02/20 18:38:49 (GMT +1100)
  lib/ansible/modules/core: (fix_docker_submit f90e143521) last updated 2016/02/20 22:35:25 (GMT +1100)
  lib/ansible/modules/extras: (detached HEAD postgres_q) last updated 2016/03/03 23:15:21 (GMT +1100)
  config file = 
  configured module search path = Default w/o overrides

Summary:

This patch adds the postgresql_query module. This fills a gap in ansible functionality in allowing running queries as part of a playbook to retrieve values or deploy applications (e.g. setting authentication in gogs).

Example output:

The following is an example response from a select query against a database which contains JSON strings:

- postgresql_query:
      db: "TestDb"
      host: localhost
      user: testuser
      password: testpassword
      sslmode: prefer
      query: |
        SELECT * FROM testtable;

TASK [postgresql_query] ********************************************************
ok: [localhost] => {"changed": false, "query_result": [{"columnA": "valueA"},{"columnA": "valueB"}], "rowcount": 1}

This patch adds a postgresql query module. This allows running arbitrary queries
from ansible scripts with safe escaping of of the supplied values.

[wrouesnel]

Migrated from ansible/ansible-modules-extras#1802

[mattclay]

@wrouesnel CI failure due to python 2.4 syntax error:

2017-01-28 13:57:13 Compiling ./lib/ansible/modules/database/postgresql/postgresql_query.py ...
2017-01-28 13:57:13   File "./lib/ansible/modules/database/postgresql/postgresql_query.py", line 196
2017-01-28 13:57:13     except Exception as e:
2017-01-28 13:57:13                       ^
2017-01-28 13:57:13 SyntaxError: invalid syntax

Here's an example of how to handle exceptions that will work with python 2.4+:

https://github.com/ansible/ansible/blob/devel/lib/ansible/modules/files/file.py#L286

[ansibot]

The test ansible-test sanity --test pep8 failed with the following error:

lib/ansible/modules/database/postgresql/postgresql_query.py:170:11: E121 continuation line under-indented for hanging indent

The test ansible-test sanity --test validate-modules failed with the following error:

lib/ansible/modules/database/postgresql/postgresql_query.py:0:0: E316 ANSIBLE_METADATA.version: not a valid value for dictionary value @ data['version']. Got '0.1'

click here for bot help

[ansibot]

@b6d @dschep @jensdepuydt

As a maintainer of a module in the same namespace this new module has been submitted to, your vote counts for shipits. Please review this module and add shipit if you would like to see it merged.

click here for bot help

[ansibot]

The test ansible-test sanity --test ansible-doc --python 2.6 failed with the following error:

Command "ansible-doc postgresql_query" returned exit status 1.
>>> Standard Error
ERROR! module postgresql_query missing documentation (or could not parse documentation): 'metadata_version'

The test ansible-test sanity --test ansible-doc --python 2.7 failed with the following error:

Command "ansible-doc postgresql_query" returned exit status 1.
>>> Standard Error
ERROR! module postgresql_query missing documentation (or could not parse documentation): 'metadata_version'

The test ansible-test sanity --test ansible-doc --python 3.5 failed with the following error:

Command "ansible-doc postgresql_query" returned exit status 1.
>>> Standard Error
b"ERROR! module postgresql_query missing documentation (or could not parse documentation): 'metadata_version'"

The test ansible-test sanity --test ansible-doc --python 3.6 failed with the following error:

Command "ansible-doc postgresql_query" returned exit status 1.
>>> Standard Error
b"ERROR! module postgresql_query missing documentation (or could not parse documentation): 'metadata_version'"

The test ansible-test sanity --test validate-modules failed with the following errors:

lib/ansible/modules/database/postgresql/postgresql_query.py:0:0: E316 ANSIBLE_METADATA.metadata_version: required key not provided @ data['metadata_version']. Got None
lib/ansible/modules/database/postgresql/postgresql_query.py:0:0: E316 ANSIBLE_METADATA.version: extra keys not allowed @ data['version']. Got '1.0'

click here for bot help

[ansibot]

The test ansible-test sanity --test validate-modules failed with the following errors:

lib/ansible/modules/database/postgresql/postgresql_query.py:0:0: E307 version_added should be 2.4. Currently 2.3
lib/ansible/modules/database/postgresql/postgresql_query.py:0:0: E316 ANSIBLE_METADATA.metadata_version: required key not provided @ data['metadata_version']. Got None
lib/ansible/modules/database/postgresql/postgresql_query.py:0:0: E316 ANSIBLE_METADATA.version: extra keys not allowed @ data['version']. Got '1.0'

click here for bot help

[ansibot]

The test ansible-test sanity --test pep8 failed with the following error:

lib/ansible/modules/database/postgresql/postgresql_query.py:156:1: E302 expected 2 blank lines, found 1

click here for bot help

[wrouesnel]

@b6d @dschep @jensdepuydt

This module fills out an important gap in Ansible's database functionality when setting up applicaitons which use Postgres. It'd be really nice to get it merged.

[archf]

@wrouesnel: I happy to see new progress on this. I made #25411 initially just because there was no activity on this PR for a long time. We all have busy lives. The code you did was a nice starting point and I think you nailed the way module input should be.

In #25411, I added new features, mainly the ability to throw .sql script as input file to module. I also shared/used code to make it somewhat more inline with other postgresql modules. I think community would benefit from this. Let's not make it a race in between 2 PR. I proposed we work together on #25411 and we add your authorship to it. Would you take a look and see if it works?

Regards,

[wrouesnel]

@archf I'll take a look this week. So long as we wind up with good functionality I'm not fussed who's PR it is. This got caught up in me trying to get done traction on a bunch of old PRs this weekend.

[japsu]

I propose we support the use case of ensuring some row exists in the database, such as creating an initial user for an application. Something like this:

- postgresql_query:
    query: INSERT INTO users (username, password_hash) VALUES (…, …)
    only_if_not_exists: SELECT id FROM users WHERE username = 'joeadmin'

What should the "run this query first and only if it returns and empty set, run the actual query" parameter be called? only_if_not_exists? test? ok_when?

The semantics should be that the query reports ok (not changed) iff the only_if_not_exists query returns a non-empty set.

[maxamillion]

Is everyone happy with continuing the work on #25411? Can we close this one?

[maxamillion]

needs_info

[ansibot]

@wrouesnel This pullrequest is waiting for your response. Please respond or the pullrequest will be closed.

click here for bot help

[wrouesnel]

I'm happy sorry 25411 capturing the work. Closing this.