New module postgresql_copy: copy data between a file and a table

[Andersson007]

SUMMARY

New module postgresql_copy: copy data between a file and a table.

Implements COPY PostgreSQL command.

ISSUE TYPE

• New Module Pull Request

SPECIFIC OPTIONS

• copy_to: Copy the contents of a table to a file. Can also copy the results of a SELECT query. Mutually exclusive with copy_from and dst.
• copy_from: Copy data from a file to a table (appending the data to whatever is in the table already). Mutually exclusive with copy_to and src.
• src: Copy data from copy_from to src=tablename. Used with copy_to only.
• dst: Copy data to dst=tablename from copy_from=/path/to/data.file. Used with copy_from only.
• columns: List of column names for the src/dst table to COPY FROM/TO.
• program: (bool) Mark src/dst as a program. Data will be copied to/from a program.
• options: (dict) Options of COPY command.

RETURN

queries:
  description: List of executed queries.
  returned: always
  type: str
  sample: [ "COPY test_table FROM '/tmp/data_file.txt' (FORMAT csv, DELIMITER ',', NULL 'NULL')" ]
src:
  description: Data source.
  returned: always
  type: str
  sample: "mytable"
dst:
  description: Data destination.
  returned: always
  type: str
  sample: "/tmp/data.csv"

EXAMPLES

- name: Copy text TAB-separated data from file /tmp/data.txt to acme table
  postgresql_copy:
    copy_from: /tmp/data.txt
    dst: acme

- name: Copy CSV (comma-separated) data from file /tmp/data.csv to columns id, name of table acme
  postgresql_copy:
    copy_from: /tmp/data.csv
    dst: acme
    columns: id,name
    options:
      format: csv

- name: >
    Copy text vertical-bar-separated data from file /tmp/data.txt to bar table.
    The NULL values are specified as N
  postgresql_copy:
    copy_from: /tmp/data.csv
    dst: bar
    options:
      delimiter: '|'
      null: 'N'

- name: Copy data from acme table to file /tmp/data.txt in text format, TAB-separated
  postgresql_copy:
    src: acme
    copy_to: /tmp/data.txt

- name: Copy data from SELECT query to/tmp/data.csv in CSV format
  postgresql_copy:
    src: 'SELECT * FROM acme'
    copy_to: /tmp/data.csv
    options:
      format: csv

- name: Copy CSV data from my_table to gzip
  postgresql_copy:
    src: my_table
    copy_to: 'gzip > /tmp/data.csv.gz'
    program: yes
    options:
      format: csv

- name: >
    Copy data from columns id, name of table bar to /tmp/data.txt.
    Output format is text, vertical-bar-separated, NULL as N
  postgresql_copy:
    src: bar
    columns:
    - id
    - name
    copy_to: /tmp/data.csv
    options:
      delimiter: '|'
      null: 'N'

[ansibot]

cc @Dorn- @amenonsen @andytom @jbscalia @kostiantyn-nemchenko @kustodian @matburt @nerzhul @sebasmannem @tcraxs
click here for bot help

[Andersson007]

ready_for_review

[ansibot]

@Dorn- @andytom @antoinell @archf @b6d @dschep @jbscalia @jensdepuydt @jscalia @kostiantyn-nemchenko @kustodian @matburt @nerzhul @sebasmannem @tcraxs @wrouesnel

As a maintainer of a module in the same namespace this new module has been submitted to, your vote counts for shipits. Please review this module and add shipit if you would like to see it merged.

click here for bot help

[tcraxs]

@Andersson007 nice module and really useful. I have some comments and one or two changes.
I also like your approaches with the mutually_exclusive checks (mental note to me, to use it in the future) and the query_fragments.append and .join(query_fragments). This is a way cleaner code to read. Good work!

[Andersson007]

@tcraxs , thank you for the great review!

[Andersson007]

@andytom , would be cool if you had a look at this

[sebasmannem]

Great job, @Andersson007 ...
I say,
shipit

[kostiantyn-nemchenko]

I have several objections to the current state of the module. Will try to provide a detailed feedback today.

[Andersson007]

@kostiantyn-nemchenko , great! waiting for this

[kostiantyn-nemchenko]

@Andersson007 thank you for this PR. I still haven't tested it manually but would be great if you take into account my current suggestions.

[Andersson007]

@kostiantyn-nemchenko , I envy your ability to concentrate ;) As always the great review

ready_for_review

[tcraxs]

Perfect review and good changes.
LGTM

[kostiantyn-nemchenko]

shipit

[kostiantyn-nemchenko]

bot_status

[ansibot]

Components

lib/ansible/modules/database/postgresql/postgresql_copy.py
support: community
maintainers: Andersson007 Dorn- amenonsen andytom jbscalia kostiantyn-nemchenko kustodian matburt nerzhul sebasmannem tcraxs

lib/ansible/plugins/doc_fragments/postgres.py
support: community
maintainers: Andersson007 Dorn- amenonsen andytom jbscalia kostiantyn-nemchenko kustodian matburt nerzhul sebasmannem tcraxs

test/integration/targets/postgresql/tasks/main.yml
support: community
maintainers: Andersson007 Dorn- amenonsen andytom jbscalia kostiantyn-nemchenko kustodian matburt nerzhul sebasmannem tcraxs

test/integration/targets/postgresql/tasks/postgresql_copy.yml
support: community
maintainers: Andersson007 Dorn- amenonsen andytom jbscalia kostiantyn-nemchenko kustodian matburt nerzhul sebasmannem tcraxs

Metadata

waiting_on: maintainer
changes_requested_by: null
needs_info: False
needs_revision: False
needs_rebase: False
merge_commits: []
too many files or commits: False
mergeable_state: clean
shippable_status: success
maintainer_shipits (module maintainers): 3
community_shipits (namespace maintainers): 0
ansible_shipits (core team members): 0
shipit_actors (maintainers or core team members): tcraxs kostiantyn-nemchenko Andersson007
shipit_actors_other: []
automerge: automerge new_module test failed

click here for bot help

[Andersson007]

Thanks all!