Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New module postgresql_copy: copy data between a file and a table #56835

Merged
merged 12 commits into from
Jun 3, 2019

Conversation

Andersson007
Copy link
Contributor

SUMMARY

New module postgresql_copy: copy data between a file and a table.

Implements COPY PostgreSQL command.

ISSUE TYPE
  • New Module Pull Request
SPECIFIC OPTIONS
  • copy_to: Copy the contents of a table to a file. Can also copy the results of a SELECT query. Mutually exclusive with copy_from and dst.
  • copy_from: Copy data from a file to a table (appending the data to whatever is in the table already). Mutually exclusive with copy_to and src.
  • src: Copy data from copy_from to src=tablename. Used with copy_to only.
  • dst: Copy data to dst=tablename from copy_from=/path/to/data.file. Used with copy_from only.
  • columns: List of column names for the src/dst table to COPY FROM/TO.
  • program: (bool) Mark src/dst as a program. Data will be copied to/from a program.
  • options: (dict) Options of COPY command.
RETURN
queries:
  description: List of executed queries.
  returned: always
  type: str
  sample: [ "COPY test_table FROM '/tmp/data_file.txt' (FORMAT csv, DELIMITER ',', NULL 'NULL')" ]
src:
  description: Data source.
  returned: always
  type: str
  sample: "mytable"
dst:
  description: Data destination.
  returned: always
  type: str
  sample: "/tmp/data.csv"
EXAMPLES
- name: Copy text TAB-separated data from file /tmp/data.txt to acme table
  postgresql_copy:
    copy_from: /tmp/data.txt
    dst: acme

- name: Copy CSV (comma-separated) data from file /tmp/data.csv to columns id, name of table acme
  postgresql_copy:
    copy_from: /tmp/data.csv
    dst: acme
    columns: id,name
    options:
      format: csv

- name: >
    Copy text vertical-bar-separated data from file /tmp/data.txt to bar table.
    The NULL values are specified as N
  postgresql_copy:
    copy_from: /tmp/data.csv
    dst: bar
    options:
      delimiter: '|'
      null: 'N'

- name: Copy data from acme table to file /tmp/data.txt in text format, TAB-separated
  postgresql_copy:
    src: acme
    copy_to: /tmp/data.txt

- name: Copy data from SELECT query to/tmp/data.csv in CSV format
  postgresql_copy:
    src: 'SELECT * FROM acme'
    copy_to: /tmp/data.csv
    options:
      format: csv

- name: Copy CSV data from my_table to gzip
  postgresql_copy:
    src: my_table
    copy_to: 'gzip > /tmp/data.csv.gz'
    program: yes
    options:
      format: csv

- name: >
    Copy data from columns id, name of table bar to /tmp/data.txt.
    Output format is text, vertical-bar-separated, NULL as N
  postgresql_copy:
    src: bar
    columns:
    - id
    - name
    copy_to: /tmp/data.csv
    options:
      delimiter: '|'
      null: 'N'

@ansibot ansibot added affects_2.9 This issue/PR affects Ansible v2.9 community_review In order to be merged, this PR must follow the community review workflow. database Database category module This issue/PR relates to a module. needs_triage Needs a first human triage before being processed. new_module This PR includes a new module. new_plugin This PR includes a new plugin. postgresql PostgreSQL community support:community This issue/PR relates to code supported by the Ansible community. test This PR relates to tests. labels May 23, 2019
@ansibot ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed community_review In order to be merged, this PR must follow the community review workflow. labels May 23, 2019
@Andersson007
Copy link
Contributor Author

ready_for_review

@ansibot
Copy link
Contributor

ansibot commented May 23, 2019

@Dorn- @andytom @antoinell @archf @b6d @dschep @jbscalia @jensdepuydt @jscalia @kostiantyn-nemchenko @kustodian @matburt @nerzhul @sebasmannem @tcraxs @wrouesnel

As a maintainer of a module in the same namespace this new module has been submitted to, your vote counts for shipits. Please review this module and add shipit if you would like to see it merged.

click here for bot help

@ansibot ansibot added community_review In order to be merged, this PR must follow the community review workflow. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels May 23, 2019
@tcraxs
Copy link
Contributor

tcraxs commented May 28, 2019

@Andersson007 nice module and really useful. I have some comments and one or two changes.
I also like your approaches with the mutually_exclusive checks (mental note to me, to use it in the future) and the query_fragments.append and .join(query_fragments). This is a way cleaner code to read. Good work!

@ansibot ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed community_review In order to be merged, this PR must follow the community review workflow. needs_triage Needs a first human triage before being processed. labels May 28, 2019
@Andersson007
Copy link
Contributor Author

Andersson007 commented May 29, 2019

@tcraxs , thank you for the great review!

@Andersson007
Copy link
Contributor Author

@andytom , would be cool if you had a look at this

@sebasmannem
Copy link
Contributor

Great job, @Andersson007 ...
I say,
shipit

@kostiantyn-nemchenko
Copy link
Contributor

I have several objections to the current state of the module. Will try to provide a detailed feedback today.

@Andersson007
Copy link
Contributor Author

@kostiantyn-nemchenko , great! waiting for this

Copy link
Contributor

@kostiantyn-nemchenko kostiantyn-nemchenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Andersson007 thank you for this PR. I still haven't tested it manually but would be great if you take into account my current suggestions.

@ansibot ansibot added needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed shipit This PR is ready to be merged by Core labels May 30, 2019
@Andersson007
Copy link
Contributor Author

Andersson007 commented May 31, 2019

@kostiantyn-nemchenko , I envy your ability to concentrate ;) As always the great review

ready_for_review

@ansibot ansibot added community_review In order to be merged, this PR must follow the community review workflow. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels May 31, 2019
Copy link
Contributor

@tcraxs tcraxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect review and good changes.
LGTM

@ansibot ansibot added shipit This PR is ready to be merged by Core and removed community_review In order to be merged, this PR must follow the community review workflow. labels May 31, 2019
@kostiantyn-nemchenko
Copy link
Contributor

shipit

@kostiantyn-nemchenko
Copy link
Contributor

bot_status

@ansibot
Copy link
Contributor

ansibot commented Jun 2, 2019

Components

lib/ansible/modules/database/postgresql/postgresql_copy.py
support: community
maintainers: Andersson007 Dorn- amenonsen andytom jbscalia kostiantyn-nemchenko kustodian matburt nerzhul sebasmannem tcraxs

lib/ansible/plugins/doc_fragments/postgres.py
support: community
maintainers: Andersson007 Dorn- amenonsen andytom jbscalia kostiantyn-nemchenko kustodian matburt nerzhul sebasmannem tcraxs

test/integration/targets/postgresql/tasks/main.yml
support: community
maintainers: Andersson007 Dorn- amenonsen andytom jbscalia kostiantyn-nemchenko kustodian matburt nerzhul sebasmannem tcraxs

test/integration/targets/postgresql/tasks/postgresql_copy.yml
support: community
maintainers: Andersson007 Dorn- amenonsen andytom jbscalia kostiantyn-nemchenko kustodian matburt nerzhul sebasmannem tcraxs

Metadata

waiting_on: maintainer
changes_requested_by: null
needs_info: False
needs_revision: False
needs_rebase: False
merge_commits: []
too many files or commits: False
mergeable_state: clean
shippable_status: success
maintainer_shipits (module maintainers): 3
community_shipits (namespace maintainers): 0
ansible_shipits (core team members): 0
shipit_actors (maintainers or core team members): tcraxs kostiantyn-nemchenko Andersson007
shipit_actors_other: []
automerge: automerge new_module test failed

click here for bot help

@willthames willthames merged commit 96bf243 into ansible:devel Jun 3, 2019
@Andersson007
Copy link
Contributor Author

Thanks all!

@ansible ansible locked and limited conversation to collaborators Aug 5, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
affects_2.9 This issue/PR affects Ansible v2.9 database Database category module This issue/PR relates to a module. new_module This PR includes a new module. new_plugin This PR includes a new plugin. postgresql PostgreSQL community shipit This PR is ready to be merged by Core support:community This issue/PR relates to code supported by the Ansible community. test This PR relates to tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants