RabbitMQ publisher module #44718

Im0 · 2018-08-27T14:34:30Z

SUMMARY

Publish a message on a rabbitmq queue using a basic publisher

ISSUE TYPE

New Module Pull Request

COMPONENT NAME

notification/rabbitmq_publish.py

ANSIBLE VERSION

ansible 2.7.0.dev0 (rabbitmq_basic_publish d1167887f5) last updated 2018/08/28 00:20:25 (GMT +1100)
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/johni/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /home/johni/ansible/hacking/ansible/lib/ansible
  executable location = /home/johni/ansible/hacking/ansible/bin/ansible
  python version = 2.7.12 (default, Dec  4 2017, 14:50:18) [GCC 5.4.0 20160609]

ADDITIONAL INFORMATION

---
- hosts: localhost
  gather_facts: no

  tasks:
    - rabbitmq_publish:
        url: "amqp://guest:guest@192.168.0.32:5672/%2F"
        queue: 'hello'
        body: "Hello world from ansible module rabitmq_publish"
        content_type: "text/plain"
      register: output
      delegate_to: localhost

    - debug: 
        var: output

    - rabbitmq_publish:
        url: "amqp://guest:guest@192.168.0.32:5672/%2F"
        queue: 'hello'
        src: 'ajax-loader.gif'
      register: output2
      delegate_to: localhost

    - debug: 
        var: output2

Output:

$ ansible-playbook rabbit_test.yml

PLAY [localhost] *******************************************************************************************************************************************************************************************

TASK [rabbitmq_publish] ******************************************************************************************************************************************************************************
changed: [localhost -> localhost]

TASK [debug] ***********************************************************************************************************************************************************************************************
ok: [localhost] => {
    "output": {
        "changed": true, 
        "failed": false, 
        "msg": "Hello world from ansible module rabitmq_publish"
    }
}
......

ansibot · 2018-08-27T14:42:53Z

The test ansible-test sanity --test pylint [explain] failed with 2 errors:

lib/ansible/modules/notification/rabbitmq_basic_publish.py:118:7: singleton-comparison Comparison to None should be 'expr is None'
lib/ansible/modules/notification/rabbitmq_basic_publish.py:121:7: singleton-comparison Comparison to None should be 'expr is None'

The test ansible-test sanity --test pep8 [explain] failed with 5 errors:

lib/ansible/modules/notification/rabbitmq_basic_publish.py:118:17: E711 comparison to None should be 'if cond is None:'
lib/ansible/modules/notification/rabbitmq_basic_publish.py:121:20: E711 comparison to None should be 'if cond is None:'
lib/ansible/modules/notification/rabbitmq_basic_publish.py:147:26: E128 continuation line under-indented for visual indent
lib/ansible/modules/notification/rabbitmq_basic_publish.py:148:26: E128 continuation line under-indented for visual indent
lib/ansible/modules/notification/rabbitmq_basic_publish.py:149:26: E128 continuation line under-indented for visual indent

The test ansible-test sanity --test validate-modules [explain] failed with 4 errors:

lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E312 No RETURN provided
lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E325 argument_spec for "auto_delete" defines type="bool" but documentation does not
lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E325 argument_spec for "durable" defines type="bool" but documentation does not
lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E325 argument_spec for "exclusive" defines type="bool" but documentation does not

click here for bot help

ansibot · 2018-10-01T13:29:49Z

The test ansible-test sanity --test pylint [explain] failed with 3 errors:

lib/ansible/module_utils/rabbitmq.py:76:79: undefined-variable Undefined variable 'queue'
lib/ansible/module_utils/rabbitmq.py:92:30: undefined-variable Undefined variable 'AnsibleError'
lib/ansible/module_utils/rabbitmq.py:97:61: undefined-variable Undefined variable 'count'

The test ansible-test sanity --test validate-modules [explain] failed with 5 errors:

lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E312 No RETURN provided
lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E317 "queue" is marked as required but specifies a default. Arguments with a default should not be marked as required
lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E325 argument_spec for "auto_delete" defines type="bool" but documentation does not
lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E325 argument_spec for "durable" defines type="bool" but documentation does not
lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E325 argument_spec for "exclusive" defines type="bool" but documentation does not

click here for bot help

Im0 · 2018-10-02T12:55:31Z

Considering adding a "contents" list of dictionaries to rabbitmq_basic_publish to allow posting multiple items in one task. If specified, would need to account for different queues/routing for each message. (Won't do... user can loop over using with_items)

~~Look at removing 'required' for queue name. Pika/Rabbit should return a random queue name.~~

Look at adding a blocking subscriber to retrieve messages.

~~Fix up new argument checks. Eg. host/port/user/pass/vhost, need to check if they are all supplied if url is not supplied.~~

~~Consider how to add filename for binary uploads. (Added a filename header)~~

ansibot · 2018-10-02T13:06:35Z

The test ansible-test sanity --test pep8 [explain] failed with 1 error:

lib/ansible/modules/notification/rabbitmq_basic_publish.py:40:33: W291 trailing whitespace

The test ansible-test sanity --test validate-modules [explain] failed with 2 errors:

lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E312 No RETURN provided
lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E317 "queue" is marked as required but specifies a default. Arguments with a default should not be marked as required

click here for bot help

ansibot · 2018-10-08T12:18:44Z

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

lib/ansible/modules/notification/rabbitmq_basic_publish.py:0:0: E312 No RETURN provided

click here for bot help

lib/ansible/modules/notification/rabbitmq_basic_publish.py

Im0 · 2018-10-17T12:38:14Z

@Jorge-Rodriguez fyi re modifications to module_utils/rabbitmq.py.

ansibot · 2018-11-03T12:33:45Z

The test ansible-test sanity --test validate-modules [explain] failed with 1 error:

lib/ansible/modules/notification/rabbitmq_publish.py:0:0: E326 Value for "choices" from the argument_spec ([]) for "proto" does not match the documentation (['ampqs', 'ampq'])

click here for bot help

tonyseek · 2018-11-04T06:57:36Z

May we call it amqp_publish instead of rabbitmq_publish? The RabbitMQ is not the only implementation of AMQP broker.

Im0 · 2018-11-04T09:46:04Z

@tonyseek Thanks for the comment.

I'm not against the idea of changing the name. Just wondered, have you tested the module against another broker? Does it work fine? I've only tested against Rabbit, including all the integration test plans.

dch · 2018-11-05T18:58:43Z

@tonyseek given that pika is only an AMQP 0.9.1 client and that the current AMQP standard is 1.0 which most other brokers provide as either default or only option, I suspect this will cause confusion.

e.g. azure has AMQP (1.0), as does apache qpid, but their clients and services don't support AMQP 0.9.1.

https://www.rabbitmq.com/specification.html is a good explanation of the differences.

Unless somebody's actually confirmed this works, with a popular and widely deployed AMQP 0.9.1 compatible alternative broker, I'd stick firmly to calling it rabbitmq until demonstrated otherwise, or perhaps the middle ground amqp091 or something like it to allow a future amqp10 module in the same namespace.

Im0 · 2018-11-06T10:48:40Z

@dch @tonyseek - I was considering leaving the module name as rabbitmq_publish but call the module_utils file amqp.py and the class something like AMQP091Client... this way module_utils\amqp.py will have a space an AMQP10 implementation. Thoughts?

tonyseek · 2018-11-06T12:39:41Z

@dch Yes. They (AMQP 0.9 and AMQP 1.0) are different protocols. I think amqp09 is a well enough name personally.

@Im0 Maybe we don't need to worry about AMQP10 because it is another protocol. I think amqp09_publish and rabbitmq_publish are both nice names. I don't prefer one than another one.

I propose to consider using protocol name because I saw the letsencrypt_account module was renamed to acme_account. I am not sure whether it is a custom in ansible community to use spec name instead of implementation name.

Im0 · 2018-11-08T11:32:06Z

@pilou- @dch @mattclay - thanks for the reviews and feedback.
@pilou- - thanks to you especially for your (as usual) great constructive feedback. I think I covered your feedback and resolved them. If I've missed anything please let me know (when you have time).

Otherwise, I'm hoping you can give your support to shipit. Thanks.

pilou- · 2018-11-14T11:01:04Z

LGTM (I am not an RabbitMQ user).

dch · 2018-11-14T13:26:25Z

👏 nice work @Im0

willthames · 2018-11-15T01:07:22Z

Merged, thanks @Im0 !

* RabbitMQ basic publisher * Split out of a module_util. Preparing for binary posts. * Can now send a file to the queue. * Allowing an empty queue to be used so RabbitMQ returns a random queue. * Added RETURN docstring. * Updated and added tests. Now returns a dictionary with msg, content_type and queue published to. * Extra tests and introduced a none url method of providing server host details. * Added testing and errors for url/host parameters. * Updating RETURN sample * Added an image file for testing binary publishing. * Minor changes to test. * Added filename key/value to headers if a binary file is published. * Adding ability to specify headers. * Renaming to rabbitmq_publish * Changed tests to reflect name, and, preparing for testing headers. * Updated some documentation * Minor pip install update * Modifications after feedback. * Updates based on feedback. * Fixing pep8 issue. * Updating module and module_util name to amqp. * Reverting back to rabbitmq_publish naming. * Minor addition to notes.

ansibot added ci_verified Changes made in this PR are causing tests to fail. needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed community_review In order to be merged, this PR must follow the community review workflow. labels Aug 27, 2018

ryansb removed the needs_triage Needs a first human triage before being processed. label Aug 28, 2018

Im0 changed the title ~~RabbitMQ basic publisher~~ WIP: RabbitMQ basic publisher Aug 30, 2018

Im0 mentioned this pull request Aug 30, 2018

Lookup plugin for rabbitmq #44070

Merged

ansibot added the WIP This issue/PR is a work in progress. Nevertheless it was shared for getting input from peers. label Aug 30, 2018

ansibot added the stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. label Sep 7, 2018

ansibot removed ci_verified Changes made in this PR are causing tests to fail. stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. labels Oct 1, 2018

Im0 force-pushed the rabbitmq_basic_publish branch from 6b752de to 718ea5d Compare October 8, 2018 12:08

ansibot added the needs_rebase https://docs.ansible.com/ansible/devel/dev_guide/developing_rebasing.html label Oct 10, 2018

dch reviewed Oct 15, 2018

View reviewed changes

lib/ansible/modules/notification/rabbitmq_basic_publish.py Outdated Show resolved Hide resolved

ansibot added stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. and removed stale_ci This PR has been tested by CI more than one week ago. Close and re-open this PR to get it retested. labels Oct 16, 2018

Im0 force-pushed the rabbitmq_basic_publish branch from c8b04db to 515f643 Compare October 17, 2018 12:34

Im0 force-pushed the rabbitmq_basic_publish branch from 1237f7b to d0e270b Compare November 3, 2018 11:36

Modifications after feedback.

1ab19fb

ansibot added ci_verified Changes made in this PR are causing tests to fail. needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. and removed community_review In order to be merged, this PR must follow the community review workflow. labels Nov 3, 2018

Updates based on feedback.

8d40e05

ansibot removed the ci_verified Changes made in this PR are causing tests to fail. label Nov 4, 2018

Fixing pep8 issue.

7ca1cb3

ansibot added community_review In order to be merged, this PR must follow the community review workflow. and removed needs_revision This PR fails CI tests or a maintainer has requested a review/revision of the PR. labels Nov 4, 2018

Updating module and module_util name to amqp.

f2e7907

Im0 changed the title ~~RabbitMQ publisher module~~ AMQP publisher module (was RabbitMQ publisher module) Nov 5, 2018

Reverting back to rabbitmq_publish naming.

3d3a527

Im0 changed the title ~~AMQP publisher module (was RabbitMQ publisher module)~~ RabbitMQ publisher module Nov 8, 2018

Minor addition to notes.

f564892

willthames merged commit 54c54fc into ansible:devel Nov 15, 2018

dagwieers added the rabbitmq RabbitMQ community label Jan 28, 2019

ansible locked and limited conversation to collaborators Jul 22, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

RabbitMQ publisher module #44718

RabbitMQ publisher module #44718

Im0 commented Aug 27, 2018 •

edited

ansibot commented Aug 27, 2018

ansibot commented Oct 1, 2018

Im0 commented Oct 2, 2018 •

edited

ansibot commented Oct 2, 2018

ansibot commented Oct 8, 2018

Im0 commented Oct 17, 2018

ansibot commented Nov 3, 2018

tonyseek commented Nov 4, 2018

Im0 commented Nov 4, 2018 •

edited

dch commented Nov 5, 2018

Im0 commented Nov 6, 2018

tonyseek commented Nov 6, 2018

Im0 commented Nov 8, 2018

pilou- commented Nov 14, 2018

dch commented Nov 14, 2018

willthames commented Nov 15, 2018

RabbitMQ publisher module #44718

RabbitMQ publisher module #44718

Conversation

Im0 commented Aug 27, 2018 • edited

SUMMARY

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

ADDITIONAL INFORMATION

ansibot commented Aug 27, 2018

ansibot commented Oct 1, 2018

Im0 commented Oct 2, 2018 • edited

ansibot commented Oct 2, 2018

ansibot commented Oct 8, 2018

Im0 commented Oct 17, 2018

ansibot commented Nov 3, 2018

tonyseek commented Nov 4, 2018

Im0 commented Nov 4, 2018 • edited

dch commented Nov 5, 2018

Im0 commented Nov 6, 2018

tonyseek commented Nov 6, 2018

Im0 commented Nov 8, 2018

pilou- commented Nov 14, 2018

dch commented Nov 14, 2018

willthames commented Nov 15, 2018

Im0 commented Aug 27, 2018 •

edited

Im0 commented Oct 2, 2018 •

edited

Im0 commented Nov 4, 2018 •

edited