Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable RabbitMQ mirrored queues #1226

Merged
merged 21 commits into from Sep 25, 2019

Conversation

@vincenzodnp
Copy link
Contributor

vincenzodnp commented Sep 5, 2019

Checklist

  • Affected Issues have been mentioned in the Closing issues section
  • Documentation has been written/updated.
  • Changelog entry has been written

Currently the RabbitMQ cluster is not set to use native HA mechanism called "Mirrored Queues".

It allows to create queues mirrored on 2 or more nodes, making the cluster fully High Availability.
In fact, at the moment, if the node (pod) where the queue is declared dies, queue's consumer are not able to use the it.

With mirrored queues, not only queue is still available for consumers but the mirror node is automatically elected new queue master (writable).

When the mirror node(s) is(are) online again, the queue is automatically synced all over the nodes defined in the policy.

Changelog Entry

Improvement - Set RabbitMQ policy to enable Mirrored Queue by default for all "lagoon-" queues (#1098 )

Closing issues

closes #1098

vincenzodnp added 2 commits Sep 5, 2019
(Mirrored Queues) queues.

Added logic to parse environment variables to set defult username,
password and patter for Mirrored Queues at container startup
- Set number of mirrors per queue to 2
- Modified broker lagoon template by adding the RABBITMQ_DEFAULT_HA_PATTER variable
@vincenzodnp vincenzodnp requested a review from Schnitzel Sep 5, 2019
images/rabbitmq/Dockerfile Outdated Show resolved Hide resolved
images/rabbitmq/Dockerfile Outdated Show resolved Hide resolved
services/broker/.lagoon.app.yml Outdated Show resolved Hide resolved
images/rabbitmq-cluster/cluster-rabbit.sh Outdated Show resolved Hide resolved
images/rabbitmq/definitions.json Outdated Show resolved Hide resolved
vincenzodnp added 4 commits Sep 6, 2019
Use of COPY for consistence in Dockerfile
Changed RABBITMQ_DEFAULT_HA_PATTERN default value
Added RABBITMQ_DEFAULT_VHOST variable
Added envplate binary to substitute variables
Same ENTRYPOINT for rabbitmq and rabbitmq-cluster images
Defined ENV variables for Lagoon broker service
Copy link
Contributor Author

vincenzodnp left a comment

Pushed last commit to fix it.
Please review.
Thanks

@vincenzodnp vincenzodnp requested a review from Schnitzel Sep 6, 2019
images/rabbitmq/Dockerfile Outdated Show resolved Hide resolved
images/rabbitmq/Dockerfile Outdated Show resolved Hide resolved
services/broker/Dockerfile Outdated Show resolved Hide resolved
@vincenzodnp vincenzodnp requested a review from dasrecht Sep 6, 2019
images/rabbitmq/Dockerfile Outdated Show resolved Hide resolved
Copy link
Contributor Author

vincenzodnp left a comment

Ready for review

@vincenzodnp vincenzodnp requested a review from Schnitzel Sep 6, 2019
images/rabbitmq/definitions.json Show resolved Hide resolved
images/rabbitmq/Dockerfile Outdated Show resolved Hide resolved
images/rabbitmq/Dockerfile Outdated Show resolved Hide resolved
Schnitzel and others added 2 commits Sep 7, 2019
@vincenzodnp vincenzodnp requested a review from Schnitzel Sep 9, 2019
@vincenzodnp

This comment has been minimized.

Copy link
Contributor Author

vincenzodnp commented Sep 9, 2019

Added documentation about image

docs/using_lagoon/docker_images/rabbitmq.md Outdated Show resolved Hide resolved
docs/using_lagoon/docker_images/rabbitmq.md Outdated Show resolved Hide resolved
@vincenzodnp vincenzodnp requested a review from Schnitzel Sep 9, 2019
Schnitzel added 2 commits Sep 19, 2019
@Schnitzel

This comment has been minimized.

Copy link
Member

Schnitzel commented Sep 23, 2019

@vincenzodnp
the testing currently fails:

Step 1/13 : FROM ${IMAGE_REPO:-lagoon}/commons as commons
pull access denied for lagoon/commons, repository does not exist or may require 'docker login': denied: requested access to the resource is denied

Makefile:149: recipe for target 'build/rabbitmq' failed

make: *** [build/rabbitmq] Error 1

this is because it tries to load lagoon/commons, but the env variable IMAGE_REPO would overwrite this to something like jenkinslagoonpr122616.
We missed the ARG on top of the rabbitmq Dockerfile, see like https://github.com/amazeeio/lagoon/blob/master/images/nginx/Dockerfile#L1

Can you add that? thanks

@vincenzodnp

This comment has been minimized.

Copy link
Contributor Author

vincenzodnp commented Sep 23, 2019

@Schnitzel added the ARG IMAGE_REPO.
Can you review?
Thanks

Schnitzel added 3 commits Sep 24, 2019
@Schnitzel Schnitzel merged commit 31842ad into master Sep 25, 2019
1 check passed
1 check passed
continuous-integration/jenkins/pr-merge This commit looks good
Details
@Schnitzel Schnitzel added this to the v1.1.0 milestone Sep 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.