Skip to content
This repository has been archived by the owner on Feb 22, 2022. It is now read-only.

[stable/redis-ha] Added HAProxy to support exposed Redis environments. #15305

Conversation

DandyDeveloper
Copy link
Collaborator

What this PR does / why we need it:

There's a fundamental problem that numerous people are experiencing with this chart and being accessed externally.

This PR aims to provide an added proxy that performs Redis / Sentinel healthchecks to establish the master and proxy to that master without the client needing to infer the master IP from Sentinel.

It seems to be working, the only problem is, the chart currently means Sentinel occasionally is reporting s_down and forcing the master into a "READ-ONLY" state. I will raise this separately.

Which issue this PR fixes

Special notes for your reviewer:

@ssalaues If you don't mind, I'd like to be a contributor to this, and help you maintain for the future.

Checklist

[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

  • DCO signed
  • Chart Version bumped
  • Variables are documented in the README.md
  • Title of the PR starts with chart name (e.g. [stable/chart])

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
@helm-bot helm-bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Jul 8, 2019
@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jul 8, 2019
@k8s-ci-robot
Copy link
Contributor

Hi @DandyDeveloper. Thanks for your PR.

I'm waiting for a helm member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
…github.com:DandyDeveloper/charts into stable/redis-ha/add-proxy-support-external-clients

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
@DandyDeveloper DandyDeveloper force-pushed the stable/redis-ha/add-proxy-support-external-clients branch 2 times, most recently from 5aa4a7f to 69c7658 Compare July 8, 2019 02:24
@helm-bot helm-bot added the Contribution Allowed If the contributor has signed the DCO or the CNCF CLA (prior to the move to a DCO). label Jul 8, 2019
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
@helm-bot helm-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 8, 2019
@DandyDeveloper
Copy link
Collaborator Author

/assign @ssalaues

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
@ssalaues
Copy link
Collaborator

ssalaues commented Jul 8, 2019

@DandyDeveloper This looks great and should alleviate a lot of the headaches people report with trying to contact the master directly. I'll test this out on my cluster as well so I can better understand the behavior.

@ssalaues If you don't mind, I'd like to be a contributor to this, and help you maintain for the future.

Yeah definitely add yourself as a contributor. I would greatly appreciate the help and you seem like you have an active interest in this chart as well.

This PR aims to provide an added proxy that performs Redis / Sentinel healthchecks to establish the master and proxy to that master without the client needing to infer the master IP from Sentinel

So before I took over maintenance of this chart, it was setup with a "master" service. It worked with the idea was that the pods themselves had RBAC permissions to update their own labels so that anytime a new master was elected, the correct pod would be updated with the correct Kubernetes label. This way the service and the selector always pointed to the master server. However the implementation was very buggy and caused more problems than it solved which is why I removed it for the sake of reliability of the service. This is something that has been on my plate to reliably reimplement as a feature but simply haven't had enough time to see it through.

With that said, my first impression is that while I strongly agree with the motivation behind this PR, it seems that adding an HAproxy deployment to the chart seems excessive when this could be possible in a very Kubernetes native way.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
@helm-bot helm-bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Aug 27, 2019
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
@helm-bot helm-bot removed the Contribution Allowed If the contributor has signed the DCO or the CNCF CLA (prior to the move to a DCO). label Aug 27, 2019
Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
@DandyDeveloper DandyDeveloper force-pushed the stable/redis-ha/add-proxy-support-external-clients branch from 6c216a1 to 970f009 Compare August 27, 2019 02:10
@helm-bot helm-bot added the Contribution Allowed If the contributor has signed the DCO or the CNCF CLA (prior to the move to a DCO). label Aug 27, 2019
@ssalaues
Copy link
Collaborator

/lgtm

@k8s-ci-robot k8s-ci-robot added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Aug 27, 2019
@DandyDeveloper
Copy link
Collaborator Author

/verify-owners

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
@k8s-ci-robot k8s-ci-robot removed lgtm Indicates that a PR is ready to be merged. do-not-merge/invalid-owners-file Indicates that a PR should not merge because it has an invalid OWNERS file in it. labels Aug 28, 2019
@JanMatas
Copy link
Contributor

Hi, thanks for great PR. I encountered the problem you are solving so I tried using this even before it gets merged. I had a problem that nodeSelector is applied to HAProxy deployment but tolerations are not, so the HAProxy pods were not schedulable in my cluster. At least for me, the expected behaviour would be to include tolerations in HAProxy deployment spec as well.

@ssalaues
Copy link
Collaborator

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Aug 28, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: DandyDeveloper, ssalaues

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ssalaues
Copy link
Collaborator

@DandyDeveloper @JanMatas I'm approving this since this PR is in a stable state and has been open for quite some time now. But the issue you bring up should be a fairly easy minor fix.

@Deepak-Routray
Copy link

@DandyDeveloper Thanks for getting this merged and for resolving the issue. Great Work!!

kengou pushed a commit to kengou/charts that referenced this pull request Sep 18, 2019
helm#15305)

* Added HAProxy Templating / Support for LB

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Updated some nodeSelector values

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Added better message to config-init script

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Cleanup / Added to README and reverted defaultS

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Bump chart version

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Added HAProxy Templating / Support for LB

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Updated some nodeSelector values

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Added better message to config-init script

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Cleanup / Added to README and reverted defaultS

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Bump chart version

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Removed previous attempt at separate chart

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Removed trailing spaces

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Fixed selectors. Working perfectly!

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Fixed deployment spec

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Fixing review comments.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Adding all AUTH in HAProxy conf.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Added tcp-check expect to auth check.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Updating scope within range.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Changed var to  in range

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Added Helm Test/CI Tests, updated existing test

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Fixing linting issues

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Fixing review comments

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Adding livenessprobe. Removing bad test.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Merging with master

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Upgraded HAProxy version. Fix Segfaulting

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Upgrading chart version

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Adding myself to redis-ha reviewers/approvers

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Removed myself from reviews. Need to be part of helm org.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Removing myself from contrib list until helm approved

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
ramkumarvs pushed a commit to yugabyte/charts-helm-fork that referenced this pull request Sep 30, 2019
helm#15305)

* Added HAProxy Templating / Support for LB

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Updated some nodeSelector values

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Added better message to config-init script

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Cleanup / Added to README and reverted defaultS

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Bump chart version

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Added HAProxy Templating / Support for LB

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Updated some nodeSelector values

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Added better message to config-init script

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Cleanup / Added to README and reverted defaultS

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Bump chart version

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Removed previous attempt at separate chart

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Removed trailing spaces

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Fixed selectors. Working perfectly!

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Fixed deployment spec

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Fixing review comments.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Adding all AUTH in HAProxy conf.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Added tcp-check expect to auth check.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Updating scope within range.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Changed var to  in range

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Added Helm Test/CI Tests, updated existing test

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Fixing linting issues

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Fixing review comments

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Adding livenessprobe. Removing bad test.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Merging with master

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Upgraded HAProxy version. Fix Segfaulting

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Upgrading chart version

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Adding myself to redis-ha reviewers/approvers

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Removed myself from reviews. Need to be part of helm org.

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>

* Removing myself from contrib list until helm approved

Signed-off-by: Aaron Layfield <aaron.layfield@gmail.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. Contribution Allowed If the contributor has signed the DCO or the CNCF CLA (prior to the move to a DCO). lgtm Indicates that a PR is ready to be merged. ok-to-test size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet