-
Notifications
You must be signed in to change notification settings - Fork 273
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unable to configure Redis Sentinel or Redis Cluster with official Helm, standalone Redis works fine #543
Comments
Sorry for the delay! I edited your description to make it a little easier to read by adding collapsibles and syntax highlighting. If you're trying to use redis cluster vs stand alone, the seeds param you specified should be correct according to docs.nextcloud.com/server/29: <?php
$CONFIG = array (
'memcache.distributed' => '\OC\Memcache\Redis',
'memcache.locking' => '\OC\Memcache\Redis',
'redis.cluster' => array(
'seeds' => array(
'redis-cluster-headless:6379'
),
'password' => getenv('REDIS_HOST_PASSWORD'),
),
); I don't use redis cluster, but if I'm understanding correctly, the seed should be the name of the redis service right? - which should be whatever this is: https://github.com/bitnami/charts/blob/1a5338daf534b712823a407b409f7b71b9ddb294/bitnami/redis-cluster/templates/redis-svc.yaml#L9 - which should be whatever your helm release name is for the redis cluster helm chart release, or the Some minor follow up questions:
If others in the community have better information or suggestions here, please feel free to chime in, as I haven't been using redis lately. |
To be honest I long gave up on this but in case we manage to figure it out:
1. Yep, deleting the persistent storage each time.
2. Confirmed the service name many a time.
…-------- Original Message --------
On 9/06/24 22:14, JesseBot wrote:
Sorry for the delay! I edited your description to make it a little easier to read by adding collapsibles and syntax highlighting.
If you're trying to use redis cluster vs stand alone, the seeds param you specified should be correct according to [docs.nextcloud.com/server/29](https://docs.nextcloud.com/server/29/developer_manual/how_to/index.html#redis-cluster):
<?php
$
CONFIG
=
array
(
'memcache.distributed'
=>
'\OC\Memcache\Redis'
,
'memcache.locking'
=>
'\OC\Memcache\Redis'
,
'redis.cluster'
=>
array
(
'seeds'
=>
array
(
'redis-cluster-headless:6379'
),
'password'
=> getenv(
'REDIS_HOST_PASSWORD'
),
),
);
I don't use redis cluster, but if I'm understanding correctly, the seed should be the name of the redis service right? - which should be whatever this is: https://github.com/bitnami/charts/blob/1a5338daf534b712823a407b409f7b71b9ddb294/bitnami/redis-cluster/templates/redis-svc.yaml#L9 - which should be whatever your helm release name is for the redis cluster helm chart release, or the fullnameOverride parameter value for that chart.
Some minor follow up questions:
-
Are you doing a fresh install of nextcloud in between testing different redis installations? I ask because I noticed you have persistence enabled in your values.yaml. If not, can you try a fresh install without persistence while also using a fresh install of redis-cluster?
-
Can you verify the name of the redis service when using bitnami's redis-cluster chart?
If others in the community have better information or suggestions here, please feel free to chime in, as I haven't been using redis lately.
—
Reply to this email directly, [view it on GitHub](#543 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ADUNCRQ2LX47VD6IECOLGD3ZGQTG3AVCNFSM6AAAAABEOZFGGCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNJWGQZDONRVGE).
You are receiving this because you authored the thread.Message ID: ***@***.***>
|
@2fst4u that's totally fine. I'll be testing out Valkey cluster later this coming weekend, and at that time, it should be the same as Redis Cluster, and I'll be able verify what's up with that and hopefully can come back to this issue to give my notes on everything. Sorry again for the huge delay 🙏 |
Is it possible that nextcloud does not support redis sentinel? The documentation is not very clear at this point, but sentinel is not explicitly mentioned. The sentinel mode works a bit different from standalone or cluster. Reads could be sent to any node on the port 6379. But for writes one has to ask a sentinel first which node is actually the current master on port 26379. None of the examples covers this. When I set |
Nextcloud supports sentinel because I've set it up with standalone pods which I manually configured before. Moving to helm I was trying to recreate that setup but I just couldn't find a way to get it working.
I'll probably just go back to standalone pods from docker images to do it.
…-------- Original Message --------
On 28/11/24 06:16, pfaelzerchen wrote:
Is it possible that nextcloud does not support redis sentinel? The documentation is not very clear at this point, but sentinel is not explicitly mentioned. The sentinel mode works a bit different from standalone or cluster. Reads could be sent to any node on the port 6379. But for writes one has to ask a sentinel first which node is actually the current master on port 26379. None of the examples covers this.
When I set redis.sentinel.enabled: true in the nextcloud chart to change the bitnami subchart into sentinel-mode the nextcloud pod comes up, but will answer with HTTP 503. Setting it back to enabled: false will make nextcloud run nicely again.
—
Reply to this email directly, [view it on GitHub](#543 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ADUNCRR4S3ZJVIS4N3VTK632CX5AVAVCNFSM6AAAAABEOZFGGCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBUGQYDGMRXG4).
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Can you give me your nextcloud configuration that worked with sentinel? I'm happy to try to get it working with the bitnami sentinel deployment. How many sentinel nodes did you deploy? |
Honestly it was years ago so no unfortunately. I don't have those Configs. But I do specifically remember the process of realising sentinel is on different ports and works differently etc before it was figured out. I'm pretty sure it got put into the documentation after that but I can't see it either so it might have been removed.
…-------- Original Message --------
On 28/11/24 06:31, pfaelzerchen wrote:
> Nextcloud supports sentinel because I've set it up with standalone pods which I manually configured before.
Can you give me your nextcloud configuration that worked with sentinel? I'm happy to try to get it working with the bitnami sentinel deployment.
How many sentinel nodes did you deploy?
—
Reply to this email directly, [view it on GitHub](#543 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ADUNCRU6TPWBJUJR7AZCXCD2CX6YPAVCNFSM6AAAAABEOZFGGCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBUGQZDSOJRHA).
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Found it. The sentinel references were removed from the docs because sentinel is not supported: nextcloud/server#35075 |
Well that's annoying, and frustrating that it worked and now doesn't. Support should have been added rather than references to it removed.
Also frustrating that no one from nextcloud was able to give a definitive answer on this issue explaining the reason.
Thanks I guess that clears it up.
…-------- Original Message --------
On 28/11/24 06:44, pfaelzerchen wrote:
Found it. The sentinel references were removed from the docs because sentinel is not supported: [nextcloud/server#35075](nextcloud/server#35075)
—
Reply to this email directly, [view it on GitHub](#543 (comment)), or [unsubscribe](https://github.com/notifications/unsubscribe-auth/ADUNCRWYDXIW6YVLLJUDAGD2CYAJPAVCNFSM6AAAAABEOZFGGCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMBUGQ2TEOBQGY).
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Just coming back to this, this doesn't explain why redis cluster doesn't work. I think that still needs resolving. |
@provokateurin is redis cluster supported nextcloud server? I haven't tried with either redis cluster or valkey cluster yet. |
It's pretty extensively documented so it would be even more frustrating if it wasn't. |
OK good, then since it's supported in the nextcloud server repo, this chart should support it as well. This is a community maintained chart, so that's up to the community to test it. I unfortunately still haven't had the time to test redis-cluster or valkey-cluster though. :( I guess a good start would be adding redis-cluster and maybe also valkey-cluster as subcharts to this chart and creating some basic tests for them to run in ci, if you or someone else wants to take a stab at that, we can start hammering out the exact details of how it works and make the helm chart do the bulk of the work of populating the appropriate configs. Even if we got a PR that was partly working, it would be nice to start somewhere, and as long as edits by maintainer are turned on, we can iterate on it async. |
Describe your Issue
Using the official Helm chart I am able to configure Redis in standalone mode with the following values:
values.yaml
When when I setup Redis with sentinel using the Bitnami Redis helm chart and the following values in the redis.config.php section:
(only host and port changes)
I get the following errors in the logs and I am usually unable to see any files in the webUI, or unable to login:
click me for logs
And when I spin up bitnami's redis cluster chart and change the redis.config.php to the following:
(I have aslo tried
redis-cluster:6379
for the seed, I get the same result)I get the following errors in the logs and an internal system error on loading the webui:
click me for more logs
Indicating it can't see any of the redis nodes at all.
Logs and Errors
As above
Describe your Environment
K3s on Ubuntu
Helm Version (or App that manages helm): k3s and fluxio
Helm Chart Version: 4.latest
values.yaml
:Included above
The text was updated successfully, but these errors were encountered: