Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to configure Redis Sentinel or Redis Cluster with official Helm, standalone Redis works fine #543

Open
2fst4u opened this issue Mar 10, 2024 · 13 comments
Labels
caching anything to do with redis or opcache help wanted Extra attention is needed needs info Not enough information provided

Comments

@2fst4u
Copy link

2fst4u commented Mar 10, 2024

Describe your Issue

Using the official Helm chart I am able to configure Redis in standalone mode with the following values:

values.yaml
values:
    image:
      repository: nextcloud
      pullPolicy: IfNotPresent
    replicaCount: 1

    ingress:
      enabled: false

    phpClientHttpsFix:
      enabled: false
      protocol: https

    nextcloud:
      host: nextcloud.domain.com
      existingSecret:
        enabled: true
        secretName: nextcloud-secret
        usernameKey: adminusername
        passwordKey: adminpassword
        smtpUsernameKey: smtp_username
        smtpPasswordKey: smtp_password
        smtpHostKey: smtp_host
      update: 0
      containerPort: 80
      datadir: /var/www/html/data
      persistence:
        subPath:
      mail:
        enabled: true
        fromAddress: nextcloud
        domain: gmail.com
        smtp:
          host: smtp.gmail.com
          secure: ssl
          port: 465
          authtype: LOGIN
      defaultConfigs:
        .htaccess: true
        redis.config.php: false
        apache-pretty-urls.config.php: true
        apcu.config.php: true
        apps.config.php: true
        autoconfig.php: true
        smtp.config.php: true

      configs:

        custom.config.php: |-
          <?php
          $CONFIG = array (
            'overwritehost' => 'nextcloud.domain.com',
            'overwriteprotocol' => 'https',
            'overwrite.cli.url' => 'https://nextcloud.domain.com',
            'filelocking.enabled' => 'true',
            'loglevel' => '0',
            'enable_previews' => true,
            'trusted_domains' =>
              [
                'nextcloud',
                'nextcloud.domain.com'
              ],
            'trusted_proxies' => ['10.0.0.0/8'],
            'default_phone_region' => 'nz',
          );

        redis.config.php: |-
          <?php
          $CONFIG = array (
            'memcache.distributed' => '\OC\Memcache\Redis',
            'memcache.locking' => '\OC\Memcache\Redis',
            'redis' => array(
              'host' => 'redis-master',
              'port' => 6379,
              'password' => getenv('REDIS_HOST_PASSWORD'),
            )
          );

      extraEnv:
      - name: REDIS_HOST_PASSWORD
        valueFrom:
          secretKeyRef:
            name: redis
            key: redis-password

      extraVolumes:
       - name: shared
         nfs:
          server: storage
          path: /mnt/Volume01/Shared
      extraVolumeMounts:
       - name: shared
         mountPath: "/shared"

    nginx:
      enabled: false
    
    internalDatabase:
      enabled: false

    externalDatabase:
      enabled: true

      type: postgresql

      host: postgres-service:5432

      database: postgres

      existingSecret:
        enabled: true
        secretName: nextcloud-secret
        usernameKey: db-username
        passwordKey: db-password

    mariadb:
      enabled: false

    postgresql:
      enabled: false

    redis:
      enabled: false

    cronjob:
      enabled: true
      resources:
        limits:
          memory: 100Mi
        requests:
         cpu: 50m
         memory: 100Mi

    service:
      type: ClusterIP
      port: 80

    persistence:
      enabled: true
      storageClass: "rook-cephfs"
      accessMode: ReadWriteMany
      size: 8Gi
      nextcloudData:
        enabled: true
        subPath:
        annotations: {}
        storageClass: "managed-nfs-storage"
        accessMode: ReadWriteMany
        size: 10Gi

    resources: 
      limits:
        memory: 500Mi
      requests:
       cpu: 500m
       memory: 500Mi

    livenessProbe:
      enabled: true
      initialDelaySeconds: 10
      periodSeconds: 20
      timeoutSeconds: 5
      failureThreshold: 3
      successThreshold: 1
    readinessProbe:
      enabled: true
      initialDelaySeconds: 10
      periodSeconds: 10
      timeoutSeconds: 5
      failureThreshold: 3
      successThreshold: 1
    startupProbe:
      enabled: true
      initialDelaySeconds: 30
      periodSeconds: 10
      timeoutSeconds: 5
      failureThreshold: 600
      successThreshold: 1

    hpa:
      enabled: false
      cputhreshold: 60
      minPods: 1
      maxPods: 10

    affinity: 
      nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 50
            preference:
              matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                - amd64
      podAntiAffinity:                                 
        requiredDuringSchedulingIgnoredDuringExecution:
        - topologyKey: kubernetes.io/hostname     
          labelSelector:                               
            matchLabels:                               
              app.kubernetes.io/name: nextcloud

    metrics:
      enabled: false

When when I setup Redis with sentinel using the Bitnami Redis helm chart and the following values in the redis.config.php section:

        redis.config.php: |-
          <?php
          $CONFIG = array (
            'memcache.distributed' => '\OC\Memcache\Redis',
            'memcache.locking' => '\OC\Memcache\Redis',
            'redis' => array(
              'host' => 'redis',
              'port' => 26379,
              'password' => getenv('REDIS_HOST_PASSWORD'),
            )
          );

(only host and port changes)

I get the following errors in the logs and I am usually unable to see any files in the webUI, or unable to login:

click me for logs
{
    "reqId": "u8DnzkBqQMKciPMlK8MJ",
    "level": 0,
    "time": "2023-12-26T07:51:16+00:00",
    "remoteAddr": "192.168.50.191",
    "user": "username",
    "app": "webdav",
    "method": "GET",
    "url": "/remote.php/dav/files/username/Shared%20Pictures/Unedited/DSC_5844.JPG",
    "message": "\"Shared Pictures/Unedited/DSC_5844.JPG\" is locked, existing lock on file: none",
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0",
    "version": "28.0.1.1",
    "exception": {
        "Exception": "OCA\\DAV\\Connector\\Sabre\\Exception\\FileLocked",
        "Message": "\"Shared Pictures/Unedited/DSC_5844.JPG\" is locked, existing lock on file: none",
        "Code": 0,
        "Trace": [
            {
                "file": "/var/www/html/apps/dav/lib/Connector/Sabre/File.php",
                "line": 501,
                "function": "convertToSabreException",
                "class": "OCA\\DAV\\Connector\\Sabre\\File",
                "type": "->",
                "args": [
                    [
                        "OCP\\Lock\\LockedException"
                    ]
                ]
            },
            {
                "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
                "line": 85,
                "function": "get",
                "class": "OCA\\DAV\\Connector\\Sabre\\File",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
                "line": 89,
                "function": "httpGet",
                "class": "Sabre\\DAV\\CorePlugin",
                "type": "->",
                "args": [
                    [
                        "Sabre\\HTTP\\Request"
                    ],
                    [
                        "Sabre\\HTTP\\Response"
                    ]
                ]
            },
            {
                "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                "line": 472,
                "function": "emit",
                "class": "Sabre\\DAV\\Server",
                "type": "->",
                "args": [
                    "method:GET",
                    [
                        [
                            "Sabre\\HTTP\\Request"
                        ],
                        [
                            "Sabre\\HTTP\\Response"
                        ]
                    ]
                ]
            },
            {
                "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                "line": 253,
                "function": "invokeMethod",
                "class": "Sabre\\DAV\\Server",
                "type": "->",
                "args": [
                    [
                        "Sabre\\HTTP\\Request"
                    ],
                    [
                        "Sabre\\HTTP\\Response"
                    ]
                ]
            },
            {
                "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                "line": 321,
                "function": "start",
                "class": "Sabre\\DAV\\Server",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/apps/dav/lib/Server.php",
                "line": 370,
                "function": "exec",
                "class": "Sabre\\DAV\\Server",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/apps/dav/appinfo/v2/remote.php",
                "line": 35,
                "function": "exec",
                "class": "OCA\\DAV\\Server",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/remote.php",
                "line": 172,
                "args": [
                    "/var/www/html/apps/dav/appinfo/v2/remote.php"
                ],
                "function": "require_once"
            }
        ],
        "File": "/var/www/html/apps/dav/lib/Connector/Sabre/File.php",
        "Line": 752,
        "Previous": {
            "Exception": "OCP\\Lock\\LockedException",
            "Message": "\"Shared Pictures/Unedited/DSC_5844.JPG\" is locked, existing lock on file: none",
            "Code": 0,
            "Trace": [
                {
                    "file": "/var/www/html/lib/private/Files/View.php",
                    "line": 2048,
                    "function": "lockPath",
                    "class": "OC\\Files\\View",
                    "type": "->",
                    "args": [
                        "/Shared Pictures/Unedited/DSC_5844.JPG",
                        1,
                        false
                    ]
                },
                {
                    "file": "/var/www/html/lib/private/Files/View.php",
                    "line": 1141,
                    "function": "lockFile",
                    "class": "OC\\Files\\View",
                    "type": "->",
                    "args": [
                        "/Shared Pictures/Unedited/DSC_5844.JPG",
                        1
                    ]
                },
                {
                    "file": "/var/www/html/lib/private/Files/View.php",
                    "line": 987,
                    "function": "basicOperation",
                    "class": "OC\\Files\\View",
                    "type": "->",
                    "args": [
                        "fopen",
                        "/Shared Pictures/Unedited/DSC_5844.JPG",
                        [
                            "read"
                        ],
                        "r"
                    ]
                },
                {
                    "file": "/var/www/html/apps/dav/lib/Connector/Sabre/File.php",
                    "line": 499,
                    "function": "fopen",
                    "class": "OC\\Files\\View",
                    "type": "->",
                    "args": [
                        "Shared Pictures/Unedited/DSC_5844.JPG",
                        "r"
                    ]
                },
                {
                    "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
                    "line": 85,
                    "function": "get",
                    "class": "OCA\\DAV\\Connector\\Sabre\\File",
                    "type": "->",
                    "args": []
                },
                {
                    "file": "/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
                    "line": 89,
                    "function": "httpGet",
                    "class": "Sabre\\DAV\\CorePlugin",
                    "type": "->",
                    "args": [
                        [
                            "Sabre\\HTTP\\Request"
                        ],
                        [
                            "Sabre\\HTTP\\Response"
                        ]
                    ]
                },
                {
                    "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                    "line": 472,
                    "function": "emit",
                    "class": "Sabre\\DAV\\Server",
                    "type": "->",
                    "args": [
                        "method:GET",
                        [
                            [
                                "Sabre\\HTTP\\Request"
                            ],
                            [
                                "Sabre\\HTTP\\Response"
                            ]
                        ]
                    ]
                },
                {
                    "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                    "line": 253,
                    "function": "invokeMethod",
                    "class": "Sabre\\DAV\\Server",
                    "type": "->",
                    "args": [
                        [
                            "Sabre\\HTTP\\Request"
                        ],
                        [
                            "Sabre\\HTTP\\Response"
                        ]
                    ]
                },
                {
                    "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                    "line": 321,
                    "function": "start",
                    "class": "Sabre\\DAV\\Server",
                    "type": "->",
                    "args": []
                },
                {
                    "file": "/var/www/html/apps/dav/lib/Server.php",
                    "line": 370,
                    "function": "exec",
                    "class": "Sabre\\DAV\\Server",
                    "type": "->",
                    "args": []
                },
                {
                    "file": "/var/www/html/apps/dav/appinfo/v2/remote.php",
                    "line": 35,
                    "function": "exec",
                    "class": "OCA\\DAV\\Server",
                    "type": "->",
                    "args": []
                },
                {
                    "file": "/var/www/html/remote.php",
                    "line": 172,
                    "args": [
                        "/var/www/html/apps/dav/appinfo/v2/remote.php"
                    ],
                    "function": "require_once"
                }
            ],
            "File": "/var/www/html/lib/private/Files/View.php",
            "Line": 1943,
            "Previous": {
                "Exception": "OCP\\Lock\\LockedException",
                "Message": "\"files/39b1b1dd24f88308dadb938d2a7907e9\"(\"Unedited/DSC_5844.JPG\") is locked, existing lock on file: none",
                "Code": 0,
                "Trace": [
                    {
                        "file": "/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php",
                        "line": 607,
                        "function": "acquireLock",
                        "class": "OC\\Files\\Storage\\Common",
                        "type": "->",
                        "args": [
                            "Unedited/DSC_5844.JPG",
                            1,
                            [
                                "OC\\Lock\\MemcacheLockingProvider"
                            ]
                        ]
                    },
                    {
                        "file": "/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php",
                        "line": 607,
                        "function": "acquireLock",
                        "class": "OC\\Files\\Storage\\Wrapper\\Wrapper",
                        "type": "->",
                        "args": [
                            "Unedited/DSC_5844.JPG",
                            1,
                            [
                                "OC\\Lock\\MemcacheLockingProvider"
                            ]
                        ]
                    },
                    {
                        "file": "/var/www/html/lib/private/Files/View.php",
                        "line": 1935,
                        "function": "acquireLock",
                        "class": "OC\\Files\\Storage\\Wrapper\\Wrapper",
                        "type": "->",
                        "args": [
                            "Unedited/DSC_5844.JPG",
                            1,
                            [
                                "OC\\Lock\\MemcacheLockingProvider"
                            ]
                        ]
                    },
                    {
                        "file": "/var/www/html/lib/private/Files/View.php",
                        "line": 2048,
                        "function": "lockPath",
                        "class": "OC\\Files\\View",
                        "type": "->",
                        "args": [
                            "/Shared Pictures/Unedited/DSC_5844.JPG",
                            1,
                            false
                        ]
                    },
                    {
                        "file": "/var/www/html/lib/private/Files/View.php",
                        "line": 1141,
                        "function": "lockFile",
                        "class": "OC\\Files\\View",
                        "type": "->",
                        "args": [
                            "/Shared Pictures/Unedited/DSC_5844.JPG",
                            1
                        ]
                    },
                    {
                        "file": "/var/www/html/lib/private/Files/View.php",
                        "line": 987,
                        "function": "basicOperation",
                        "class": "OC\\Files\\View",
                        "type": "->",
                        "args": [
                            "fopen",
                            "/Shared Pictures/Unedited/DSC_5844.JPG",
                            [
                                "read"
                            ],
                            "r"
                        ]
                    },
                    {
                        "file": "/var/www/html/apps/dav/lib/Connector/Sabre/File.php",
                        "line": 499,
                        "function": "fopen",
                        "class": "OC\\Files\\View",
                        "type": "->",
                        "args": [
                            "Shared Pictures/Unedited/DSC_5844.JPG",
                            "r"
                        ]
                    },
                    {
                        "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
                        "line": 85,
                        "function": "get",
                        "class": "OCA\\DAV\\Connector\\Sabre\\File",
                        "type": "->",
                        "args": []
                    },
                    {
                        "file": "/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
                        "line": 89,
                        "function": "httpGet",
                        "class": "Sabre\\DAV\\CorePlugin",
                        "type": "->",
                        "args": [
                            [
                                "Sabre\\HTTP\\Request"
                            ],
                            [
                                "Sabre\\HTTP\\Response"
                            ]
                        ]
                    },
                    {
                        "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                        "line": 472,
                        "function": "emit",
                        "class": "Sabre\\DAV\\Server",
                        "type": "->",
                        "args": [
                            "method:GET",
                            [
                                [
                                    "Sabre\\HTTP\\Request"
                                ],
                                [
                                    "Sabre\\HTTP\\Response"
                                ]
                            ]
                        ]
                    },
                    {
                        "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                        "line": 253,
                        "function": "invokeMethod",
                        "class": "Sabre\\DAV\\Server",
                        "type": "->",
                        "args": [
                            [
                                "Sabre\\HTTP\\Request"
                            ],
                            [
                                "Sabre\\HTTP\\Response"
                            ]
                        ]
                    },
                    {
                        "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                        "line": 321,
                        "function": "start",
                        "class": "Sabre\\DAV\\Server",
                        "type": "->",
                        "args": []
                    },
                    {
                        "file": "/var/www/html/apps/dav/lib/Server.php",
                        "line": 370,
                        "function": "exec",
                        "class": "Sabre\\DAV\\Server",
                        "type": "->",
                        "args": []
                    },
                    {
                        "file": "/var/www/html/apps/dav/appinfo/v2/remote.php",
                        "line": 35,
                        "function": "exec",
                        "class": "OCA\\DAV\\Server",
                        "type": "->",
                        "args": []
                    },
                    {
                        "file": "/var/www/html/remote.php",
                        "line": 172,
                        "args": [
                            "/var/www/html/apps/dav/appinfo/v2/remote.php"
                        ],
                        "function": "require_once"
                    }
                ],
                "File": "/var/www/html/lib/private/Files/Storage/Common.php",
                "Line": 767,
                "Previous": {
                    "Exception": "OCP\\Lock\\LockedException",
                    "Message": "\"files/39b1b1dd24f88308dadb938d2a7907e9\"(\"local::/shared/Pictures/::Unedited/DSC_5844.JPG\") is locked, existing lock on file: none",
                    "Code": 0,
                    "Trace": [
                        {
                            "file": "/var/www/html/lib/private/Files/Storage/Common.php",
                            "line": 765,
                            "function": "acquireLock",
                            "class": "OC\\Lock\\MemcacheLockingProvider",
                            "type": "->",
                            "args": [
                                "files/39b1b1dd24f88308dadb938d2a7907e9",
                                1,
                                "local::/shared/Pictures/::Unedited/DSC_5844.JPG"
                            ]
                        },
                        {
                            "file": "/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php",
                            "line": 607,
                            "function": "acquireLock",
                            "class": "OC\\Files\\Storage\\Common",
                            "type": "->",
                            "args": [
                                "Unedited/DSC_5844.JPG",
                                1,
                                [
                                    "OC\\Lock\\MemcacheLockingProvider"
                                ]
                            ]
                        },
                        {
                            "file": "/var/www/html/lib/private/Files/Storage/Wrapper/Wrapper.php",
                            "line": 607,
                            "function": "acquireLock",
                            "class": "OC\\Files\\Storage\\Wrapper\\Wrapper",
                            "type": "->",
                            "args": [
                                "Unedited/DSC_5844.JPG",
                                1,
                                [
                                    "OC\\Lock\\MemcacheLockingProvider"
                                ]
                            ]
                        },
                        {
                            "file": "/var/www/html/lib/private/Files/View.php",
                            "line": 1935,
                            "function": "acquireLock",
                            "class": "OC\\Files\\Storage\\Wrapper\\Wrapper",
                            "type": "->",
                            "args": [
                                "Unedited/DSC_5844.JPG",
                                1,
                                [
                                    "OC\\Lock\\MemcacheLockingProvider"
                                ]
                            ]
                        },
                        {
                            "file": "/var/www/html/lib/private/Files/View.php",
                            "line": 2048,
                            "function": "lockPath",
                            "class": "OC\\Files\\View",
                            "type": "->",
                            "args": [
                                "/Shared Pictures/Unedited/DSC_5844.JPG",
                                1,
                                false
                            ]
                        },
                        {
                            "file": "/var/www/html/lib/private/Files/View.php",
                            "line": 1141,
                            "function": "lockFile",
                            "class": "OC\\Files\\View",
                            "type": "->",
                            "args": [
                                "/Shared Pictures/Unedited/DSC_5844.JPG",
                                1
                            ]
                        },
                        {
                            "file": "/var/www/html/lib/private/Files/View.php",
                            "line": 987,
                            "function": "basicOperation",
                            "class": "OC\\Files\\View",
                            "type": "->",
                            "args": [
                                "fopen",
                                "/Shared Pictures/Unedited/DSC_5844.JPG",
                                [
                                    "read"
                                ],
                                "r"
                            ]
                        },
                        {
                            "file": "/var/www/html/apps/dav/lib/Connector/Sabre/File.php",
                            "line": 499,
                            "function": "fopen",
                            "class": "OC\\Files\\View",
                            "type": "->",
                            "args": [
                                "Shared Pictures/Unedited/DSC_5844.JPG",
                                "r"
                            ]
                        },
                        {
                            "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/CorePlugin.php",
                            "line": 85,
                            "function": "get",
                            "class": "OCA\\DAV\\Connector\\Sabre\\File",
                            "type": "->",
                            "args": []
                        },
                        {
                            "file": "/var/www/html/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
                            "line": 89,
                            "function": "httpGet",
                            "class": "Sabre\\DAV\\CorePlugin",
                            "type": "->",
                            "args": [
                                [
                                    "Sabre\\HTTP\\Request"
                                ],
                                [
                                    "Sabre\\HTTP\\Response"
                                ]
                            ]
                        },
                        {
                            "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                            "line": 472,
                            "function": "emit",
                            "class": "Sabre\\DAV\\Server",
                            "type": "->",
                            "args": [
                                "method:GET",
                                [
                                    [
                                        "Sabre\\HTTP\\Request"
                                    ],
                                    [
                                        "Sabre\\HTTP\\Response"
                                    ]
                                ]
                            ]
                        },
                        {
                            "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                            "line": 253,
                            "function": "invokeMethod",
                            "class": "Sabre\\DAV\\Server",
                            "type": "->",
                            "args": [
                                [
                                    "Sabre\\HTTP\\Request"
                                ],
                                [
                                    "Sabre\\HTTP\\Response"
                                ]
                            ]
                        },
                        {
                            "file": "/var/www/html/3rdparty/sabre/dav/lib/DAV/Server.php",
                            "line": 321,
                            "function": "start",
                            "class": "Sabre\\DAV\\Server",
                            "type": "->",
                            "args": []
                        },
                        {
                            "file": "/var/www/html/apps/dav/lib/Server.php",
                            "line": 370,
                            "function": "exec",
                            "class": "Sabre\\DAV\\Server",
                            "type": "->",
                            "args": []
                        },
                        {
                            "file": "/var/www/html/apps/dav/appinfo/v2/remote.php",
                            "line": 35,
                            "function": "exec",
                            "class": "OCA\\DAV\\Server",
                            "type": "->",
                            "args": []
                        },
                        {
                            "file": "/var/www/html/remote.php",
                            "line": 172,
                            "args": [
                                "/var/www/html/apps/dav/appinfo/v2/remote.php"
                            ],
                            "function": "require_once"
                        }
                    ],
                    "File": "/var/www/html/lib/private/Lock/MemcacheLockingProvider.php",
                    "Line": 62
                }
            }
        },
        "message": "\"Shared Pictures/Unedited/DSC_5844.JPG\" is locked, existing lock on file: none",
        "exception": {},
        "CustomMessage": "\"Shared Pictures/Unedited/DSC_5844.JPG\" is locked, existing lock on file: none"
    }
}

And when I spin up bitnami's redis cluster chart and change the redis.config.php to the following:

        redis.config.php: |-
          <?php
          $CONFIG = array (
            'memcache.distributed' => '\OC\Memcache\Redis',
            'memcache.locking' => '\OC\Memcache\Redis',
            'redis.cluster' => array(
              'seeds' => array(
                'redis-cluster-headless:6379'
              ),
              'password' => getenv('REDIS_HOST_PASSWORD'),
            ),
          );

(I have aslo tried redis-cluster:6379 for the seed, I get the same result)

I get the following errors in the logs and an internal system error on loading the webui:

click me for more logs
{
    "reqId": "XN0dcqwWLpLXe7Dt2RId",
    "level": 3,
    "time": "2023-12-26T07:58:56+00:00",
    "remoteAddr": "192.168.50.191",
    "user": "--",
    "app": "core",
    "method": "GET",
    "url": "/favicon.ico",
    "message": "Exception thrown: RedisClusterException",
    "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:121.0) Gecko/20100101 Firefox/121.0",
    "version": "28.0.1.1",
    "exception": {
        "Exception": "RedisClusterException",
        "Message": "Couldn't map cluster keyspace using any provided seed",
        "Code": 0,
        "Trace": [
            {
                "file": "/var/www/html/lib/private/RedisFactory.php",
                "line": 88,
                "function": "__construct",
                "class": "RedisCluster",
                "type": "->",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/html/lib/private/RedisFactory.php",
                "line": 158,
                "function": "create",
                "class": "OC\\RedisFactory",
                "type": "->",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/html/lib/private/Memcache/Redis.php",
                "line": 66,
                "function": "getInstance",
                "class": "OC\\RedisFactory",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/lib/private/Memcache/Redis.php",
                "line": 72,
                "function": "getCache",
                "class": "OC\\Memcache\\Redis",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/lib/private/URLGenerator.php",
                "line": 203,
                "function": "get",
                "class": "OC\\Memcache\\Redis",
                "type": "->",
                "args": [
                    "core-favicon.ico"
                ]
            },
            {
                "file": "/var/www/html/lib/private/legacy/template/functions.php",
                "line": 249,
                "function": "imagePath",
                "class": "OC\\URLGenerator",
                "type": "->",
                "args": [
                    "core",
                    "favicon.ico"
                ]
            },
            {
                "file": "/var/www/html/core/templates/layout.guest.php",
                "line": 20,
                "function": "image_path",
                "args": [
                    "core",
                    "favicon.ico"
                ]
            },
            {
                "file": "/var/www/html/lib/private/Template/Base.php",
                "line": 180,
                "args": [
                    "/var/www/html/core/templates/layout.guest.php"
                ],
                "function": "include"
            },
            {
                "file": "/var/www/html/lib/private/Template/Base.php",
                "line": 150,
                "function": "load",
                "class": "OC\\Template\\Base",
                "type": "->",
                "args": [
                    "/var/www/html/core/templates/layout.guest.php",
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/html/lib/private/legacy/OC_Template.php",
                "line": 139,
                "function": "fetchPage",
                "class": "OC\\Template\\Base",
                "type": "->",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/html/lib/private/legacy/OC_Template.php",
                "line": 170,
                "function": "fetchPage",
                "class": "OC_Template",
                "type": "->",
                "args": [
                    "*** sensitive parameters replaced ***"
                ]
            },
            {
                "file": "/var/www/html/lib/private/Template/Base.php",
                "line": 132,
                "function": "fetchPage",
                "class": "OC_Template",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/lib/private/legacy/OC_Template.php",
                "line": 291,
                "function": "printPage",
                "class": "OC\\Template\\Base",
                "type": "->",
                "args": []
            },
            {
                "file": "/var/www/html/index.php",
                "line": 104,
                "function": "printExceptionErrorPage",
                "class": "OC_Template",
                "type": "::",
                "args": [
                    [
                        "RedisClusterException"
                    ],
                    500
                ]
            }
        ],
        "File": "/var/www/html/lib/private/RedisFactory.php",
        "Line": 88,
        "CustomMessage": "Exception thrown: RedisClusterException"
    }
}

Indicating it can't see any of the redis nodes at all.

Logs and Errors

As above

Describe your Environment

  • K3s on Ubuntu

  • Helm Version (or App that manages helm): k3s and fluxio

  • Helm Chart Version: 4.latest

  • values.yaml:

Included above

@jessebot jessebot added the caching anything to do with redis or opcache label Jun 9, 2024
@jessebot
Copy link
Collaborator

jessebot commented Jun 9, 2024

Sorry for the delay! I edited your description to make it a little easier to read by adding collapsibles and syntax highlighting.

If you're trying to use redis cluster vs stand alone, the seeds param you specified should be correct according to docs.nextcloud.com/server/29:

          <?php
          $CONFIG = array (
            'memcache.distributed' => '\OC\Memcache\Redis',
            'memcache.locking' => '\OC\Memcache\Redis',
            'redis.cluster' => array(
              'seeds' => array(
                'redis-cluster-headless:6379'
              ),
              'password' => getenv('REDIS_HOST_PASSWORD'),
            ),
          );

I don't use redis cluster, but if I'm understanding correctly, the seed should be the name of the redis service right? - which should be whatever this is: https://github.com/bitnami/charts/blob/1a5338daf534b712823a407b409f7b71b9ddb294/bitnami/redis-cluster/templates/redis-svc.yaml#L9 - which should be whatever your helm release name is for the redis cluster helm chart release, or the fullnameOverride parameter value for that chart.

Some minor follow up questions:

  1. Are you doing a fresh install of nextcloud in between testing different redis installations? I ask because I noticed you have persistence enabled in your values.yaml. If not, can you try a fresh install without persistence while also using a fresh install of redis-cluster?

  2. Can you verify the name of the redis service when using bitnami's redis-cluster chart?

If others in the community have better information or suggestions here, please feel free to chime in, as I haven't been using redis lately.

@jessebot jessebot added help wanted Extra attention is needed needs info Not enough information provided labels Jun 9, 2024
@2fst4u
Copy link
Author

2fst4u commented Jun 9, 2024 via email

@jessebot
Copy link
Collaborator

@2fst4u that's totally fine. I'll be testing out Valkey cluster later this coming weekend, and at that time, it should be the same as Redis Cluster, and I'll be able verify what's up with that and hopefully can come back to this issue to give my notes on everything. Sorry again for the huge delay 🙏

@pfaelzerchen
Copy link

Is it possible that nextcloud does not support redis sentinel? The documentation is not very clear at this point, but sentinel is not explicitly mentioned. The sentinel mode works a bit different from standalone or cluster. Reads could be sent to any node on the port 6379. But for writes one has to ask a sentinel first which node is actually the current master on port 26379. None of the examples covers this.

When I set redis.sentinel.enabled: true in the nextcloud chart to change the bitnami subchart into sentinel-mode the nextcloud pod comes up, but will answer with HTTP 503. Setting it back to enabled: false will make nextcloud run nicely again.

@2fst4u
Copy link
Author

2fst4u commented Nov 27, 2024 via email

@pfaelzerchen
Copy link

Nextcloud supports sentinel because I've set it up with standalone pods which I manually configured before.

Can you give me your nextcloud configuration that worked with sentinel? I'm happy to try to get it working with the bitnami sentinel deployment.

How many sentinel nodes did you deploy?

@2fst4u
Copy link
Author

2fst4u commented Nov 27, 2024 via email

@pfaelzerchen
Copy link

Found it. The sentinel references were removed from the docs because sentinel is not supported: nextcloud/server#35075

@2fst4u
Copy link
Author

2fst4u commented Nov 27, 2024 via email

@2fst4u
Copy link
Author

2fst4u commented Dec 16, 2024

Just coming back to this, this doesn't explain why redis cluster doesn't work. I think that still needs resolving.

@jessebot
Copy link
Collaborator

@provokateurin is redis cluster supported nextcloud server? I haven't tried with either redis cluster or valkey cluster yet.

@2fst4u
Copy link
Author

2fst4u commented Dec 17, 2024

It's pretty extensively documented so it would be even more frustrating if it wasn't.

https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/config_sample_php_parameters.html#memory-caching-backend-configuration

@jessebot
Copy link
Collaborator

OK good, then since it's supported in the nextcloud server repo, this chart should support it as well. This is a community maintained chart, so that's up to the community to test it. I unfortunately still haven't had the time to test redis-cluster or valkey-cluster though. :(

I guess a good start would be adding redis-cluster and maybe also valkey-cluster as subcharts to this chart and creating some basic tests for them to run in ci, if you or someone else wants to take a stab at that, we can start hammering out the exact details of how it works and make the helm chart do the bulk of the work of populating the appropriate configs. Even if we got a PR that was partly working, it would be nice to start somewhere, and as long as edits by maintainer are turned on, we can iterate on it async.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
caching anything to do with redis or opcache help wanted Extra attention is needed needs info Not enough information provided
Projects
None yet
Development

No branches or pull requests

3 participants