Sticky Session Cookie get's lost when unrelated configurations change #9728
Description
Welcome!
- Yes, I've searched similar issues on GitHub and didn't find any.
- Yes, I've searched similar issues on the Traefik community forum and didn't find any.
What did you do?
I run a Deployment of an application and a corresponding Service in K8s.
Traefik is used as load balancer for the cluster and is configured via the CRD IngressRoute.
This setup works and I can connect to my application via the configured domain.
Because of higher load it was neccessary to increase the number of instances of the application to two and use Sticky Sessions in Traefik, because the application is stateful.
At first it seams to work and a cookie with a shortened hash as name and hash as value is set and the application runs normally.
My expectation is, that this setup keeps working until I change something in the configuraiton of the IngressRoute.
What did you see instead?
After some time the sticky session cookie is not set anymore and therefore the application breaks because the sessions break.
I tried to increase the logs of Treafik to debugging but couldn't find any messages related to the sticky session cookie except, that it is set:
Sticky session cookie name: _68c5
After some observations of the changes the loss of the cookie seems to be related to the propagation of new settings after the change of unrelated deployments in the cluster. I see a Configuration received log message message in the logs. After that the cookie is gone. When I scaled down my application to one instance, to avoid the boken sessions. Therefore another configuration change is triggered and the cookies are back.
The first change was only the scaling of another application that is totaly unrelated to the application with the sticky sessions but is also configured with an IngressRoute.
I searched the internet, read the documentation and tried different settings like cookie names, http-only flag etc.
But I couldn't figure out, why the cookie is missing.
What version of Traefik are you using?
Version: 2.9.6
Codename: banon
Go version: go1.19.4
Built: 2022-12-07T14:17:58Z
OS/Arch: linux/amd64
What is your environment & configuration?
- Kubernetes v1.23.1
- Traefik v2.9.6 installed with Helm chart 20.8.0
- 2 instances of a webservice pod from a deployment and a service with port 8080
- a set of ingressRoutes with https redirect and the TLS configuration for the service
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: my-application-redirect-to-https
namespace: my-application-live
spec:
entryPoints:
- web
routes:
- kind: Rule
match: Host(`my-application.de`)
middlewares:
- name: http-to-https-redirect
services:
- kind: TraefikService
name: noop@internal
---
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
name: my-application
namespace: my-application
spec:
entryPoints:
- websecure
routes:
- kind: Rule
match: Host(`my-application.de`)
services:
- kind: Service
name: my-application-svc
namespace: my-application
passHostHeader: true
port: 8080
sticky:
cookie: {}
tls:
domains:
- main: my-application.de
secretName: tls-my-application-secret
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
name: tls-my-application-secret
namespace: my-application
spec:
dnsNames:
- my-application.de
duration: 2160h
isCA: false
issuerRef:
kind: ClusterIssuer
name: live-traefik-cert-manager
privateKey:
algorithm: RSA
encoding: PKCS1
size: 4096
renewBefore: 774h
secretName: tls-my-application-secret
subject:
organizations:
- my-company
usages:
- server auth
- client authIf applicable, please paste the log output in DEBUG level
Here is an excert from the "Configuration received log message". This part is always the same.
"my-application-545d74e73d4aefdff8cb": {
"entryPoints": [
"websecure"
],
"service": "my-application-545d74e73d4aefdff8cb",
"rule": "Host(`my-application.de`)",
"tls": {
"domains": [
{
"main": "my-application.de"
}
]
}
},
"my-application-redirect-to-https-545d74e73d4aefdff8cb": {
"entryPoints": [
"web"
],
"middlewares": [
"http-to-https-redirect"
],
"service": "noop@internal",
"rule": "Host(`my-application.de`)"
},
"my-application-545d74e73d4aefdff8cb": {
"loadBalancer": {
"sticky": {
"cookie": {}
},
"servers": [
{
"url": "http://10.101.5.99:8080"
},
{
"url": "http://10.101.6.159:8080"
}
],
"passHostHeader": true
}
}