double-compresses upstream in some cases #1851
Description
Do you want to request a feature or report a bug?
Bug
What did you do?
I want to have Traefik proxy kubernetes-dashboard. The issue exists on both minikube and a kubernetes cluster. For reproduceability I'm going to use minikube as the target here.
First, deploy Traefik ingress controller. I'm using the stable/traefik helm chart with the local override to set the imageTag to 1.3.3 (latest as of writing):
$ cat values.yaml
imageTag: 1.3.3
$ helm install -f values.yaml stable/traefik --namespace kube-system
...
helm ls
NAME REVISION UPDATED STATUS CHART NAMESPACE
plucking-jellyfish 1 Mon Jul 10 13:53:29 2017 DEPLOYED traefik-1.3.0 kube-system
and check the service is deployed:
$ kubectl get svc -n kube-system
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kube-dns 10.0.0.10 <none> 53/UDP,53/TCP 31d
kubernetes-dashboard 10.0.0.202 <nodes> 80:30000/TCP 31d
plucking-jellyfish-traefik 10.0.0.164 <pending> 80:31586/TCP,443:32730/TCP 18m
tiller-deploy 10.0.0.237 <none> 44134/TCP 31d
Create an ingress for kubernetes-dashboard:
$ cat ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: kubernetes-dashboard
labels:
app: kubernetes-dashboard
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: k8s-dashboard.example.com
http:
paths:
- backend:
serviceName: kubernetes-dashboard
servicePort: 80
$ kubectl apply -f ingress.yaml -n kube-system
ingress "kubernetes-dashboard" configured
Find out the serviceable url of the traefik service in the cluster:
$ minikube -n kube-system service plucking-jellyfish-traefik --url
http://192.168.99.101:31586
http://192.168.99.101:32730
The first one is for HTTP and second HTTPS. For simplicity, let's use the HTTP endpoint.
$ export TRAEFIK=http://192.168.99.101:31586
$ curl -H"Host: k8s-dashboard.example.com" $TRAEFIK
<!doctype html> <html ng-app="kubernetesDashboard"> <head> <meta charset="utf-8"> <title ng-controller="kdTitle as $ctrl" ng-bind="$ctrl.title()"></title> <link rel="icon" type="image/png" href="assets/images/kubernetes-logo.png"> <meta name="viewport" content="width=device-width"> <link rel="stylesheet" href="static/vendor.4f4b705f.css"> <link rel="stylesheet" href="static/app.93b90a74.css"> </head> <body> <!--[if lt IE 10]>
<p class="browsehappy">You are using an <strong>outdated</strong> browser.
Please <a href="http://browsehappy.com/">upgrade your browser</a> to improve your
experience.</p>
<![endif]--> <kd-chrome layout="column" layout-fill> </kd-chrome> <script src="static/vendor.6952e31e.js"></script> <script src="api/appConfig.json"></script> <script src="static/app.8a6b8127.js"></script> </body> </html>
So far so good. We can also curl the static asset file shown at the end of the script.
$ curl -H"Host: k8s-dashboard.example.com" $TRAEFIK/static/vendor.6952e31e.js
... plain text javascript ...
However, most modern browsers send the request with Accept-Encoding header gzip,deflate. We can simulate this behaviour with curl -H"Accept-Encoding: gzip,deflate" ... --compressed:
What did you expect to see?
The plain text javascript file
What did you see instead?
Garbled data:
$ curl -H"Host: k8s-dashboard.example.com" -H"Accept-Encoding: gzip,deflate" $TRAEFIK/static/vendor.6952e31e.js --compressed
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 n{w۶ GT;93Cq5&! 5 E}HEx?{o}_˒rpw<rt<z\,^G-{9,yk8#S-
...
and if I pipe the output to gunzip I do get the plain text javascript:
curl -H"Host: k8s-dashboard.example.com" -H"Accept-Encoding: gzip,deflate" $TRAEFIK/static/vendor.6952e31e.js --compressed | gunzip | head
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0/** * @license AngularJS v1.6.3 * (c) 2010-2017 Google, Inc. http://angularjs.org * License: MIT */ !function(t){"use strict";function e(t,e){return e=e||Error,function(){var n=arguments[0],i=arguments[1],r="["+(t?t+":":"")+n+"] ",o=U(arguments,2).map(function(t){return ...
It appears Traefik is double-compressing the upstream response.
Output of traefik version: (What version of Traefik are you using?)
1.3.3
$ kubectl describe pod -n kube-system plucking-jellyfish-traefik-4111740596-wc0mr
...
Containers:
plucking-jellyfish-traefik:
Container ID: docker://f5b448eb076c4a4ded6900bf90b674746077e9677c61b0510ad76c7ab6299138
Image: traefik:1.3.3
Image ID: docker://sha256:d877ced70df7fb9ec465ba2b4aee9b5059ac24c1db98369ec574addab495f39b
...
I think this is related to #1714. A couple of people have reported that the fix did not seem to work:
- fix: Double compression. #1714 (comment)
- Malformed/garbled HTML response when compress is enabled #1060 (comment)
Thanks!