Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Panic in runtime.go when using TLS #814

Closed
3dbrows opened this issue Jan 19, 2018 · 1 comment · Fixed by #836
Assignees

Comments

@3dbrows
Copy link

@3dbrows 3dbrows commented Jan 19, 2018

Hi, I'm getting a panic when using an ingress with TLS support on Voyager 5.0.0-rc.11 and K8S 1.9 (kubeadm, bare metal). It works if I remove the tls block.

Voyager was installed with curl -fsSL https://raw.githubusercontent.com/appscode/voyager/5.0.0-rc.11/hack/deploy/voyager.sh | bash -s -- -p baremetal --rbac --run-on-master. I'm running in HostPort mode as I'm not in an environment with loadbalancers i.e. I want to attach a specific IP to the ingress.

Please let me know if you need further details to reproduce.

ingress.yml:

apiVersion: voyager.appscode.com/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: voyager
    ingress.appscode.com/type: HostPort
    ingress.kubernetes.io/auth-tls-secret: ca
    ingress.kubernetes.io/auth-tls-verify-client: required
    ingress.kubernetes.io/auth-tls-error-page: "https://[redacted]/tlsError.html"
spec:
  externalIPs:
  - 10.44.55.116
  backend:
    serviceName: default-http-backend
    servicePort: '80'
  tls:
  - hosts:
    - [redacted]
    ref:
      kind: Secret
      name: server
  rules:
  - host: [redacted]
    http:
      paths:
      - backend:
          serviceName: hello-world-svc
          servicePort: '80'

Logs:

$  kubectl logs voyager-operator-5fc87bc866-x5lcp --namespace=kube-system
I0119 15:12:29.110753       1 logs.go:19] FLAG: --address=":56790"
I0119 15:12:29.110808       1 logs.go:19] FLAG: --alsologtostderr="false"
I0119 15:12:29.110816       1 logs.go:19] FLAG: --analytics="true"
I0119 15:12:29.110828       1 logs.go:19] FLAG: --burst="1000000"
I0119 15:12:29.110833       1 logs.go:19] FLAG: --cloud-config=""
I0119 15:12:29.110837       1 logs.go:19] FLAG: --cloud-provider=""
I0119 15:12:29.110842       1 logs.go:19] FLAG: --custom-templates=""
I0119 15:12:29.110847       1 logs.go:19] FLAG: --exporter-sidecar-image="appscode/voyager:5.0.0-rc.11"
I0119 15:12:29.110855       1 logs.go:19] FLAG: --haproxy-image="appscode/haproxy:1.7.9-5.0.0-rc.11"
I0119 15:12:29.110861       1 logs.go:19] FLAG: --haproxy.server-metric-fields="2,3,4,5,6,7,8,9,13,14,15,16,17,18,21,24,33,35,38,39,40,41,42,43,44"
I0119 15:12:29.110872       1 logs.go:19] FLAG: --haproxy.timeout="5s"
I0119 15:12:29.110877       1 logs.go:19] FLAG: --help="false"
I0119 15:12:29.110882       1 logs.go:19] FLAG: --ingress-class=""
I0119 15:12:29.110890       1 logs.go:19] FLAG: --kubeconfig=""
I0119 15:12:29.110898       1 logs.go:19] FLAG: --log.format="\"logger:stderr\""
I0119 15:12:29.110903       1 logs.go:19] FLAG: --log.level="\"info\""
I0119 15:12:29.110909       1 logs.go:19] FLAG: --log_backtrace_at=":0"
I0119 15:12:29.110914       1 logs.go:19] FLAG: --log_dir=""
I0119 15:12:29.110918       1 logs.go:19] FLAG: --logtostderr="true"
I0119 15:12:29.110923       1 logs.go:19] FLAG: --master=""
I0119 15:12:29.110929       1 logs.go:19] FLAG: --operator-service="voyager-operator"
I0119 15:12:29.110936       1 logs.go:19] FLAG: --qps="1e+06"
I0119 15:12:29.110941       1 logs.go:19] FLAG: --rbac="true"
I0119 15:12:29.110946       1 logs.go:19] FLAG: --restrict-to-operator-namespace="false"
I0119 15:12:29.110951       1 logs.go:19] FLAG: --resync-period="5m0s"
I0119 15:12:29.110961       1 logs.go:19] FLAG: --stderrthreshold="2"
I0119 15:12:29.110966       1 logs.go:19] FLAG: --v="3"
I0119 15:12:29.110971       1 logs.go:19] FLAG: --vmodule=""
W0119 15:12:29.289212       1 client_config.go:529] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0119 15:12:29.290104       1 run.go:121] Starting Voyager operator...
I0119 15:12:29.290148       1 operator.go:55] Ensuring CRD registration
I0119 15:12:32.313070       1 validator.go:52] Checking ingress test-ingress@default
I0119 15:12:32.319261       1 run.go:153] URL pattern: /:apiGroup/v1beta1/namespaces/:namespace/ingresses/:name/metrics
I0119 15:12:32.319283       1 run.go:157] Listening on :56790
I0119 15:12:32.335928       1 servicemonitors.go:21] Skipping watching non-preferred GroupVersion:monitoring.coreos.com/v1 Kind:ServiceMonitor
I0119 15:12:32.336168       1 reflector.go:202] Starting reflector *v1beta1.Deployment (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336168       1 reflector.go:202] Starting reflector *v1.Service (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336190       1 reflector.go:240] Listing and watching *v1beta1.Deployment from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336196       1 reflector.go:240] Listing and watching *v1.Service from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336203       1 reflector.go:202] Starting reflector *v1beta1.DaemonSet (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336218       1 reflector.go:240] Listing and watching *v1beta1.DaemonSet from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336215       1 reflector.go:202] Starting reflector *v1.Endpoints (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336244       1 reflector.go:240] Listing and watching *v1.Endpoints from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336296       1 reflector.go:202] Starting reflector *v1.Node (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336311       1 reflector.go:240] Listing and watching *v1.Node from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336334       1 reflector.go:202] Starting reflector *v1beta1.Certificate (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336374       1 reflector.go:240] Listing and watching *v1beta1.Certificate from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336421       1 reflector.go:202] Starting reflector *v1.Secret (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336435       1 reflector.go:240] Listing and watching *v1.Secret from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336430       1 reflector.go:202] Starting reflector *v1.Namespace (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336444       1 reflector.go:240] Listing and watching *v1.Namespace from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336423       1 reflector.go:202] Starting reflector *v1beta1.Ingress (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336493       1 reflector.go:240] Listing and watching *v1beta1.Ingress from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336622       1 reflector.go:202] Starting reflector *v1beta1.Ingress (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336633       1 reflector.go:240] Listing and watching *v1beta1.Ingress from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336705       1 reflector.go:202] Starting reflector *v1.ConfigMap (5m0s) from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.336751       1 reflector.go:240] Listing and watching *v1.ConfigMap from github.com/appscode/voyager/pkg/operator/operator.go:84
I0119 15:12:32.345048       1 ingress_crds.go:41] [fd2dc86a-9fc8-414c-86b8-ca2fcf55553f] voyager.appscode.com/v1beta1 test-ingress@default added
I0119 15:12:32.345163       1 hostport.go:67] fd2dc86a-9fc8-414c-86b8-ca2fcf55553f Initializing cloud manager for provider
I0119 15:12:32.345175       1 hostport.go:95] fd2dc86a-9fc8-414c-86b8-ca2fcf55553f No cloud manager found for provider
I0119 15:12:32.359761       1 services.go:38] [cb1707a6-5855-413a-9785-1f7b84e42015] Service default-http-backend@default added
I0119 15:12:32.364114       1 parser.go:27] fd2dc86a-9fc8-414c-86b8-ca2fcf55553f getting endpoints for  default default-http-backend port {1 0 80}
I0119 15:12:32.364177       1 parser.go:40] fd2dc86a-9fc8-414c-86b8-ca2fcf55553f looking for services in namespace default with name default-http-backend
I0119 15:12:32.365067       1 hostport.go:67] cb1707a6-5855-413a-9785-1f7b84e42015 Initializing cloud manager for provider
I0119 15:12:32.365081       1 hostport.go:95] cb1707a6-5855-413a-9785-1f7b84e42015 No cloud manager found for provider
I0119 15:12:32.367834       1 services.go:138] [cb1707a6-5855-413a-9785-1f7b84e42015] One or more offshoots of voyager.appscode.com/v1beta1 Ingress default/test-ingress is missing, trying to create
I0119 15:12:32.369171       1 parser.go:27] cb1707a6-5855-413a-9785-1f7b84e42015 getting endpoints for  default default-http-backend port {1 0 80}
I0119 15:12:32.369217       1 parser.go:40] cb1707a6-5855-413a-9785-1f7b84e42015 looking for services in namespace default with name default-http-backend
I0119 15:12:32.369297       1 services.go:38] [2bc37881-9d0d-460f-adaf-a19251ffc485] Service hello-world-svc@default added
I0119 15:12:32.373791       1 hostport.go:67] 2bc37881-9d0d-460f-adaf-a19251ffc485 Initializing cloud manager for provider
I0119 15:12:32.373840       1 hostport.go:95] 2bc37881-9d0d-460f-adaf-a19251ffc485 No cloud manager found for provider
I0119 15:12:32.374782       1 services.go:138] [2bc37881-9d0d-460f-adaf-a19251ffc485] One or more offshoots of voyager.appscode.com/v1beta1 Ingress default/test-ingress is missing, trying to create
I0119 15:12:32.378800       1 parser.go:27] 2bc37881-9d0d-460f-adaf-a19251ffc485 getting endpoints for  default default-http-backend port {1 0 80}
I0119 15:12:32.378833       1 parser.go:40] 2bc37881-9d0d-460f-adaf-a19251ffc485 looking for services in namespace default with name default-http-backend
I0119 15:12:33.398869       1 parser.go:122] [2bc37881-9d0d-460f-adaf-a19251ffc485] Found target port 8080 for service default-http-backend
I0119 15:12:33.398913       1 parser.go:27] 2bc37881-9d0d-460f-adaf-a19251ffc485 getting endpoints for  default hello-world-svc port {1 0 80}
I0119 15:12:33.398927       1 parser.go:40] 2bc37881-9d0d-460f-adaf-a19251ffc485 looking for services in namespace default with name hello-world-svc
I0119 15:12:33.400456       1 parser.go:122] [2bc37881-9d0d-460f-adaf-a19251ffc485] Found target port 80 for service hello-world-svc
E0119 15:12:33.401022       1 runtime.go:66] Observed a panic: "invalid memory address or nil pointer dereference" (runtime error: invalid memory address or nil pointer dereference)
/go/src/github.com/appscode/voyager/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:72
/go/src/github.com/appscode/voyager/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:65
/go/src/github.com/appscode/voyager/vendor/k8s.io/apimachinery/pkg/util/runtime/runtime.go:51
/usr/local/go/src/runtime/asm_amd64.s:509
/usr/local/go/src/runtime/panic.go:491
/usr/local/go/src/runtime/panic.go:63
/usr/local/go/src/runtime/signal_unix.go:367
/go/src/github.com/appscode/voyager/pkg/haproxy/types.go:75
/go/src/github.com/appscode/voyager/pkg/ingress/parser.go:715
/go/src/github.com/appscode/voyager/pkg/ingress/hostport.go:131
/go/src/github.com/appscode/voyager/pkg/operator/services.go:139
/go/src/github.com/appscode/voyager/pkg/operator/services.go:39
/go/src/github.com/appscode/voyager/vendor/k8s.io/client-go/tools/cache/controller.go:195
<autogenerated>:1
/go/src/github.com/appscode/voyager/vendor/k8s.io/client-go/tools/cache/controller.go:381
/go/src/github.com/appscode/voyager/vendor/k8s.io/client-go/tools/cache/delta_fifo.go:451
/go/src/github.com/appscode/voyager/vendor/k8s.io/client-go/tools/cache/controller.go:150
/go/src/github.com/appscode/voyager/vendor/k8s.io/client-go/tools/cache/controller.go:124
/go/src/github.com/appscode/voyager/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:133
/go/src/github.com/appscode/voyager/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:134
/go/src/github.com/appscode/voyager/vendor/k8s.io/apimachinery/pkg/util/wait/wait.go:88
/go/src/github.com/appscode/voyager/vendor/k8s.io/client-go/tools/cache/controller.go:124
/usr/local/go/src/runtime/asm_amd64.s:2337
@3dbrows

This comment has been minimized.

Copy link
Author

@3dbrows 3dbrows commented Jan 19, 2018

Further to this: I was successful with the following ingress.yml:

apiVersion: voyager.appscode.com/v1beta1
kind: Ingress
metadata:
  name: test-ingress
  namespace: default
  annotations:
    kubernetes.io/ingress.class: voyager
    ingress.appscode.com/type: HostPort
spec:
  tls:
  - secretName: tls-secret
    hosts:
    - [redacted]
  rules:
  - host: [redacted]
    http:
      paths:
      - backend:
          serviceName: hello-world-svc
          servicePort: '80'
  externalIPs:
  - 10.44.55.116
  backend:
    serviceName: default-http-backend
    servicePort: '80'

I think I got the configuration wrong due to confusing these docs:
https://appscode.com/products/voyager/5.0.0-rc.11/guides/ingress/tls/overview/
and
https://appscode.com/products/voyager/5.0.0-rc.11/guides/certificate/http/overview/

Not sure what the correct approach should be, but perhaps a validation step to catch this would be useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.