Kibana console redirect loop #1457

danbev · 2016-01-20T08:29:28Z

We are currently working on deploying centralised logging in OpenShift V3.1 and running into an issue when trying to access the Kibana console. We are running OpenShift on a VM using Vagrant. Unfortunately I cannot give a link to the github repository as the repository is private and not yet open source (it will be in the near future). If it would help I can try to create a zip for testing purposes.

These are the instructions we have followed to configure the centralised logging. These are pretty much the same as aggregate_logging.html:

$ oc login --username=system:admin

$ oadm new-project logging
$ oc project logging

$ oadm policy add-role-to-user admin test -n logging

$ openssl genrsa -out key.pem 2048
$ openssl req -new -key key.pem -out csr.pem
$ openssl req -new -key key.pem -out csr.pem
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:SE
State or Province Name (full name) []:Stockholm
Locality Name (eg, city) [Default City]:Stockholm
Organization Name (eg, company) [Default Company Ltd]:Red Hat
Organizational Unit Name (eg, section) []:AeroGear
Common Name (eg, your name or your server's hostname) []:kibana.local.feedhenry.io
Email Address []:daniel.bevenius@gmail.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
$ openssl req -x509 -days 365 -key key.pem -in csr.pem -out certificate.pem

$ oc secrets new logging-deployer kibana.crt=certificate.pem kibana.key=key.pem

$ oc create -f - <<API
apiVersion: v1
kind: ServiceAccount
metadata:
  name: logging-deployer
secrets:
- name: logging-deployer
API

$ oc policy add-role-to-user edit \
            system:serviceaccount:logging:logging-deployer

$ oadm policy add-scc-to-user privileged system:serviceaccount:logging:aggregated-logging-fluentd

$ oadm policy add-cluster-role-to-user cluster-reader \
              system:serviceaccount:logging:aggregated-logging-fluentd

$ oc create -n openshift -f /usr/share/openshift/examples/infrastructure-templates/enterprise/logging-deployer.yaml

$ docker pull registry.access.redhat.com/openshift3/logging-deployment:3.1.0

$ oc process logging-deployer-template -n openshift \
    -v KIBANA_HOSTNAME=kibana.local.feedhenry.io,ES_CLUSTER_SIZE=1,PUBLIC_MASTER_URL=https://local.feedhenry.io:8443,MASTER_URL=https://kubernetes.default.svc.cluster.local:8443 \
     | oc create -f -

$ oc process logging-support-template | oc create -f -

Giving some time for the pods to start up running `oc get all`` produces:

[vagrant@local ~]$ oc get all
NAME                          DOCKER REPO                                                   TAGS                                 UPDATED
logging-auth-proxy            registry.access.redhat.com/openshift3/logging-auth-proxy      3.1.0-1,3.1.0-4,latest + 1 more...   9 minutes ago
logging-elasticsearch         registry.access.redhat.com/openshift3/logging-elasticsearch   latest,3.1.0-1,3.1.0-4 + 1 more...   9 minutes ago
logging-fluentd               registry.access.redhat.com/openshift3/logging-fluentd         3.1.0-1,3.1.0,3.1.0-6 + 1 more...    9 minutes ago
logging-kibana                registry.access.redhat.com/openshift3/logging-kibana          3.1.0-4,3.1.0-1,latest + 1 more...   9 minutes ago
NAME                          TRIGGERS                                                      LATEST
logging-es-s6vgko2m           ConfigChange, ImageChange                                     1
logging-fluentd               ConfigChange, ImageChange                                     1
logging-kibana                ConfigChange, ImageChange                                     1
CONTROLLER                    CONTAINER(S)                                                  IMAGE(S)                                                             SELECTOR                                                                                                REPLICAS   AGE
logging-es-s6vgko2m-1         elasticsearch                                                 registry.access.redhat.com/openshift3/logging-elasticsearch:latest   component=es,deployment=logging-es-s6vgko2m-1,deploymentconfig=logging-es-s6vgko2m,provider=openshift   1          9m
logging-fluentd-1             fluentd-elasticsearch                                         registry.access.redhat.com/openshift3/logging-fluentd:latest         component=fluentd,deployment=logging-fluentd-1,deploymentconfig=logging-fluentd,provider=openshift      0          9m
logging-kibana-1              kibana                                                        registry.access.redhat.com/openshift3/logging-kibana:latest          component=kibana,deployment=logging-kibana-1,deploymentconfig=logging-kibana,provider=openshift         1          9m
                              kibana-proxy                                                  registry.access.redhat.com/openshift3/logging-auth-proxy:latest
NAME                          HOST/PORT                                                     PATH                                                                 SERVICE                                                                                                 LABELS                                                       INSECURE POLICY   TLS TERMINATION
kibana                        kibana.local.feedhenry.io                                                                                                          logging-kibana                                                                                          component=support,logging-infra=support,provider=openshift                     passthrough
kibana-ops                    kibana-ops.example.com                                                                                                             logging-kibana-ops                                                                                      component=support,logging-infra=support,provider=openshift                     passthrough
NAME                          CLUSTER_IP                                                    EXTERNAL_IP                                                          PORT(S)                                                                                                 SELECTOR                                                     AGE
logging-es                    172.30.163.210                                                <none>                                                               9200/TCP                                                                                                component=es,provider=openshift                              9m
logging-es-cluster            None                                                          <none>                                                               9300/TCP                                                                                                component=es,provider=openshift                              9m
logging-es-ops                172.30.203.223                                                <none>                                                               9200/TCP                                                                                                component=es-ops,provider=openshift                          9m
logging-es-ops-cluster        None                                                          <none>                                                               9300/TCP                                                                                                component=es-ops,provider=openshift                          9m
logging-kibana                172.30.49.41                                                  <none>                                                               443/TCP                                                                                                 component=kibana,provider=openshift                          9m
logging-kibana-ops            172.30.212.155                                                <none>                                                               443/TCP                                                                                                 component=kibana-ops,provider=openshift                      9m
NAME                          READY                                                         STATUS                                                               RESTARTS                                                                                                AGE
logging-deployer-v5wln        0/1                                                           Completed                                                            0                                                                                                       10m
logging-es-s6vgko2m-1-kl6j2   1/1                                                           Running                                                              0                                                                                                       9m
logging-kibana-1-htgd6        2/2                                                           Running                                                              1

Accessing https://kibana.local.feedhenry.io will redirect to the OpenShift Console login screen, and when credentials are entered we enter a redirect loop coming back to the same login screen again.

We have followed the trouble shooting section and tried out the suggestions there but with out success.

Please let me know if there is any additional information that I can provide.

Thanks!

Motivation: When running in a Vagrant VM we noticed that we were hitting a redirect loop when trying to access the kibana console. We created the and issue for this in openshift-docs (see Issue section below). When processing the template we specify the following: $ oc process logging-deployer-template -n openshift \ -v KIBANA_HOSTNAME=kibana.local.feedhenry.io,ES_CLUSTER_SIZE=1, \ PUBLIC_MASTER_URL=https://local.feedhenry.io:8443, \ MASTER_URL=https://kubernetes.default.svc.cluster.local:8443 \ | oc create -f - Notice that we have specified a MASTER_URL. But when I tried run describe on the pod I see it with out the port (defaulting to 443): $ oc describe po logging-kibana-6-9aac9 ... Environment Variables: OAP_BACKEND_URL: http://localhost:5601 OAP_AUTH_MODE: oauth2 OAP_TRANSFORM: user_header,token_header OAP_OAUTH_ID: kibana-proxy OAP_MASTER_URL: https://kubernetes.default.svc.cluster.local It looks like the environment variable OAP_MASTER_URL is never when deployment/templates/kibana.yaml is processed. So the default value specified in that file is used which is: name: OAP_MASTER_URL value: "https://kubernetes.default.svc.cluster.local" Modifications: Added OAP_MASTER_URL to the run.sh script when processing templates/kibana.yaml Result: We can now access the Kibana console via the OpenShift console and also directly. Issue: openshift/openshift-docs#1457

danbev mentioned this issue Jan 21, 2016

Adding OAP_MASTER_URL variable when processing kibana.yaml openshift/origin-aggregated-logging#50

Merged

danbev closed this as completed Jan 23, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Kibana console redirect loop #1457

Kibana console redirect loop #1457

danbev commented Jan 20, 2016

Kibana console redirect loop #1457

Kibana console redirect loop #1457

Comments

danbev commented Jan 20, 2016