Kibana base URL is misconfigured when using kubectl proxy to access the service

[gajus]

Kibana version:

5.3.0

Elasticsearch version:
Server OS version:
Browser version:
Browser OS version:

N/A

Original install method (e.g. download page, yum, from source, etc.):

Kubernetes/ Docker.

More or less, following instructions from https://github.com/NestAway/efk-stack.

Description of the problem including expected versus actual behavior:

Steps to reproduce:

1. Going to http://localhost:8001/api/v1/proxy/namespaces/efk/services/kibana-logging/
2. Redirects to http://localhost:8001/api/v1/proxy/namespaces/efk/services/kibana-logging/login?next=%2F
3. Produces UI with an error "Kibana did not load properly. Check the server output for more information."

Errors in browser console (if relevant):

The following resources produce error 404:

commons.style.css?v=14723
login.style.css?v=14723
commons.bundle.js?v=14723

The same resources can be accessed from:

http://localhost:8001/api/v1/proxy/namespaces/efk/services/kibana-logging/[resource path]

Provide logs and/or server output (if relevant):
Describe the feature:

N/A

[gajus]

A somewhat workaround is to setup port forwarding,

kubectl port-forward kibana-logging-2362227606-0gkjq --namespace efk 5601:5601

This makes it work.

The underlying issue remains though – Kibana interface does not work when it is accessed from a path.

[epixa]

We're not the ones behind the efk-stack repo you linked, so I'm not sure what that setup looks like or what they're doing with Kibana configuration. If you can break this down into a native Kibana setup from official sources, then we can probably provide some help, but based on what you posted here this seems to be a bug with that third party project.

[gajus]

We're not the ones behind the efk-stack repo you linked, so I'm not sure what that setup looks like or what they're doing with Kibana configuration. If you can break this down into a native Kibana setup from official sources, then we can probably provide some help, but based on what you posted here this seems to be a bug with that third party project.

This has nothing to do with how it is setup, it only has to do with how it is accessed.

But yeah, good luck dismissively closing issues.

[epixa]

@gajus Pardon the confusion here, I didn't mean to offend.

Can you provide details that are native to Kibana such as the contents of kibana.yml? Or if you're using the Elastic-provided docker containers, can you include the relevant environment variables? Is the server.basePath set properly? Is the proxy correctly removing the basePath before sending to the Kibana server?

It sounds like the basePath isn't configured properly, but it's hard to tell. I'm not at all familiar with kubectl proxies, so I'm not sure what the expected behavior is when you hit any of those URLs. I'm happy to help if we can get the problem closer to Kibana, and if there is a bug here I'm happy to reopen this issue and triage it accordingly.

[gajus]

The basePath is not configured.

I am presuming that Kibana is able to function without this configuration, or put it another way – it was working in v4 using kubectl proxy without configuring a basepath. If this is a breaking change, then you will have a lot of people complaining as an active migration happens in k8s world.

I am suspecting that this issue is the same issue #10093 as well as this issue #8967.

[epixa]

The issue you're encountering is manifesting in the same way as the two you linked, but there isn't enough information to say for sure that they're related. Unfortunately, the scenario you're describing is one that can be caused by many different issues behind the scenes.

#10093 is running dev mode, so it has our basePath proxy automatically set up and configured by default, so it is likely caused by either permissions issues or the a lack of sh access.

#8967 could be the same issue you're encountering, but it's not clear. My best guess in that ticket is a permissions issue or corrupted bundles, but the author didn't circle back to it, so we don't know for sure.

server.basePath is only necessary in Kibana when you are running Kibana behind a reverse proxy that is configured to serve Kibana at a non-root path. As I said before, I don't know much about kubectl proxies, but it looks like that is what is happening here. Can you set up the basePath configuration and see if that fixes it? So you don't need to hunt it down in the docs:

server.basePath: Enables you to specify a path to mount Kibana at if you are running behind a proxy. This only affects the URLs generated by Kibana, your proxy is expected to remove the basePath value before forwarding requests to Kibana. This setting cannot end in a slash

[gajus]

As I said before, I don't know much about kubectl proxies, but it looks like that is what is happening here. Can you set up the basePath configuration and see if that fixes it? So you don't need to hunt it down in the docs:

A URL generated using proxy is a dynamic URL, therefore configuring Kibana to use one or another baseurl at the setup time is of little value.

Either way, for all intents and purposes, kubectl port-forwarding fixes the problem.

kubectl port-forward kibana-logging-2362227606-0gkjq --namespace efk 5601:5601

Thank you for your help.

[epixa]

@gajus Ah, I see. Well, I'm glad you got something working. Sorry for the rocky start here!