Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP 302 loop - ERR_TOO_MANY_REDIRECTS #10335

Closed
nellicus opened this issue Feb 14, 2017 · 27 comments
Closed

HTTP 302 loop - ERR_TOO_MANY_REDIRECTS #10335

nellicus opened this issue Feb 14, 2017 · 27 comments

Comments

@nellicus
Copy link
Contributor

nellicus commented Feb 14, 2017

Kibana version:
5.2.0

Elasticsearch version:
N/A

Server OS version:
Debian

Browser version:
curl

Browser OS version:
Debian 7

Original install method (e.g. download page, yum, from source, etc.):
tar.gz

Description of the problem including expected versus actual behavior:
Kibana expose 302 loop when ES is unreachable

Steps to reproduce:
configure Kibana to connect to imaginary cluster as per default docs

Errors in browser console (if relevant):
ERR_TOO_MANY_REDIRECTS

Provide logs and/or server output (if relevant):

spons [10:15:17.698]  GET / 302 4ms - 9.0B
  log   [10:15:18.048] [warning][admin][elasticsearch] Unable to revive connection: https://w530:9200/
  log   [10:15:18.049] [warning][admin][elasticsearch] No living connections
respons [10:15:18.606]  GET / 302 7ms - 9.0B
respons [10:15:18.614]  GET /login?next=%2F 302 2ms - 9.0B
respons [10:15:18.626]  GET / 302 6ms - 9.0B
respons [10:15:18.648]  GET /login?next=%2F 302 2ms - 9.0B
respons [10:15:18.660]  GET / 302 5ms - 9.0B
respons [10:15:18.667]  GET /login?next=%2F 302 2ms - 9.0B
respons [10:15:18.681]  GET / 302 5ms - 9.0B
respons [10:15:18.689]  GET /login?next=%2F 302 2ms - 9.0B
respons [10:15:18.704]  GET / 302 4ms - 9.0B
respons [10:15:18.712]  GET /login?next=%2F 302 1ms - 9.0B
respons [10:15:18.723]  GET / 302 5ms - 9.0B
respons [10:15:18.732]  GET /login?next=%2F 302 2ms - 9.0B
respons [10:15:18.751]  GET / 302 4ms - 9.0B
respons [10:15:18.763]  GET /login?next=%2F 302 1ms - 9.0B
respons [10:15:18.787]  GET / 302 5ms - 9.0B
respons [10:15:18.801]  GET /login?next=%2F 302 1ms - 9.0B
respons [10:15:18.819]  GET / 302 2ms - 9.0B
respons [10:15:18.828]  GET /login?next=%2F 302 1ms - 9.0B
respons [10:15:18.846]  GET / 302 5ms - 9.0B
respons [10:15:18.850]  GET / 302 4ms - 9.0B
respons [10:15:18.858]  GET /login?next=%2F 302 1ms - 9.0B
respons [10:15:18.873]  GET / 302 4ms - 9.0B
  log   [10:15:19.455] [warning][elasticsearch][monitoring-ui] Unable to revive connection: https://w530:9200/
  log   [10:15:19.456] [warning][elasticsearch][monitoring-ui] No living connections
  ops   [10:15:19.638]  memory: 96.6MB uptime: 0:02:09 load: [2.17 1.45 0.67] delay: 0.566
$ time curl https://w530:5601  -k  -L  --max-redirs 10000
curl: (47) Maximum (10000) redirects followed

real	0m13.535s
user	0m0.608s
sys	0m0.252s
user@w530 $ /opt/elk/prod/version/5.2.0/kibana-5.2.0-linux-x86_64/bin/kibana-plugin list
x-pack@5.2.0

user@w530 $ egrep '^[^#]+' /opt/elk/prod/common/kibana-conf/kibana.yml
server.port: 5601
server.host: "w530"
server.name: "tonybana"
elasticsearch.url: "https://w530:9200"
elasticsearch.username: "kibana"
elasticsearch.password: "XXXXXX"
server.ssl.cert: /opt/elk/prod/common/kibana-conf/kibana-ssl/kibana.crt.pem
server.ssl.key: /opt/elk/prod/common/kibana-conf/kibana-ssl/kibana.key.pem
elasticsearch.ssl.cert: /opt/elk/prod/common/ssl/certgen_output/w530/w530.crt
elasticsearch.ssl.key: /opt/elk/prod/common/ssl/certgen_output/w530/w530.key
elasticsearch.ssl.ca: /opt/elk/prod/common/ssl/certgen_output/ca/ca.crt
elasticsearch.ssl.verify: true
logging.verbose: true
xpack.security.encryptionKey: "dsiajdiojoijeiojewiofhqhfuierhqguihreiupfhripfqhewpfhiowehqfpuhgpwuqihfuhat_least_32_characters"
xpack.reporting.encryptionKey: "dfkfkdsjfkldjsklfjdsfjioasjfiodsjfio;dsjf;idsjfo;idjsoi;fjdsio;fjdsa;iofhds;oiahf;iashdi;ofhsd;iah"
console.proxyConfig:
  - match:
      host: "*"
      port: "{9200..9202}"
    ssl:
      ca: "/opt/elk/prod/common/ssl/certgen_output/ca/ca.crt"
@nellicus nellicus added the bug Fixes for quality problems that affect the customer experience label Feb 14, 2017
@junoteam
Copy link

junoteam commented Feb 16, 2017

Hello,
I have the same issue exactly.
kibana doing redirect loop with status code 302.
And this is happening after I change user's password for: elastic and kibana users trough the web interface of kibana.

199.203.151.53 - - [16/Feb/2017:15:36:40 +0000] "GET /login?next=%2F HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-" 199.203.151.53 - - [16/Feb/2017:15:36:40 +0000] "GET / HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-" 199.203.151.53 - - [16/Feb/2017:15:36:40 +0000] "GET /login?next=%2F HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-" 199.203.151.53 - - [16/Feb/2017:15:36:41 +0000] "GET / HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-" 199.203.151.53 - - [16/Feb/2017:15:36:41 +0000] "GET /login?next=%2F HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-" 199.203.151.53 - - [16/Feb/2017:15:36:41 +0000] "GET / HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-" 199.203.151.53 - - [16/Feb/2017:15:36:41 +0000] "GET /login?next=%2F HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-" 199.203.151.53 - - [16/Feb/2017:15:36:42 +0000] "GET / HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-" 199.203.151.53 - - [16/Feb/2017:15:36:42 +0000] "GET /login?next=%2F HTTP/1.1" 302 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-"

@armandoltx
Copy link

After installing x-pack I changed the password of an existing user, "Kibana User" and since I did it, I cannot access to my app. When I uninstall the X-Pack I can access as always, but reinstall the plugin and the problem came back.

@luckydonald
Copy link

luckydonald commented Apr 20, 2017

Can confirm, that forward loop started after changing the default password for elastic in the kibana web gui.
kibana.example.com redirected you too many times.

Running in docker, behind a reverse proxy.

$ docker version
Server:
 Version:      1.13.0
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:58:26 2017
 OS/Arch:      linux/amd64

Image: docker.elastic.co/kibana/kibana:5.3.0

@luckydonald
Copy link

In fact, changing the passwords back to changeme is a workaround.

I just executed:

POST _xpack/security/user/elastic/_password
{
  "password": "changeme"
}

@lukasolson
Copy link
Member

Can you check your cookies to see if you have multiple cookies for the domain?

@m4rkw
Copy link

m4rkw commented Apr 26, 2017

I had the same issue with Kibana 5.3.0. It seems if you change the default passwords you need to give kibana a password it can use to talk to the ES cluster in kibana.yml:

elasticsearch.username: "kibana"
elasticsearch.password: "pass"

after doing this it seems to work. not sure why it can't just use the credentials provided in the web interface when you log in though. maybe someone can clarify?

@lukasolson
Copy link
Member

Usually what causes this is having multiple cookies for the same "domain" and "name", but with different values for "path". If you open the developer tools in Chrome, then click on the Application tab, then expand the Cookies section, and click on the domain, do you have multiple cookies with the name "sid"? If so, you can fix this issue by clearing all of them.

@luckydonald
Copy link

luckydonald commented Apr 27, 2017

@lukasolson After kibana didn't work any more, I openend it in a new browser (Safari) where I never used this instance of kibana before. Got the same issue there too.
I had no time to redo it while monitoring my cookies.
@m4rkw yeah, which is strange. After using kibanas "change password" field, I expected same kibana to pick up the entered changes.

@andrewmclagan
Copy link

Also seeing this on 5.1.1. Cleared browser cache, used different browsers

@chrissound
Copy link

Same issue here.

@Panblack
Copy link

Panblack commented May 27, 2017

{"type":"log","@timestamp":"2017-05-27T09:45:56Z","tags":["warning","elasticsearch","admin"],"pid":19343,"message":"No living connections"}

For me, this error caused 302 loop.
Edit /config/kibana.yml and fix elasticsearch.url: ,the problem solved.

@chrissound
Copy link

If it's helpful at all, this issue occurred to me when using 2 elastic search nodes. When I just had 1 the error stopped.

Here is the working docker-compose.yaml I ended up with:

version: '2'
services:
  kibanna:
    image: docker.elastic.co/kibana/kibana:5.4.0
    ports:
      - 5601:5601
    networks:
      - esnet
    environment:
       ELASTICSEARCH_URL: http://elasticsearch1
  elasticsearch1:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.4.0
    container_name: elasticsearch1
    environment:
      - cluster.name=docker-cluster
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    mem_limit: 1g
    volumes:
      - esdata1:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - esnet
volumes:
  esdata1:
    driver: local

networks:
  esnet:

@caperea
Copy link

caperea commented Jun 24, 2017

one seemingly workaround is using http://domain:port/login?next=/login

@shijij
Copy link

shijij commented Jun 24, 2017

This bug appeared when I add node.local: true to the config file and restart ES.
Then even when I remove that and restart won't solve the problem.
Have to clear all cookies and goto /login?next=/login to end the loop. as @caperea mentioned

@chrowe
Copy link

chrowe commented Jun 26, 2017

I am having this issue as well. I use Chrome usually and have been testing with Incognito mode to avoid issues with cookies but that does not resolve the issue.

I am using version 5.4.1 of everything.

@lukasolson just to be sure, I tried your suggestion and see no cookies.

@dili91
Copy link

dili91 commented Jun 27, 2017

same as @chrowe

@chrowe
Copy link

chrowe commented Jun 27, 2017

My issue was solved by https://discuss.elastic.co/t/cant-access-kibana-after-changing-password/90701/2

This is the same solution as #10335 (comment)

@chrowe
Copy link

chrowe commented Jul 1, 2017

Has something changed in 5.4.3 that would fix this?

After chrowe/docker-elk@8bf9c62 I was able to just change the password in the Kibana GUI and did not have this issue.

@nellicus
Copy link
Contributor Author

nellicus commented Jul 3, 2017

closing as per the above

@nellicus nellicus closed this as completed Jul 3, 2017
@nellicus nellicus removed the bug Fixes for quality problems that affect the customer experience label Jul 3, 2017
@luckydonald
Copy link

@nellicus which do you mean?

@jbrant
Copy link

jbrant commented Nov 30, 2017

This is still an issue in elasticsearch 5.5.1.

@aclowkey
Copy link

6.7 - still happens!

@c4milo
Copy link

c4milo commented Apr 30, 2019

We manually patched Kibana 5.5.3 to fix this problem, here is the patch just in case anyone else finds it useful:

diff --git a/src/server/config/schema.js b/src/server/config/schema.js
index e49033f4..f23d195b 100644
--- a/src/server/config/schema.js
+++ b/src/server/config/schema.js
@@ -59,7 +59,9 @@ module.exports = () => _joi2.default.object({
     uuid: _joi2.default.string().guid().default(),
     name: _joi2.default.string().default(_os2.default.hostname()),
     host: _joi2.default.string().hostname().default('localhost'),
+    address: _joi2.default.string().hostname().default('localhost'),
     port: _joi2.default.number().default(5601),
+    uri: _joi2.default.string(),
     maxPayloadBytes: _joi2.default.number().default(1048576),
     autoListen: _joi2.default.boolean().default(true),
     defaultRoute: _joi2.default.string().default('/app/kibana').regex(/^\//, `start with a slash`),
diff --git a/src/server/http/setup_connection.js b/src/server/http/setup_connection.js
index cbb522f6..2bc98e83 100644
--- a/src/server/http/setup_connection.js
+++ b/src/server/http/setup_connection.js
@@ -10,10 +10,14 @@ exports.default = function (kbnServer, server, config) {
   // this mixin is used outside of the kbn server, so it MUST work without a full kbnServer object.
   kbnServer = null;

+  const address = config.get('server.address');
+  const uri = config.get('server.uri');
   const host = config.get('server.host');
   const port = config.get('server.port');

   const connectionOptions = {
+    address,
+    uri,
     host,
     port,
     state: {

And, In kibana.yml we configure those 2 parameters like this:

# Tell kibana to bind to all network addresses
server.address: "0.0.0.0"
server.host: "mykibana.com"
server.port: 5601
# This is what gets used to build redirect URIs
server.uri: "https://mykibana.com"

@chrissound
Copy link

This issue appeared when I deleted all the elasticsearch indexes with curl -XDELETE "http://127.0.0.1:9200/_all".

@ValentinMATHIEU
Copy link

7.10.1 - The issue is still present ... 🤔

@kyosifov
Copy link

Is there any fix for this. I am using OpenVidu -- https://s3-eu-west-1.amazonaws.com/aws.openvidu.io/install_openvidu_pro_2.17.0.sh and the docker-compose.yml file is using the kibana and elastic search services, but when i try to connect to kibana I get the error message "Too many redirects"

@cromat
Copy link

cromat commented Sep 27, 2021

The error still persists in version 7.14.1.

Reproducible via latest docker image by setting SERVER_BASEPATH env variable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests