Failing test: X-Pack OpenID Connect API Integration Tests.x-pack/test/oidc_api_integration/apis/authorization_code_flow/oidc_auth·js - apis OpenID Connect authentication finishing handshake should succeed if both the OpenID Connect response and the cookie are provided #43736

kibanamachine · 2019-08-22T05:58:49Z

A test failed on a tracked branch


        Error: expected 302 "Found", got 401 "Unauthorized"
    at Test._assertStatus (/var/lib/jenkins/workspace/elastic+kibana+7.x/JOB/x-pack-ciGroup6/node/linux-immutable/kibana/node_modules/supertest/lib/test.js:268:12)
    at Test._assertFunction (/var/lib/jenkins/workspace/elastic+kibana+7.x/JOB/x-pack-ciGroup6/node/linux-immutable/kibana/node_modules/supertest/lib/test.js:283:11)
    at Test.assert (/var/lib/jenkins/workspace/elastic+kibana+7.x/JOB/x-pack-ciGroup6/node/linux-immutable/kibana/node_modules/supertest/lib/test.js:173:18)
    at assert (/var/lib/jenkins/workspace/elastic+kibana+7.x/JOB/x-pack-ciGroup6/node/linux-immutable/kibana/node_modules/supertest/lib/test.js:131:12)
    at /var/lib/jenkins/workspace/elastic+kibana+7.x/JOB/x-pack-ciGroup6/node/linux-immutable/kibana/node_modules/supertest/lib/test.js:128:5
    at Test.Request.callback (/var/lib/jenkins/workspace/elastic+kibana+7.x/JOB/x-pack-ciGroup6/node/linux-immutable/kibana/node_modules/superagent/lib/node/index.js:718:3)
    at parser (/var/lib/jenkins/workspace/elastic+kibana+7.x/JOB/x-pack-ciGroup6/node/linux-immutable/kibana/node_modules/superagent/lib/node/index.js:906:18)
    at IncomingMessage.res.on (/var/lib/jenkins/workspace/elastic+kibana+7.x/JOB/x-pack-ciGroup6/node/linux-immutable/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7)
    at endReadableNT (_stream_readable.js:1103:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)

First failure: Jenkins Build

The text was updated successfully, but these errors were encountered:

elasticmachine · 2019-08-22T05:58:51Z

Pinging @elastic/kibana-test-triage

kibanamachine · 2019-09-11T10:53:41Z

New failure: Jenkins Build

elasticmachine · 2019-09-11T12:56:16Z

Pinging @elastic/kibana-security

spalger · 2019-09-11T12:56:36Z

This has only failed 3 times in the last month (including PRs), but I think we should try to figure out how this is happening

azasypkin · 2019-09-11T14:45:57Z

Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to exchange code for Id Token using the Token Endpoint.]; nested: TimeoutException[Connection lease request time out];)

It seems Elasticsearch can't get token from an endpoint that is exposed by our test Kibana plugin for some reason, I wasn't able to reproduce, but I'll keep trying.

Just in case (since we still use ES from Aug 15th) - @jkakavas was there any ES OIDC issue that was fixed after Aug 15 and could have been the reason why ES couldn't retrieve token in exchange to authorization code?

jkakavas · 2019-09-11T14:52:09Z

Just in case (since we still use ES from Aug 15th) - @jkakavas was there any ES OIDC issue that was fixed after Aug 15 and could have been the reason why ES couldn't retrieve token in exchange to authorization code?

Nope... Is maybe something wrong with the dummy endpoint in Kibana (are there any logs that ES accessed that ) or could it be that ES is configured with the wrong URL for token endpoint ?

azasypkin · 2019-09-11T15:23:49Z

Nope... Is maybe something wrong with the dummy endpoint in Kibana (are there any logs that ES accessed that ) or could it be that ES is configured with the wrong URL for token endpoint ?

Thanks for confirming. Yeah, likely something wrong with this test endpoint then, no logs there unfortunately. I'll see if I can reproduce.

could it be that ES is configured with the wrong URL for token endpoint ?

Unlikely, it fails intermittently, just 3 times in a month.

azasypkin · 2019-09-11T15:53:40Z

Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to exchange code for Id Token using the Token Endpoint.]; nested: TimeoutException[Connection lease request time out];)

Well, now I'm wondering what this Connection lease request time out means. I see different errors when Kibana test endpoint either a) fails/doesn't exist or b) hangs/never returns:

a)

Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to exchange code for Id Token using the Token Endpoint. Unable to parse Token Response]; nested: ParseException[Missing JSON object member with key "token_type"];)

b)

Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to exchange code for Id Token using the Token Endpoint.]; nested: SocketTimeoutException[5,000 milliseconds timeout on connection http-outgoing-2 [ACTIVE]];)

azasypkin · 2019-09-19T09:09:22Z

Ioannis recommended to try to increase ES HTTP client timeout settings and see if it helps:

xpack.security.authc.realms.oidc.oidc1.http.connect_timeout
and/or
xpack.security.authc.realms.oidc.oidc1.http.connection_read_timeout

I'm going to wait and see if this issue comes up again and then try to increase these timeouts, locally I couldn't reproduce it even with very low timeout settings....

spalger · 2019-10-09T16:32:32Z

Just had this fail in a PR

azasypkin · 2019-10-10T15:32:46Z

Thanks for reporting, so I'll try to increase timeouts and see if it helps.

soumendrak · 2019-10-25T07:10:19Z

I am getting the point a) issue. Is there any way to resolve that? Increasing the timeout value did not help.

a)
Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused >by ElasticsearchSecurityException[Failed to exchange code for Id Token using the Token Endpoint. >Unable to parse Token Response]; nested: ParseException[Missing JSON object member with key ">token_type"];)

jkakavas · 2019-10-25T07:28:22Z

Hi @soumendrak , please do not hijack threads. This github issue is specifically for the CI tests of Kibana. Please ask your user question in the forums, we have a very active community there and someone will hopefully assist you get your configuration in order.

kibanamachine · 2019-11-12T16:12:59Z

New failure: Jenkins Build

azasypkin · 2020-01-14T07:19:41Z

Okay, it's been a month since last failure, closing for now. We'll see if it fails with the most recent ES versions when we switch to them.

kibanamachine · 2020-03-06T20:33:51Z

New failure: Jenkins Build

spalger · 2020-03-06T20:39:29Z

Getting the same 401 somewhat randomly

https://kibana-ci.elastic.co/job/elastic+kibana+7.x/3444/testReport/X-Pack%20OpenID%20Connect%20API%20Integration%20Tests/x-pack_test_oidc_api_integration_apis_authorization_code_flow_oidc_auth%C2%B7js/Kibana_Pipeline___kibana_xpack_agent___apis_OpenID_Connect_authentication_finishing_handshake_should_succeed_if_both_the_OpenID_Connect_response_and_the_cookie_are_provided/

Error: expected 302 "Found", got 401 "Unauthorized"
    at Test._assertStatus (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:268:12)
    at Test._assertFunction (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:283:11)
    at Test.assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:173:18)
    at assert (/dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:131:12)
    at /dev/shm/workspace/kibana/node_modules/supertest/lib/test.js:128:5
    at Test.Request.callback (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:718:3)
    at parser (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/index.js:906:18)
    at IncomingMessage.res.on (/dev/shm/workspace/kibana/node_modules/superagent/lib/node/parsers/json.js:19:7)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)

[00:00:00]       │
[00:00:00]         └-: apis
[00:00:00]           └-> "before all" hook
[00:00:00]           └-: OpenID Connect authentication
[00:00:00]             └-> "before all" hook
[00:00:00]             └-> should reject API requests if client is not authenticated
[00:00:00]               └-> "before each" hook: global before each
[00:00:00]               └- ✓ pass  (101ms) "apis OpenID Connect authentication should reject API requests if client is not authenticated"
[00:00:00]             └-: initiating handshake
[00:00:00]               └-> "before all" hook
[00:00:00]             └-: finishing handshake
[00:00:00]               └-> "before all" hook
[00:00:00]               └-> should fail if OpenID Connect response is not complemented with handshake cookie
[00:00:00]                 └-> "before each" hook: global before each
[00:00:00]                 └-> "before each" hook
[00:00:00]                 └- ✓ pass  (73ms) "apis OpenID Connect authentication finishing handshake should fail if OpenID Connect response is not complemented with handshake cookie"
[00:00:00]               └-> should fail if state is not matching
[00:00:00]                 └-> "before each" hook: global before each
[00:00:00]                 └-> "before each" hook
[00:00:00]                 │ info [o.e.x.s.a.AuthenticationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1583520435318159677] Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Invalid state parameter [someothervalue], while [B4OkYr04qDIlRqZ4tHuITus4RzDA9n3w6SPlrADKTJU] was expected])
[00:00:00]                 └- ✓ pass  (117ms) "apis OpenID Connect authentication finishing handshake should fail if state is not matching"
[00:00:00]               └-> should succeed if both the OpenID Connect response and the cookie are provided
[00:00:00]                 └-> "before each" hook: global before each
[00:00:00]                 └-> "before each" hook
[00:00:00]                 │ info [o.e.x.s.a.AuthenticationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1583520435318159677] Authentication to realm oidc1 failed - Failed to authenticate user with OpenID Connect (Caused by ElasticsearchSecurityException[Failed to exchange code for Id Token using the Token Endpoint.]; nested: TimeoutException[Connection lease request time out];)
[00:00:00]                 └- ✖ fail: "apis OpenID Connect authentication finishing handshake should succeed if both the OpenID Connect response and the cookie are provided"
[00:00:00]                 │

azasypkin · 2020-04-14T06:46:10Z

I haven't succeeded in reproducing this issue this time as well... There is a chance it may be related to some of the token related issues we're going to fix soon, let's see.

I'm closing this for now as we haven't seen any failed builds since March 6th.

kibanamachine · 2020-04-14T10:35:59Z

New failure: Jenkins Build

azasypkin · 2020-04-14T10:36:53Z

Haha, that's a destiny 🙂

legrego · 2020-04-14T11:54:04Z

You couldn't have planned that any better 😆

azasypkin · 2020-05-13T09:16:48Z

29 days without failure, I'll give it one week more this time 🤞

jkakavas · 2020-05-13T09:18:35Z

🤞

spalger · 2020-05-13T23:13:59Z

To be clear, this is mostly failing in PRs which aren't reported on issues, 6 failures in the last 60 days:

azasypkin · 2020-05-14T11:41:22Z

To be clear, this is mostly failing in PRs which aren't reported on issues, 6 failures in the last 60 days:

Aha, good to know, thanks. Let me track that there then. I still hope it will be automagically fixed some day 🙂

azasypkin · 2021-03-29T08:42:58Z

Flaky test runner: https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/1407/

azasypkin · 2021-03-29T12:56:00Z

Flaky test runner passed with no failures, closing.

kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Aug 22, 2019

spalger added the Team:Security Team focused on: Auth, Users, Roles, Spaces, Audit Logging, and more! label Sep 11, 2019

elasticmachine mentioned this issue Oct 9, 2019

[Uptime] Improve UI tests and platform #46269

Merged

7 tasks

azasypkin closed this as completed Jan 14, 2020

kibanamachine reopened this Mar 6, 2020

azasypkin closed this as completed Apr 14, 2020

kibanamachine reopened this Apr 14, 2020

azasypkin closed this as completed Mar 29, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Failing test: X-Pack OpenID Connect API Integration Tests.x-pack/test/oidc_api_integration/apis/authorization_code_flow/oidc_auth·js - apis OpenID Connect authentication finishing handshake should succeed if both the OpenID Connect response and the cookie are provided #43736

Failing test: X-Pack OpenID Connect API Integration Tests.x-pack/test/oidc_api_integration/apis/authorization_code_flow/oidc_auth·js - apis OpenID Connect authentication finishing handshake should succeed if both the OpenID Connect response and the cookie are provided #43736

kibanamachine commented Aug 22, 2019 •

edited

Loading

elasticmachine commented Aug 22, 2019

kibanamachine commented Sep 11, 2019

elasticmachine commented Sep 11, 2019

spalger commented Sep 11, 2019 •

edited

Loading

azasypkin commented Sep 11, 2019

jkakavas commented Sep 11, 2019

azasypkin commented Sep 11, 2019

azasypkin commented Sep 11, 2019

azasypkin commented Sep 19, 2019

spalger commented Oct 9, 2019

azasypkin commented Oct 10, 2019

soumendrak commented Oct 25, 2019

jkakavas commented Oct 25, 2019

kibanamachine commented Nov 12, 2019

azasypkin commented Jan 14, 2020

kibanamachine commented Mar 6, 2020

spalger commented Mar 6, 2020 •

edited

Loading

azasypkin commented Apr 14, 2020

kibanamachine commented Apr 14, 2020

azasypkin commented Apr 14, 2020

legrego commented Apr 14, 2020

azasypkin commented May 13, 2020

jkakavas commented May 13, 2020

spalger commented May 13, 2020

azasypkin commented May 14, 2020

azasypkin commented Mar 29, 2021

azasypkin commented Mar 29, 2021

Failing test: X-Pack OpenID Connect API Integration Tests.x-pack/test/oidc_api_integration/apis/authorization_code_flow/oidc_auth·js - apis OpenID Connect authentication finishing handshake should succeed if both the OpenID Connect response and the cookie are provided #43736

Failing test: X-Pack OpenID Connect API Integration Tests.x-pack/test/oidc_api_integration/apis/authorization_code_flow/oidc_auth·js - apis OpenID Connect authentication finishing handshake should succeed if both the OpenID Connect response and the cookie are provided #43736

Comments

kibanamachine commented Aug 22, 2019 • edited Loading

elasticmachine commented Aug 22, 2019

kibanamachine commented Sep 11, 2019

elasticmachine commented Sep 11, 2019

spalger commented Sep 11, 2019 • edited Loading

azasypkin commented Sep 11, 2019

jkakavas commented Sep 11, 2019

azasypkin commented Sep 11, 2019

azasypkin commented Sep 11, 2019

azasypkin commented Sep 19, 2019

spalger commented Oct 9, 2019

azasypkin commented Oct 10, 2019

soumendrak commented Oct 25, 2019

jkakavas commented Oct 25, 2019

kibanamachine commented Nov 12, 2019

azasypkin commented Jan 14, 2020

kibanamachine commented Mar 6, 2020

spalger commented Mar 6, 2020 • edited Loading

azasypkin commented Apr 14, 2020

kibanamachine commented Apr 14, 2020

azasypkin commented Apr 14, 2020

legrego commented Apr 14, 2020

azasypkin commented May 13, 2020

jkakavas commented May 13, 2020

spalger commented May 13, 2020

azasypkin commented May 14, 2020

azasypkin commented Mar 29, 2021

azasypkin commented Mar 29, 2021

kibanamachine commented Aug 22, 2019 •

edited

Loading

spalger commented Sep 11, 2019 •

edited

Loading

spalger commented Mar 6, 2020 •

edited

Loading