Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[kbn-es] await native realm setup, error if there are failures #36290

Merged
merged 14 commits into from May 10, 2019

Conversation

Projects
None yet
3 participants
@spalger
Copy link
Member

commented May 8, 2019

While trying to debug some transient es security failures we've been seeing I noticed that the native realm setup logic is outside of the promise chain and isn't guaranteed to be done before the promise returned by Cluster#start() resolves. Additionally, errors in that process do not bubble up, but are logged in a way that's too easy to miss.

This moves the nativeRealm setup to its own promise, along with the http port discovery, and consumes them as necessary.

@elasticmachine

This comment has been minimized.

Copy link

commented May 8, 2019

@tylersmalley

This comment has been minimized.

Copy link
Member

commented May 8, 2019

Looks like the port check is failing:

(node:1847) UnhandledPromiseRejectionWarning: TypeError [ERR_INVALID_URL]: Invalid URL: http://elastic:changeme@localhost:undefined
    at onParseError (internal/url.js:241:17)
    at parse (internal/url.js:257:3)
    at new URL (internal/url.js:332:5)
    at ConnectionPool.urlToHost (/mnt/c/elastic/kibana/node_modules/@elastic/elasticsearch/lib/ConnectionPool.js:390:12)
    at ConnectionPool.addConnection (/mnt/c/elastic/kibana/node_modules/@elastic/elasticsearch/lib/ConnectionPool.js:216:19)
    at new Client (/mnt/c/elastic/kibana/node_modules/@elastic/elasticsearch/index.js:104:27)
    at new NativeRealm (/mnt/c/elastic/kibana/packages/kbn-es/src/utils/native_realm.js:27:20)
    at _nativeRealmSetup._httpPort.then (/mnt/c/elastic/kibana/packages/kbn-es/src/cluster.js:272:27)
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:1847) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)
(node:1847) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Additionally, if we are getting rid of the try/catch in isSecurityEnabled. We will need to handle that path not being present in OSS and in turn not calling setPasswords

@spalger spalger removed the request for review from tylersmalley May 8, 2019

@spalger spalger added WIP and removed review labels May 8, 2019

@elasticmachine

This comment was marked as outdated.

Copy link

commented May 8, 2019

@spalger spalger requested a review from tylersmalley May 9, 2019

@spalger spalger added review and removed WIP labels May 9, 2019

@spalger

This comment has been minimized.

Copy link
Member Author

commented May 9, 2019

Fixed the port issue (typo) and filtered out 400 errors, which is what we get when xpack is not included since the _xpack path is parsed as an index name and the categories parameter is unknown triggering a Bad Request response.

@elasticmachine

This comment was marked as outdated.

Copy link

commented May 9, 2019

@elasticmachine

This comment was marked as outdated.

Copy link

commented May 9, 2019

@spalger

This comment has been minimized.

Copy link
Member Author

commented May 9, 2019

retest

@elasticmachine

This comment has been minimized.

Copy link

commented May 9, 2019

@tylersmalley
Copy link
Member

left a comment

LGTM, just one minor nit regarding the log message with the mocked bin.

@elasticmachine

This comment has been minimized.

Copy link

commented May 10, 2019

@spalger spalger merged commit 6ef9fbe into elastic:master May 10, 2019

43 checks passed

API integration tests node scripts/functional_tests --config test/api_integration/config.js --bail --debug
Details
Browser tests yarn run grunt test:browser-ci
Details
Build kbn_tp_sample_panel_action yarn build
Details
CLA All commits in pull request signed
Details
Check core API changes node scripts/check_core_api_changes
Details
Check file casing node scripts/check_file_casing --quiet
Details
Functional tests / Group 1 yarn run grunt run:functionalTests_ciGroup1
Details
Functional tests / Group 10 yarn run grunt run:functionalTests_ciGroup10
Details
Functional tests / Group 11 yarn run grunt run:functionalTests_ciGroup11
Details
Functional tests / Group 12 yarn run grunt run:functionalTests_ciGroup12
Details
Functional tests / Group 2 yarn run grunt run:functionalTests_ciGroup2
Details
Functional tests / Group 3 yarn run grunt run:functionalTests_ciGroup3
Details
Functional tests / Group 4 yarn run grunt run:functionalTests_ciGroup4
Details
Functional tests / Group 5 yarn run grunt run:functionalTests_ciGroup5
Details
Functional tests / Group 6 yarn run grunt run:functionalTests_ciGroup6
Details
Functional tests / Group 7 yarn run grunt run:functionalTests_ciGroup7
Details
Functional tests / Group 8 yarn run grunt run:functionalTests_ciGroup8
Details
Functional tests / Group 9 yarn run grunt run:functionalTests_ciGroup9
Details
Internationalization check node scripts/i18n_check --ignore-missing
Details
Interpreter functional tests node scripts/functional_tests --config test/interpreter_functional/config.js --bail --debug --kibana-install-dir ./build/oss/kibana-8.0.0-SNAPSHOT-linux-x86_64
Details
Jest integration tests yarn run grunt test:jest_integration
Details
Jest tests yarn run grunt test:jest
Details
Licenses yarn run grunt licenses
Details
Mocha tests node scripts/mocha
Details
Plugin functional tests node scripts/functional_tests --config test/plugin_functional/config.js --bail --debug --kibana-install-dir ./build/oss/kibana-8.0.0-SNAPSHOT-linux-x86_64
Details
Project tests yarn run grunt test:projects
Details
Server tests yarn run grunt test:server
Details
Type check node scripts/type_check
Details
TypeScript - all files belong to a TypeScript project node scripts/check_ts_projects
Details
Verify NOTICE.txt node scripts/notice --validate
Details
Verify dependency versions yarn run grunt verifyDependencyVersions
Details
X-Pack Functional tests / Group 1 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup1/node/immutable/install/kibana --include-tag ciGroup1
Details
X-Pack Functional tests / Group 2 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup2/node/immutable/install/kibana --include-tag ciGroup2
Details
X-Pack Functional tests / Group 3 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup3/node/immutable/install/kibana --include-tag ciGroup3
Details
X-Pack Functional tests / Group 4 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup4/node/immutable/install/kibana --include-tag ciGroup4
Details
X-Pack Functional tests / Group 5 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup5/node/immutable/install/kibana --include-tag ciGroup5
Details
X-Pack Functional tests / Group 6 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup6/node/immutable/install/kibana --include-tag ciGroup6
Details
X-Pack Functional tests / Group 7 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup7/node/immutable/install/kibana --include-tag ciGroup7
Details
X-Pack Jest node scripts/jest --ci --no-cache --verbose
Details
X-Pack Mocha yarn test
Details
eslint node scripts/eslint --no-cache
Details
kibana-ci Build finished.
Details
sasslint node scripts/sasslint
Details

spalger added a commit to spalger/kibana that referenced this pull request May 10, 2019

[kbn-es] await native realm setup, error if there are failures (elast…
…ic#36290)

* [kbn-es] await native realm setup, error if there are failures

* cleanup listener from started promise too

* fix typo

* log native realm setup errors on run and handle 400 when xpack isn't available

* avoid creating potentially unused promises

* avoid Promise constructor that's larger than necessary

* group similar operations, add some light comments

* set passwords in parallel so that logging is grouped

* update native realm tests

* update es_bin fixture to handle _xpack requests

* log started after server is listening

* remove unused mockEsBin args

* use more realistic http address log

spalger added a commit that referenced this pull request May 10, 2019

spalger added a commit to spalger/kibana that referenced this pull request May 10, 2019

@spalger spalger deleted the spalger:await-native-realm-setup branch May 10, 2019

@spalger spalger added the reverted label May 10, 2019

spalger added a commit that referenced this pull request May 20, 2019

[kbn/es] await native realm setup, error if there are failures (#36475)
* Revert "Revert "[kbn-es] await native realm setup, error if there are failures (#36290)""

This reverts commit a102ca1.

* [kbn/es] retry setPassword call if it fails

spalger added a commit to spalger/kibana that referenced this pull request May 20, 2019

[kbn/es] await native realm setup, error if there are failures (elast…
…ic#36475)

* Revert "Revert "[kbn-es] await native realm setup, error if there are failures (elastic#36290)""

This reverts commit a102ca1.

* [kbn/es] retry setPassword call if it fails

spalger added a commit that referenced this pull request May 21, 2019

[7.x] [kbn/es] await native realm setup, error if there are failures (#…
…36475) (#36722)

* Revert "Revert "[kbn-es] await native realm setup, error if there are failures (#36290)""

This reverts commit a102ca1.

* [kbn/es] retry setPassword call if it fails
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.