Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jenkins pipeline with parallel cigroups #45285

Merged

Conversation

@brianseeders
Copy link
Contributor

commented Sep 10, 2019

Implement Jenkins Pipeline (i.e. Jenkinsfile) and run oss/x-pack ciGroups on in parallel on two machines, bringing the total number of jenkins agents per build down to four.

There are many changes that are temporary (such as anywhere you see $IS_PIPELINE_JOB), while we are in a transition period. Once all jobs are converted over to pipeline jobs, those can be cleaned up.

Also see #44925 for a discussion around visibility into logs/build status/etc, which is the biggest downside to this work.

@brianseeders brianseeders requested a review from elastic/kibana-operations Sep 10, 2019

@brianseeders brianseeders self-assigned this Sep 10, 2019

@brianseeders brianseeders added this to In progress in CI At Scale via automation Sep 10, 2019

@elasticmachine

This comment has been minimized.

Copy link
Contributor

commented Sep 10, 2019

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 10, 2019

@brianseeders

This comment has been minimized.

Copy link
Contributor Author

commented Sep 10, 2019

pipeline test

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 10, 2019

💚 Build Succeeded

@brianseeders

This comment has been minimized.

Copy link
Contributor Author

commented Sep 10, 2019

pipeline test

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 10, 2019

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 10, 2019

💔 Build Failed

@spalger

This comment has been minimized.

Copy link
Member

commented Sep 10, 2019

Failure handling looks good 👍

@brianseeders

This comment has been minimized.

Copy link
Contributor Author

commented Sep 10, 2019

pipeline test

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 10, 2019

@spalger
Copy link
Member

left a comment

Ultimately I think we should be good to go here, but I do have some nitpicks that we can move to follow up issues if you prefer.

Jenkinsfile Outdated Show resolved Hide resolved

kbnBranch="$(jq -r .branch "$KIBANA_DIR/package.json")"
export KIBANA_PKG_BRANCH="$kbnBranch"
source src/dev/ci_setup/setup_env.sh true

This comment has been minimized.

Copy link
@spalger

spalger Sep 10, 2019

Member

What does true do here?

This comment has been minimized.

Copy link
@brianseeders

brianseeders Sep 10, 2019

Author Contributor

I split out setup_env, because I need to set up all of the environment stuff every time sh is called in the Jenkinsfile.

I was originally having issues with the parallel workers each trying to re-install node/yarn, so I introduced an argument to skip node/yarn installing, and just set up the env. But now that I look at the script again, I'm not sure why that was happening in the first place. Will take a second look

src/dev/ci_setup/setup.sh Show resolved Hide resolved
src/dev/jest/integration_tests/junit_reporter.test.js Outdated Show resolved Hide resolved
@@ -32,6 +32,12 @@ export default function ({ getService, getPageObjects }) {
const loadingScreenShown = () =>
testSubjects.existOrFail('kbnLoadingMessage');

const getKibanaUrl = path => {
const host = process.env.TEST_KIBANA_HOST || 'localhost';

This comment has been minimized.

Copy link
@spalger

spalger Sep 10, 2019

Member

I know it's silly, but generally prefer to avoid creating properly formatted strings when there's a perfectly good format function available:

return url.format({
  protocol: 'http:',
  hostname: process.env.TEST_KIBANA_HOST || 'localhost',
  port: process.env.TEST_KIBANA_PORT || '5620',
  path
})

A lot of these formats have tricky little edge cases that pop up over time.

test/scripts/jenkins_build_kibana.sh Show resolved Hide resolved
@brianseeders

This comment has been minimized.

Copy link
Contributor Author

commented Sep 10, 2019

pipeline test

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 10, 2019

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 10, 2019

💔 Build Failed

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 11, 2019

@brianseeders

This comment has been minimized.

Copy link
Contributor Author

commented Sep 11, 2019

pipeline test

sendKibanaMail()
}

def sendInfraMail() {

This comment has been minimized.

Copy link
@tylersmalley

tylersmalley Sep 11, 2019

Member

Wonder if infra still wants these emails.

@@ -16,13 +16,16 @@ export const registerHelpers = (supertest) => {
const addCluster = (name = REMOTE_CLUSTER_NAME) => {
remoteClustersCreated.push(name);

// TODO how could this get pulled out of the ES config instead?

This comment has been minimized.

Copy link
@tylersmalley

tylersmalley Sep 11, 2019

Member

Probably the only way would be to grab it from stdout during startup since it increments if the port isn't available.

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 11, 2019

💔 Build Failed

@brianseeders brianseeders force-pushed the brianseeders:jenkins-pipeline-cigroups-parallel branch from 1ebf6c2 to ae6f73d Sep 11, 2019

@brianseeders

This comment has been minimized.

Copy link
Contributor Author

commented Sep 11, 2019

pipeline test

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 11, 2019

💔 Build Failed

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 11, 2019

@brianseeders brianseeders force-pushed the brianseeders:jenkins-pipeline-cigroups-parallel branch from ae6f73d to 4ffe6fd Sep 11, 2019

@brianseeders brianseeders force-pushed the brianseeders:jenkins-pipeline-cigroups-parallel branch from 3fef546 to 3253cee Sep 11, 2019

@brianseeders

This comment has been minimized.

Copy link
Contributor Author

commented Sep 11, 2019

pipeline test

@elasticmachine

This comment was marked as outdated.

Copy link
Contributor

commented Sep 11, 2019

@elasticmachine

This comment has been minimized.

Copy link
Contributor

commented Sep 11, 2019

💚 Build Succeeded

@spalger spalger merged commit 27d23c4 into elastic:master Sep 11, 2019

40 of 55 checks passed

percy/kibana 9 visual changes need review
Details
X-Pack Chrome Functional tests / Group 4 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup4/node/linux-immutable/install/kibana --include-tag ciGroup4
Details
X-Pack Chrome Functional tests / Group 6 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup6/node/linux-immutable/install/kibana --include-tag ciGroup6
Details
X-Pack Chrome Functional tests / Group 7 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup7/node/linux-immutable/install/kibana --include-tag ciGroup7
Details
Firefox smoke test node scripts/functional_tests --bail --debug --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/kibana-firefoxSmoke/node/linux-immutable/install/kibana --include-tag smoke --config test/functional/config.firefox.js
Details
Jest tests yarn run grunt test:jest
Details
X-Pack Chrome Functional tests / Group 1 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup1/node/linux-immutable/install/kibana --include-tag ciGroup1
Details
X-Pack Chrome Functional tests / Group 10 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup10/node/linux-immutable/install/kibana --include-tag ciGroup10
Details
X-Pack Chrome Functional tests / Group 2 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup2/node/linux-immutable/install/kibana --include-tag ciGroup2
Details
X-Pack Chrome Functional tests / Group 3 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup3/node/linux-immutable/install/kibana --include-tag ciGroup3
Details
X-Pack Chrome Functional tests / Group 5 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup5/node/linux-immutable/install/kibana --include-tag ciGroup5
Details
X-Pack Chrome Functional tests / Group 8 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup8/node/linux-immutable/install/kibana --include-tag ciGroup8
Details
X-Pack Chrome Functional tests / Group 9 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup9/node/linux-immutable/install/kibana --include-tag ciGroup9
Details
X-Pack firefox smoke test node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-firefoxSmoke/node/linux-immutable/install/kibana --include-tag smoke --config test/functional/config.firefox.js
Details
X-Pack visual regression tests yarn run percy exec node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-visualRegression/node/linux-immutable/install/kibana --config test/visual_regression/config.js
Details
API integration tests node scripts/functional_tests --config test/api_integration/config.js --bail --debug
Details
Browser tests yarn run grunt test:browser-ci
Details
Build kbn_tp_sample_panel_action yarn build
Details
CLA All commits in pull request signed
Details
Check core API changes node scripts/check_core_api_changes
Details
Check file casing node scripts/check_file_casing --quiet
Details
Check licenses node scripts/check_licenses --dev
Details
Functional tests / Group 1 yarn run grunt run:functionalTests_ciGroup1
Details
Functional tests / Group 10 yarn run grunt run:functionalTests_ciGroup10
Details
Functional tests / Group 11 yarn run grunt run:functionalTests_ciGroup11
Details
Functional tests / Group 12 yarn run grunt run:functionalTests_ciGroup12
Details
Functional tests / Group 2 yarn run grunt run:functionalTests_ciGroup2
Details
Functional tests / Group 3 yarn run grunt run:functionalTests_ciGroup3
Details
Functional tests / Group 4 yarn run grunt run:functionalTests_ciGroup4
Details
Functional tests / Group 5 yarn run grunt run:functionalTests_ciGroup5
Details
Functional tests / Group 6 yarn run grunt run:functionalTests_ciGroup6
Details
Functional tests / Group 7 yarn run grunt run:functionalTests_ciGroup7
Details
Functional tests / Group 8 yarn run grunt run:functionalTests_ciGroup8
Details
Functional tests / Group 9 yarn run grunt run:functionalTests_ciGroup9
Details
Internationalization check node scripts/i18n_check --ignore-missing
Details
Interpreter functional tests node scripts/functional_tests --config test/interpreter_functional/config.js --bail --debug --kibana-install-dir ./build/oss/kibana-8.0.0-SNAPSHOT-linux-x86_64
Details
Jest integration tests yarn run grunt test:jest_integration
Details
Kibana visual regression tests yarn run percy exec node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/kibana-visualRegression/node/linux-immutable/install/kibana --config test/visual_regression/config.ts
Details
Mocha tests node scripts/mocha
Details
Plugin functional tests node scripts/functional_tests --config test/plugin_functional/config.js --bail --debug --kibana-install-dir ./build/oss/kibana-8.0.0-SNAPSHOT-linux-x86_64
Details
Project tests yarn run grunt test:projects
Details
Type check node scripts/type_check
Details
TypeScript - all files belong to a TypeScript project node scripts/check_ts_projects
Details
Verify NOTICE.txt node scripts/notice --validate
Details
Verify dependency versions yarn run grunt verifyDependencyVersions
Details
X-Pack Jest node scripts/jest --ci --verbose
Details
X-Pack Mocha yarn test
Details
X-Pack SIEM cyclic dependency test node legacy/plugins/siem/scripts/check_circular_deps
Details
elasticsearch-ci/docs Build finished.
Details
eslint node scripts/eslint --no-cache
Details
kibana-ci Build finished.
Details
prbot:outdated run `node scripts/update_prs 45285` to update
prbot:release note labels
prbot:release version labels
sasslint node scripts/sasslint
Details

CI At Scale automation moved this from In progress to Done Sep 11, 2019

spalger added a commit that referenced this pull request Sep 11, 2019
Jenkins pipeline with parallel cigroups (#45285)
* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)""

This reverts commit 078ac28.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem

(cherry picked from commit 27d23c4)

# Conflicts:
#	src/dev/ci_setup/setup.sh
#	src/dev/failed_tests/cli.js
spalger added a commit that referenced this pull request Sep 11, 2019
Jenkins pipeline with parallel cigroups (#45285)
* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)""

This reverts commit 078ac28.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem

(cherry picked from commit 27d23c4)

# Conflicts:
#	src/dev/ci_setup/setup.sh
#	src/dev/failed_tests/cli.js
#	test/plugin_functional/test_suites/core_plugins/applications.js
#	test/scripts/jenkins_xpack_ci_group.sh
spalger added a commit that referenced this pull request Sep 11, 2019
Jenkins pipeline with parallel cigroups (#45285)
* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)""

This reverts commit 078ac28.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem

(cherry picked from commit 27d23c4)
spalger added a commit to spalger/kibana that referenced this pull request Sep 13, 2019
Jenkins pipeline with parallel cigroups (elastic#45285)
* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (elastic#44584)""

This reverts commit 078ac28.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem

(cherry picked from commit 27d23c4)
spalger added a commit to spalger/kibana that referenced this pull request Sep 13, 2019
Jenkins pipeline with parallel cigroups (elastic#45285)
* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (elastic#44584)""

This reverts commit 078ac28.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem

(cherry picked from commit 27d23c4)
spalger added a commit to spalger/kibana that referenced this pull request Sep 13, 2019
Jenkins pipeline with parallel cigroups (elastic#45285)
* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (elastic#44584)""

This reverts commit 078ac28.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem

(cherry picked from commit 27d23c4)
spalger added a commit to spalger/kibana that referenced this pull request Sep 13, 2019
Jenkins pipeline with parallel cigroups (elastic#45285)
* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (elastic#44584)""

This reverts commit 078ac28.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem

(cherry picked from commit 27d23c4)
spalger added a commit that referenced this pull request Sep 13, 2019
[6.8] Jenkins pipeline with parallel cigroups (#45285) (#45599)
* Jenkins pipeline with parallel cigroups (#45285)

* Pipeline

* WIP some work for parallelization with ciGroups

* Fix xpack kibana install dir, and add some debugging

* Attempt to quick fix a few tests

* Revert "Revert "Revert "[ci] compress jobs for CI stability" (#44584)""

This reverts commit 078ac28.

* Recombine test groups, and try runbld again

* Mostly cleanup, and fix failed_tests reporting to hopefully work for both pipeline and non-pipeline

* Fix typo in shell script

* Remove some debug code

* Add support for changing es transport.port during testing via TEST_ES_TRANSPORT_PORT

* Fix test that uses hard-coded es transport port and add it back in to parallel groups

* Disable checks reporter again for now

* Set env var for TEST_ES_TRANSPORT_PORT in pipeline

* Update Jenkinsfile for shorter testrunner labels

* Fix another hard-coded transport port

* Fix a new test with hard-coded URLs

* Jenkinsfile cleanup and fix one of the groups

* Fix double slash

* Testing vault credentials on jenkins server

* Add a non-existent credential

* Revert "Add a non-existent credential"

This reverts commit 0dc234c.

* Try github-checks-reporter again

* github-checks-reporter should only run for elastic/kibana, forks won't work

* Clean up some debug code

* Changing names around to try to make BlueOcean UI a little better

* Add more stages

* Make some changes to stage structure to mirror a nested example from CloudBees

* Handle TODOs, and some cleanup in Jenkinsfile

* Pass GIT_BRANCH when started without GHPRB, fix branch check

* Fix mailer problem and add code that ensures all tests are in cigroups back in

* Test adding worker/job name to junit report paths

* Remove some duplication from ci_setup scripts

* Fix unit test that uses junit path

* Don't reinstall node every time setup_env is run

* Fix yarn install logic

* Fix another unit test that uses junit output dir

* Download latest ES snapshot after kibana builds

* Make sure junit reports are always processed

* Add two failing tests for testing purposes

* Add support to Jenkinsfile for kibana build e-mails

* Remove some debug code for email sending

* Change JOB env handling in junit paths and move it to a sub-directory

* Revert "Add two failing tests for testing purposes"

This reverts commit 5715203.

* Fix junit report path in test

* Don't send kibana emails on build abort

* Address PR feedback, formatting and use built-in url formatting library

* Fix path formatting for functional test

* Add email sending back in to Jenkinsfile

* Fix another unit test with path problem

(cherry picked from commit 27d23c4)

* remove reference to extract_bootstrap_cache.sh

* remove extra jobs from Jenkinsfile

* increment leadfoot server port for parallel workers

* add worker index to port number, rather than concatenating with it

* well, for some reason the PR job wasn't triggered for 351eaaf...

* huh, PR job wasn't triggered for 1b99c69 either...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.