Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix percy target branch for PRs #43160

Merged
merged 11 commits into from Aug 14, 2019

Conversation

@liza-mae
Copy link
Member

commented Aug 12, 2019

After we enabled visual testing for each PR in #42989 I noticed the PR branch is used as the baseline which would cause each PR to have an unreviewed Percy check (failed Github check). I think the baseline for PRs should be master.

Relates to #33817

@liza-mae liza-mae requested a review from elastic/kibana-operations as a code owner Aug 12, 2019

@liza-mae liza-mae self-assigned this Aug 12, 2019

@liza-mae liza-mae requested review from spalger, nreese and tylersmalley Aug 12, 2019

@@ -25,5 +25,8 @@ const shortCommit = commit.slice(0, 8);

const isPr = process.env.JOB_NAME.includes('elastic+kibana+pull-request');

const { stdout: branch } = execa.sync('git', ['rev-parse', '--abbrev-ref', 'HEAD']);

This comment has been minimized.

Copy link
@tylersmalley

tylersmalley Aug 12, 2019

Member

Any reason for this not to be what is tracked in the package.json? https://github.com/elastic/kibana/blob/master/package.json#L15

This comment has been minimized.

Copy link
@spalger

spalger Aug 13, 2019

Member

I want to keep it separate so that feature branches can deviate from the base snapshots.

@elasticmachine

This comment has been minimized.

Copy link
Collaborator

commented Aug 12, 2019

@elasticmachine

This comment has been minimized.

Copy link
Collaborator

commented Aug 13, 2019

liza-mae added some commits Aug 13, 2019

@elasticmachine

This comment has been minimized.

Copy link
Collaborator

commented Aug 13, 2019

@liza-mae

This comment has been minimized.

Copy link
Member Author

commented Aug 13, 2019

So the branch baseline is origin/master and I was originally pointing to master using process.env.ghprbTargetBranch, but changing to PR_TARGET_BRANCH it returns the same.
Still trying to figure out why Percy is using origin/master.

The CI failure is related to Firefox smoke test and the Percy failure is on the lower left Kibana icon slight shift, same characteristics as other failures I have seen, hoping it is one root cause and we can fix, looking.

@liza-mae

This comment has been minimized.

Copy link
Member Author

commented Aug 13, 2019

So origin/master is being set automatically since Jenkins is fetching remote origin. Although according to Percy support it would be best to have this set to master and we would need to set env variable PERCY_BRANCH also.

@elasticmachine

This comment has been minimized.

Copy link
Collaborator

commented Aug 13, 2019

spalger added some commits Aug 13, 2019

@elasticmachine

This comment has been minimized.

Copy link
Collaborator

commented Aug 13, 2019

@spalger
Copy link
Member

left a comment

LGTM

Would you please send out an email once this is merged explaining what percy is, that regular contributors should check their email for an invite, and how they can request access if they don't have one.

@liza-mae

This comment has been minimized.

Copy link
Member Author

commented Aug 14, 2019

Sounds good @spalger -- will do :) thanks !

@liza-mae liza-mae merged commit fcba9a6 into elastic:master Aug 14, 2019

53 checks passed

API integration tests node scripts/functional_tests --config test/api_integration/config.js --bail --debug
Details
Browser tests yarn run grunt test:browser-ci
Details
Build kbn_tp_sample_panel_action yarn build
Details
CLA All commits in pull request signed
Details
Check core API changes node scripts/check_core_api_changes
Details
Check file casing node scripts/check_file_casing --quiet
Details
Check licenses node scripts/check_licenses --dev
Details
Firefox smoke test node scripts/functional_tests --bail --debug --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/kibana-firefoxSmoke/node/linux-immutable/install/kibana --include-tag smoke --config test/functional/config.firefox.js
Details
Functional tests / Group 1 yarn run grunt run:functionalTests_ciGroup1
Details
Functional tests / Group 10 yarn run grunt run:functionalTests_ciGroup10
Details
Functional tests / Group 11 yarn run grunt run:functionalTests_ciGroup11
Details
Functional tests / Group 12 yarn run grunt run:functionalTests_ciGroup12
Details
Functional tests / Group 2 yarn run grunt run:functionalTests_ciGroup2
Details
Functional tests / Group 3 yarn run grunt run:functionalTests_ciGroup3
Details
Functional tests / Group 4 yarn run grunt run:functionalTests_ciGroup4
Details
Functional tests / Group 5 yarn run grunt run:functionalTests_ciGroup5
Details
Functional tests / Group 6 yarn run grunt run:functionalTests_ciGroup6
Details
Functional tests / Group 7 yarn run grunt run:functionalTests_ciGroup7
Details
Functional tests / Group 8 yarn run grunt run:functionalTests_ciGroup8
Details
Functional tests / Group 9 yarn run grunt run:functionalTests_ciGroup9
Details
Internationalization check node scripts/i18n_check --ignore-missing
Details
Interpreter functional tests node scripts/functional_tests --config test/interpreter_functional/config.js --bail --debug --kibana-install-dir ./build/oss/kibana-8.0.0-SNAPSHOT-linux-x86_64
Details
Jest integration tests yarn run grunt test:jest_integration
Details
Jest tests yarn run grunt test:jest
Details
Kibana visual regression tests yarn run percy exec node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/kibana-visualRegression/node/linux-immutable/install/kibana --config test/visual_regression/config.ts
Details
Mocha tests node scripts/mocha
Details
Plugin functional tests node scripts/functional_tests --config test/plugin_functional/config.js --bail --debug --kibana-install-dir ./build/oss/kibana-8.0.0-SNAPSHOT-linux-x86_64
Details
Project tests yarn run grunt test:projects
Details
Type check node scripts/type_check
Details
TypeScript - all files belong to a TypeScript project node scripts/check_ts_projects
Details
Verify NOTICE.txt node scripts/notice --validate
Details
Verify dependency versions yarn run grunt verifyDependencyVersions
Details
X-Pack Chrome Functional tests / Group 1 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup1/node/linux-immutable/install/kibana --include-tag ciGroup1
Details
X-Pack Chrome Functional tests / Group 10 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup10/node/linux-immutable/install/kibana --include-tag ciGroup10
Details
X-Pack Chrome Functional tests / Group 2 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup2/node/linux-immutable/install/kibana --include-tag ciGroup2
Details
X-Pack Chrome Functional tests / Group 3 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup3/node/linux-immutable/install/kibana --include-tag ciGroup3
Details
X-Pack Chrome Functional tests / Group 4 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup4/node/linux-immutable/install/kibana --include-tag ciGroup4
Details
X-Pack Chrome Functional tests / Group 5 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup5/node/linux-immutable/install/kibana --include-tag ciGroup5
Details
X-Pack Chrome Functional tests / Group 6 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup6/node/linux-immutable/install/kibana --include-tag ciGroup6
Details
X-Pack Chrome Functional tests / Group 7 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup7/node/linux-immutable/install/kibana --include-tag ciGroup7
Details
X-Pack Chrome Functional tests / Group 8 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup8/node/linux-immutable/install/kibana --include-tag ciGroup8
Details
X-Pack Chrome Functional tests / Group 9 node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-ciGroup9/node/linux-immutable/install/kibana --include-tag ciGroup9
Details
X-Pack Jest node scripts/jest --ci --verbose
Details
X-Pack Mocha yarn test
Details
X-Pack SIEM cyclic dependency test node legacy/plugins/siem/scripts/check_circular_deps
Details
X-Pack firefox smoke test node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-firefoxSmoke/node/linux-immutable/install/kibana --include-tag smoke --config test/functional/config.firefox.js
Details
X-Pack visual regression tests yarn run percy exec node scripts/functional_tests --debug --bail --kibana-install-dir /var/lib/jenkins/workspace/elastic+kibana+pull-request/JOB/x-pack-visualRegression/node/linux-immutable/install/kibana --config test/visual_regression/config.js
Details
eslint node scripts/eslint --no-cache
Details
kibana-ci Build finished.
Details
percy/kibana Visual review automatically approved, no visual changes found.
Details
prbot:outdated run `node scripts/update_prs 43160` to update
prbot:release note labels
sasslint node scripts/sasslint
Details

@liza-mae liza-mae deleted the liza-mae:liza-fix/percy-branch branch Aug 14, 2019

liza-mae added a commit that referenced this pull request Aug 14, 2019

Fix percy target branch for PRs (#43160) (#43292)
* Fix percy target branch for PRs

* Add debug logging

* Try diff branch name

* Use PR_TARGET_BRANCH instead

* renable visReg jobs

* parse target branch from `branch_specifier`

* rename debugging print out

* set `PERCY_BRANCH` and `PERCY_TARGET_BRANCH`

* log PERCY_TARGET_BRANCH

* cleanup error messages

jloleysens added a commit to jloleysens/kibana that referenced this pull request Aug 15, 2019

Merge branch 'master' of github.com:elastic/kibana into feature/rollu…
…p-metrics-selectall

* 'master' of github.com:elastic/kibana: (50 commits)
  [Uptime] update monitor list configs for mobile view (elastic#43218)
  [APM] Local UI filters (elastic#41588)
  [Code] Upgrade ctags langserver (elastic#43252)
  [Code] show multiple definition results in panel (elastic#43249)
  Adds Metric Type to full screen launch tracking (elastic#42692)
  [Canvas] Convert Autocomplete to Typescript (elastic#42502)
  [telemetry] add spacesEnabled config back to xpack_main (elastic#43312)
  [ML] Adds DF Transform Analytics list to Kibana management (elastic#43151)
  Add TLS client authentication support. (elastic#43090)
  [csp] Telemetry for csp configuration (elastic#43223)
  [SIEM] Run Cypress Tests Against Elastic Cloud & Cypress Command Line / Reporting (elastic#42804)
  docs: add tip on agent config in a dt (elastic#43301)
  [ML] Adding bucket span estimator to new wizards (elastic#43288)
  disable flaky tests (elastic#43017)
  Fix percy target branch for PRs (elastic#43160)
  [ML] Adding post create job options (elastic#43205)
  Restore discover histogram selection triggering fetch (elastic#43097)
  Per panel time range (elastic#43153)
  [Infra UI] Add APM to Metadata Endpoint (elastic#42197)
  Sentence case copy changes (elastic#43215)
  ...

chrisronline added a commit to chrisronline/kibana that referenced this pull request Aug 15, 2019

Fix percy target branch for PRs (elastic#43160)
* Fix percy target branch for PRs

* Add debug logging

* Try diff branch name

* Use PR_TARGET_BRANCH instead

* renable visReg jobs

* parse target branch from `branch_specifier`

* rename debugging print out

* set `PERCY_BRANCH` and `PERCY_TARGET_BRANCH`

* log PERCY_TARGET_BRANCH

* cleanup error messages
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.