[APM] E2E: Zero config for running e2e locally #59152

sorenlouv · 2020-03-03T15:01:52Z

Changes

Use env vars for cypress (instead of importing kibana.dev.yml)
Use --config to load kibana.e2e.yml instead of the user's kibana.dev.yml.
Added run-e2e.sh script to automatically setup environment and run tests

TODO:

apm-integration-testing should return a non-zero exit code if it encounters errors (necessary to stop subsequent steps) (feat: return the exit code of docker compose in case of failure apm-integration-testing#803)
Kibana should be started in --dev mode. This is currently not possible because it disables the --config flag.

Helpful error messages

sorenlouv · 2020-03-03T15:03:09Z

x-pack/legacy/plugins/apm/e2e/README.md

@@ -7,7 +7,7 @@
 ```shell
 $ git clone https://github.com/elastic/apm-integration-testing.git
 $ cd apm-integration-testing
-./scripts/compose.py start master --no-kibana --no-xpack-secure
+./scripts/compose.py start master --no-kibana --elasticsearch-port 9201 --apm-server-port 8201


Enabling Security and ES and APM Server on non-default ports to avoid clashing with local development

sorenlouv · 2020-03-03T15:04:14Z

x-pack/legacy/plugins/apm/e2e/README.md


 4. Start Kibana

 ```shell
 $ yarn kbn bootstrap
-$ yarn start --no-base-path --csp.strict=false
+$ yarn start --no-base-path --config x-pack/legacy/plugins/apm/e2e/ci/kibana.e2e.yml


Refer to e2e specific config file (instead of moving it around)

sorenlouv · 2020-03-03T15:04:43Z

x-pack/legacy/plugins/apm/e2e/README.md

 ```

-> Content Security Policy (CSP) Settings: Your Kibana instance must have the `csp.strict: false`.


This is now specified in kibana.e2e.yml. Not something the user has to worry about.

sorenlouv · 2020-03-03T15:05:38Z

x-pack/legacy/plugins/apm/e2e/ci/kibana.e2e.yml

@@ -0,0 +1,31 @@
+# Kibana
+server.port: 5701


Running kibana on a non-default port to avoid clashing with local development

sorenlouv · 2020-03-03T15:06:55Z

x-pack/legacy/plugins/apm/e2e/ci/kibana.e2e.yml

+xpack.apm.autocreateApmIndexPattern: true
+xpack.apm.ui.enabled: true
+xpack.apm.ui.transactionGroupBucketSize: 100
+xpack.apm.ui.maxTraceItems: 1000


Added all apm configurations to ensure they don't break without us noticing

sorenlouv · 2020-03-03T15:08:00Z

x-pack/legacy/plugins/apm/e2e/cypress.json

+  "env": {
+    "elasticsearch_username": "admin",
+    "elasticsearch_password": "changeme"
+  }


Provide credentials to Cypress via environment vars

sorenlouv · 2020-03-03T23:59:35Z

retest

v1v · 2020-03-04T09:07:12Z

x-pack/legacy/plugins/apm/e2e/ci/kibana.e2e.yml

+# Started via apm-integration-testing
+# ./scripts/compose.py start master --no-kibana --elasticsearch-port 9201 --apm-server-port 8201
+elasticsearch.hosts: http://localhost:9201
+elasticsearch.username: 'kibana_system_user'


kibana_system_user didn't work when I tried, so maybe admin could be a good one, what do you think?

Suggested change

elasticsearch.username: 'kibana_system_user'

elasticsearch.username: 'admin'

Kibana shouldn't connect to ES with superuser privileges. According to the docs kibana_system_user should exist.

I can confirm that kibana_system_user can access ES like:

curl --user kibana_system_user:changeme -X GET "localhost:9201/_security/user/_has_privileges?pretty" -H 'Content-Type: application/json' -d'{ "cluster": [ "monitor", "manage" ] }'

which returns:

{ "username" : "kibana_system_user", "has_all_requested" : false, "cluster" : { "monitor" : true, "manage" : false }, "index" : { }, "application" : { } }

Can you try running that and LMK if it works?

smith

👍 once conflicts resolved and up to date and tests pass.

x-pack/legacy/plugins/apm/e2e/ci/prepare-kibana.sh

Co-Authored-By: Victor Martinez <victormartinezrubio@gmail.com>

cachedout · 2020-03-24T09:36:56Z

Hi @sqren

I'm helping to triage high-priority issues for @elastic/observablt-robots . Just so I understand, are the TODO items listed above part of this PR or is the intention to address them in follow-up PRs?

sorenlouv · 2020-03-24T11:14:11Z

Hey @cachedout,

I've just pushed a commit that fixes most of the issues. Only remaining issue is that apm-integration-testing does not return a non-zero exit code upon error. This should be fixed in https://github.com/elastic/apm-integration-testing but not something that should block this PR.

I'll merge this as soon as CI is green

kuisathaverat · 2020-03-24T12:22:56Z

@sqren Do exist an issue for that on the https://github.com/elastic/apm-integration-testing? I do not see any. Would you mind fill one with the details of what happens?

sorenlouv · 2020-03-24T12:38:34Z

@kuisathaverat elastic/apm-integration-testing#800

sorenlouv · 2020-03-24T15:10:59Z

retest

v1v · 2020-03-24T15:29:56Z

x-pack/legacy/plugins/apm/e2e/ci/prepare-kibana.sh


 echo "3/3 Start Kibana ..."
 popd
 ## Might help to avoid FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 export NODE_OPTIONS="--max-old-space-size=4096"
-nohup node scripts/kibana --no-base-path --csp.strict=false --optimize.watch=false> kibana.log 2>&1 &
+nohup node scripts/kibana --config "${E2E_DIR}/ci/kibana.e2e.yml" --no-base-path --optimize.watch=false> kibana.log 2>&1 &


What's the difference with https://github.com/elastic/kibana/pull/59152/files#diff-ccfe59ee0ec66fe86f6596aaf1fce60cR24? Are those flags also required in the CI?

This starts in dev mode

node ./scripts/kibana --no-base-path --dev --no-dev-config --config x-pack/legacy/plugins/apm/e2e/ci/kibana.e2e.yml

This starts in prod mode

node scripts/kibana --no-base-path --optimize.watch=false --config "${E2E_DIR}/ci/kibana.e2e.yml"

I think we should choose whatever is faster on CI. I guess we can test by running it 100 times or so on the CI in each mode.
I imagine dev compiles faster (because it doesn't need to compress resources) but prod has faster browser load times because assets are smaller. So could go both ways.

The reason I opted for using dev mode locally is because this is what developers is normally running, and I saw some issues when running prod mode without cleaning up after having run dev mode.

Thanks for the clarification, I'd say to use the same as the local one to avoid any misleading, then it will be easy to reproduce it anyway. Thanks again

sorenlouv · 2020-03-24T15:43:24Z

@elasticmachine merge upstream

then we don't need to trap the errorlevel

kibanamachine · 2020-03-26T00:12:44Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 46b518c

History

💚 Build #35850 succeeded 46b518c
💚 Build #35830 succeeded e3d0391
💔 Build #35796 failed e3d0391
💚 Build #35772 succeeded 1bd70fa
💚 Build #30774 succeeded bdff7d0

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

sorenlouv requested a review from a team as a code owner March 3, 2020 15:01

sorenlouv requested a review from v1v March 3, 2020 15:02

sorenlouv added release_note:skip Skip the PR/issue when compiling release notes v7.7.0 labels Mar 3, 2020

sorenlouv commented Mar 3, 2020

View reviewed changes

sorenlouv force-pushed the use-env-variables-e2e branch from 3ca9e26 to 1dfe5a3 Compare March 3, 2020 15:03

sorenlouv commented Mar 3, 2020

View reviewed changes

v1v approved these changes Mar 4, 2020

View reviewed changes

smith approved these changes Mar 18, 2020

View reviewed changes

sorenlouv changed the title ~~[APM] E2E: Use env vars over kibana.yml~~ [APM] E2E: Improve local DX by running all with single command Mar 20, 2020

sorenlouv changed the title ~~[APM] E2E: Improve local DX by running all with single command~~ [APM] E2E: Zero config for running e2e locally Mar 20, 2020

v1v reviewed Mar 21, 2020

View reviewed changes

x-pack/legacy/plugins/apm/e2e/ci/prepare-kibana.sh Outdated Show resolved Hide resolved

v1v added a commit to v1v/kibana that referenced this pull request Mar 21, 2020

cherry-pick some changes from elastic#59152

94814da

v1v added a commit to v1v/kibana that referenced this pull request Mar 21, 2020

cherry-pick some changes from elastic#59152

7139757

v1v added a commit to v1v/kibana that referenced this pull request Mar 21, 2020

cherry-pick from elastic#59152

e18c9ed

v1v added a commit to v1v/kibana that referenced this pull request Mar 22, 2020

cherry-pick from elastic#59152

26d1761

sorenlouv and others added 5 commits March 24, 2020 09:13

[APM] E2E: Use env vars over kibana.yml

51083c0

Add ./run-e2e script

27cf088

Add ./run-e2e script

5d456e9

Comment out debug stuff

3f2d266

Update x-pack/legacy/plugins/apm/e2e/ci/prepare-kibana.sh

d0b8706

Co-Authored-By: Victor Martinez <victormartinezrubio@gmail.com>

Fix cypress issues

4e8d51a

sorenlouv force-pushed the use-env-variables-e2e branch from e2dbda9 to 4e8d51a Compare March 24, 2020 11:12

Improve formatting

1bd70fa

Add --no-dev-config to kibana command

e3d0391

v1v reviewed Mar 24, 2020

View reviewed changes

v1v added a commit to v1v/kibana that referenced this pull request Mar 24, 2020

cherry-pick from elastic#59152

d5aa1db

Merge branch 'master' into use-env-variables-e2e

46b518c

v1v added a commit to v1v/kibana that referenced this pull request Mar 24, 2020

Cherry-pick from elastic#59152

bb22a3a

then we don't need to trap the errorlevel

sorenlouv merged commit 0fc0440 into elastic:master Mar 24, 2020

sorenlouv deleted the use-env-variables-e2e branch March 24, 2020 19:28

sorenlouv mentioned this pull request Mar 24, 2020

[7.x] [APM] E2E: Zero config for running e2e locally (#59152) #61146

Merged

sorenlouv added a commit that referenced this pull request Mar 24, 2020

[APM] E2E: Zero config for running e2e locally (#59152) (#61146)

ad57ddb

v1v mentioned this pull request Mar 30, 2020

ci(jenkins): use e2e in the CI #61803

Merged

2 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[APM] E2E: Zero config for running e2e locally #59152

[APM] E2E: Zero config for running e2e locally #59152

sorenlouv commented Mar 3, 2020 •

edited

Loading

sorenlouv Mar 3, 2020

sorenlouv Mar 3, 2020

sorenlouv Mar 3, 2020

sorenlouv Mar 3, 2020

sorenlouv Mar 3, 2020

sorenlouv Mar 3, 2020 •

edited

Loading

sorenlouv commented Mar 3, 2020

v1v Mar 4, 2020

sorenlouv Mar 4, 2020

sorenlouv Mar 4, 2020

smith left a comment

cachedout commented Mar 24, 2020

sorenlouv commented Mar 24, 2020

kuisathaverat commented Mar 24, 2020

sorenlouv commented Mar 24, 2020

sorenlouv commented Mar 24, 2020

v1v Mar 24, 2020

sorenlouv Mar 24, 2020 •

edited

Loading

v1v Mar 24, 2020

sorenlouv commented Mar 24, 2020

kibanamachine commented Mar 26, 2020

		```

		> Content Security Policy (CSP) Settings: Your Kibana instance must have the `csp.strict: false`.

	elasticsearch.username: 'kibana_system_user'
	elasticsearch.username: 'admin'

[APM] E2E: Zero config for running e2e locally #59152

[APM] E2E: Zero config for running e2e locally #59152

Conversation

sorenlouv commented Mar 3, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sorenlouv Mar 3, 2020 • edited Loading

Choose a reason for hiding this comment

sorenlouv commented Mar 3, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

smith left a comment

Choose a reason for hiding this comment

cachedout commented Mar 24, 2020

sorenlouv commented Mar 24, 2020

kuisathaverat commented Mar 24, 2020

sorenlouv commented Mar 24, 2020

sorenlouv commented Mar 24, 2020

Choose a reason for hiding this comment

sorenlouv Mar 24, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sorenlouv commented Mar 24, 2020

kibanamachine commented Mar 26, 2020

💚 Build Succeeded

History

sorenlouv commented Mar 3, 2020 •

edited

Loading

sorenlouv Mar 3, 2020 •

edited

Loading

sorenlouv Mar 24, 2020 •

edited

Loading