Call Prometheus, Loki and Jaeger via Grafana API #13973

dennis-ge · 2022-04-13T09:44:55Z

Description

Changes proposed in this pull request:

Calling prometheus, jaeger, and loki via the Grafana datasource API (see proxyGrafanaDashboard).
- By doing this, we get rid of kubectl forward (see kubectlportForward), and execute the integration tests more from a user perspective.
- The initial idea was to call the services via the apiserver proxy. This worked for prometheus but didn't work for loki and jaeger as istio authorizationpolicies and peerauthorization resources restricted the access to these services.
As a consequence, Grafana needs to be exposed before the first call to any datasource (prometheus, jaeger, and loki) is triggered. That's why the monitoring tests are now executed first in the standard test suite. The Grafana proxy is then reset at the end of the test.
Fixing type coercion
Use the new logging functions (info, error) in the observability tests
Extract Jaeger client calls to tracing directory

Pipelines:

Related issue(s)
Fixes #13375

…ercion

dennis-ge · 2022-04-13T09:45:06Z

/hold

netlify · 2022-04-13T09:50:08Z

✅ 🥰 Documentation preview ready! 🥰

Name	Link
🔨 Latest commit	`5c47024`
🔍 Latest deploy log	https://app.netlify.com/sites/kyma-project-docs-preview/deploys/62612f2b657cb90009aa8fd6
😎 Deploy Preview	https://deploy-preview-13973--kyma-project-docs-preview.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

dennis-ge · 2022-04-21T10:47:59Z

/test pre-main-kyma-integration-k3d-central-app-connectivity

dennis-ge · 2022-04-21T10:49:07Z

/test pre-main-kyma-gardener-gcp-eventing-upgrade
/test post-main-kyma-gardener-azure-alpha-eval

dennis-ge · 2022-04-22T09:30:43Z

/retest

dennis-ge · 2022-04-25T07:36:14Z

/retest

dennis-ge · 2022-04-25T08:47:57Z

/retest

dennis-ge · 2022-04-25T09:31:44Z

/test pre-main-kyma-integration-k3d-central-app-connectivity
/test pre-main-kyma-skr-eventing
/test pre-main-kyma-gardener-gcp-eventing

dennis-ge · 2022-04-25T09:32:25Z

/retest

skhalash · 2022-04-25T09:21:40Z

tests/fast-integration/monitoring/grafana.js

-    console.log('Checking redirect for kyma docs');
-    return await retryUrl(url, redirectURL, ignoreSSL, 403);
+async function setGrafanaProxy() {
+  if (getEnvOrDefault('KYMA_MAJOR_VERSION', '2') === '2') {


Do you think we still need it? I thought all Kyma 1.* remnants are gone

I would keep it for now as it is still used in other places in the code and remove them all in one follow-up PR.

tests/fast-integration/monitoring/grafana.js

skhalash · 2022-04-25T09:26:26Z

tests/fast-integration/monitoring/grafana.js

+    await updateProxyDeployment('--reverse-proxy=true', '--trusted-ip=0.0.0.0/0');
+    info('Checking grafana redirect to grafana URL');
+    const res = await checkGrafanaRedirect('https://grafana.', 200);
+    assert.isTrue(res, 'Grafana redirect to grafana landing page does not work!');


I don't think we should assert inside helper functions? Maybe throw an exception?

We also assert in other helper functions

Still think it hinders test glanceability, but let's keep it for now

tests/fast-integration/monitoring/grafana.js

tests/fast-integration/eventing-test/eventing-test.js

tests/fast-integration/package-lock.json

skhalash · 2022-04-25T11:16:07Z

/lgtm

skhalash · 2022-04-25T11:16:17Z

/approve

kyma-bot · 2022-04-25T11:16:24Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: skhalash, VladislavPaskar

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~tests/fast-integration/OWNERS~~ [skhalash]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

skhalash · 2022-04-25T12:19:40Z

/unhold

kyma-bot · 2022-04-25T12:40:53Z

@dennis-ge: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
post-main-kyma-gardener-azure-alpha-eval	`13b9bb6`	link	unknown	``
pre-main-kyma-integration-k3d-central-app-connectivity-compass	`9d5f5b3`	link	false	`/test pre-main-kyma-integration-k3d-central-app-connectivity-compass`
pre-main-kyma-skr-eventing	`9d5f5b3`	link	false	`/test pre-main-kyma-skr-eventing`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

dennis-ge · 2022-04-25T12:43:57Z

/retest-required

* add callServiceViaProxy, remove pf references, fix unexpected type coercion * add async await * add log statement * add return * call jaeger and loki via grafana * switch to new info and error functions * add await * first execute grafana tests * don't reset proxy * comment out reset proxy * add async await * improve logging * get dashboard id before querying grafana * remove dedicated httpsAgent * fix jaeger datasource id * remove opts args * outsource logic into proxyGrafanaDatasource * call prometheus via grafana * lint * cleanup * fix upgrade test * update eventing tests * refactor grafana tests * remove port forward from telemetry test * add async await * telemetry test before and after * update telemetry test * use mocha it * add setGrafanaProxy method * remove unused var * update mocha tests * adjust ordering of tests * remove resetting proxy * small refactoring of grafana.js * fix lint * change it to after * change to mocha describe * uncomment code * small refactoring * revert unintended changes * fix skr * apply suggestions

add callServiceViaProxy, remove pf references, fix unexpected type co…

5881151

…ercion

kyma-bot added the area/tests Issues or PRs related to tests label Apr 13, 2022

kyma-bot requested review from lindnerby and tillknuesting April 13, 2022 09:45

kyma-bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 13, 2022

dennis-ge added 6 commits April 13, 2022 12:06

add async await

97b58d0

add log statement

1dad628

add return

b839ed0

call jaeger and loki via grafana

a88f472

merge main into apiserver-proxy

a57528d

switch to new info and error functions

10c9374

dennis-ge marked this pull request as draft April 14, 2022 12:29

kyma-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 14, 2022

dennis-ge added 3 commits April 14, 2022 14:32

add await

4699070

first execute grafana tests

5869590

don't reset proxy

4b2f072

kyma-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Apr 14, 2022

dennis-ge added 9 commits April 14, 2022 15:04

comment out reset proxy

7ebe4eb

add async await

5270895

improve logging

34f0fcb

get dashboard id before querying grafana

8815bc5

remove dedicated httpsAgent

db48768

fix jaeger datasource id

4416802

remove opts args

6bc6f1f

outsource logic into proxyGrafanaDatasource

fd185f9

call prometheus via grafana

e4c92ea

dennis-ge added 2 commits April 22, 2022 10:58

small refactoring

6c0b2c4

revert unintended changes

eb4b0e8

kyma-bot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Apr 22, 2022

vpaskar approved these changes Apr 22, 2022

View reviewed changes

kyma-bot assigned vpaskar Apr 22, 2022

kyma-bot added the lgtm Looks good to me! label Apr 22, 2022

fix skr

722ea3e

kyma-bot removed the lgtm Looks good to me! label Apr 25, 2022

skhalash requested changes Apr 25, 2022

View reviewed changes

kyma-bot assigned skhalash Apr 25, 2022

apply suggestions

9d5f5b3

kyma-bot added the lgtm Looks good to me! label Apr 25, 2022

kyma-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 25, 2022

kyma-bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 25, 2022

kyma-bot merged commit 138bb64 into kyma-project:main Apr 25, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Call Prometheus, Loki and Jaeger via Grafana API #13973

Call Prometheus, Loki and Jaeger via Grafana API #13973

dennis-ge commented Apr 13, 2022 •

edited

dennis-ge commented Apr 13, 2022

netlify bot commented Apr 13, 2022 •

edited

dennis-ge commented Apr 21, 2022

dennis-ge commented Apr 21, 2022

dennis-ge commented Apr 22, 2022

dennis-ge commented Apr 25, 2022

dennis-ge commented Apr 25, 2022

dennis-ge commented Apr 25, 2022

dennis-ge commented Apr 25, 2022

skhalash Apr 25, 2022

dennis-ge Apr 25, 2022

skhalash Apr 25, 2022

dennis-ge Apr 25, 2022

skhalash Apr 25, 2022

skhalash commented Apr 25, 2022

skhalash commented Apr 25, 2022

kyma-bot commented Apr 25, 2022

skhalash commented Apr 25, 2022

kyma-bot commented Apr 25, 2022 •

edited

dennis-ge commented Apr 25, 2022

Call Prometheus, Loki and Jaeger via Grafana API #13973

Call Prometheus, Loki and Jaeger via Grafana API #13973

Conversation

dennis-ge commented Apr 13, 2022 • edited

dennis-ge commented Apr 13, 2022

netlify bot commented Apr 13, 2022 • edited

✅ 🥰 Documentation preview ready! 🥰

dennis-ge commented Apr 21, 2022

dennis-ge commented Apr 21, 2022

dennis-ge commented Apr 22, 2022

dennis-ge commented Apr 25, 2022

dennis-ge commented Apr 25, 2022

dennis-ge commented Apr 25, 2022

dennis-ge commented Apr 25, 2022

skhalash Apr 25, 2022

Choose a reason for hiding this comment

dennis-ge Apr 25, 2022

Choose a reason for hiding this comment

skhalash Apr 25, 2022

Choose a reason for hiding this comment

dennis-ge Apr 25, 2022

Choose a reason for hiding this comment

skhalash Apr 25, 2022

Choose a reason for hiding this comment

skhalash commented Apr 25, 2022

skhalash commented Apr 25, 2022

kyma-bot commented Apr 25, 2022

skhalash commented Apr 25, 2022

kyma-bot commented Apr 25, 2022 • edited

dennis-ge commented Apr 25, 2022

dennis-ge commented Apr 13, 2022 •

edited

netlify bot commented Apr 13, 2022 •

edited

kyma-bot commented Apr 25, 2022 •

edited