grafana-test fails - cannot look up grafana url successfull

[jmazzitelli]

@leandroberetta can you confirm if you see this error too?

I am running the grafana-test molecule test on OpenShift 4.7 with istio 1.11.

It is failing.

When I look at the Kiali ConfigMap while the test is running, I see this in it:

spec.
  external_services:
    grafana:
        health_check_url: https://grafana.istio-system.svc:3000
        in_cluster_url: http://wrong.url

Notice the in_cluster_url is still the "wrong.url", but at the time I saw this, the molecule test was trying to test that the health check succeeds. I believe it failed at this assert: https://github.com/kiali/kiali-operator/blob/master/molecule/grafana-test/converge.yml#L125

I ran the molecule test like this:

1. oc login to an 4.7 OpenShift
2. Install Istio 1.11
3. Log into the OpenShift image registry (via docker login or podman login - get the login command via make cluster-status)
4. Build the latest images and push to the cluster: make clean build cluster-push
5. Run make -e MOLECULE_SCENARIO="grafana-test" -e MOLECULE_USE_DEV_IMAGES="true" molecule-test

PR that fixes the test: kiali/kiali-operator#398
Backport PR: kiali/kiali-operator#399

[jmazzitelli]

here's the end of the ansible run - full log temporarily available here: https://raw.githubusercontent.com/jmazzitelli/kiali-molecule-test-logs/openshift/molecule-tests-2021-08-19_02-00-01/grafana-test.log:

    TASK [Results: /api/istio/status] **********************************************
    ok: [localhost] => {
        "msg": [
            {
                "is_core": false,
                "name": "grafana",
                "status": "Unreachable"
            }
        ]
    }
    
    TASK [Check if Grafana status is present] **************************************
    ok: [localhost] => (item={'name': 'grafana', 'status': 'Unreachable', 'is_core': False})
    
    TASK [Assert that there is no error related to Grafana] ************************
fatal: [localhost]: FAILED! => {
    "assertion": "grafana_health_url_error is not defined",
    "changed": false,
    "evaluated_to": false,
    "msg": "Assertion failed"
}
    
    PLAY RECAP *********************************************************************
    localhost                  : ok=68   changed=2    unreachable=0    failed=1    skipped=14   rescued=0    ignored=0

[jmazzitelli]

and here is the server logs:

"2021-08-18T23:50:38Z DBG [WRONG] URL discovery for service 'wrong', namespace 'url'...",
"2021-08-18T23:50:38Z DBG Rest perf config QPS: 175.000000 Burst: 200",
"2021-08-18T23:50:38Z DBG [WRONG] Discovery failed: routes.route.openshift.io \"wrong\" not found",
"2021-08-18T23:50:38Z DBG [GRAFANA] URL discovery failed: routes.route.openshift.io \"wrong\" not found",

[jmazzitelli]

I think I found a problem in the test itself. I'll submit a PR soon, once I confirm my theory.