bug 1652224. Fix defaultIndex for Kibana uprade

[jcantrill]

This bug fixes https://bugzilla.redhat.com/show_bug.cgi?id=1652224 by making sure their exists a current kibana config object for a user's kibana index.

Started here since master CI is not functional. Will need to be cherrypicked foward into master

blocked by fabric8io/openshift-elasticsearch-plugin#166

[jcantrill]

/hold

[richm]

lgtm

[jcantrill]

/hold cancel

[jcantrill]

@richm This will need re-approval assuming the tests pass again

[richm]

lgtm - have you built the downstream release yet? We'll need that before this PR can be merged

[jcantrill]

@richm I have not

[richm]

/lgtm

[jcantrill]

/cherrypick master

[openshift-cherrypick-robot]

@jcantrill: new pull request created: #1481

In response to this:

/cherrypick master

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[ayyappansubbiah]

Hello, i did delete the .kibana INDEX but still experiencing the same issue. What I see is there is no default INDEX pattern. When I add the .operations.* and save it on the "advanced settings" on Kibana console, i repeatedly receiving the error message "Config: Error 404 Not Found: [doc][config:5.6.16]: document missing: [document_missing_exception] [doc][config:5.6.16]: document missing, with { index_uuid" with new kibana index UUID. Please advise.

[jcantrill]

Hello, i did delete the .kibana INDEX but still experiencing the same issue. What I see is there is no default INDEX pattern. When I add the .operations.* and save it on the "advanced settings" on Kibana console, i repeatedly receiving the error message "Config: Error 404 Not Found: [doc][config:5.6.16]: document missing: [document_missing_exception] [doc][config:5.6.16]: document missing, with { index_uuid" with new kibana index UUID. Please advise.

You removed the index created by Kibana which is used to seed all user's indices and thus the error 404 not found. Try deleting the kibana pod which should cause it to recreate the index

[ayyappansubbiah]

Thanks for the update. After deleting the index, it does create a new KIBANA index when the "defaultIndex" is set to ".operations.*", but i still get the same error. I terminated the KIBANA pod per your recommendation, still experiencing same error. Any thoughts?

[ayyappansubbiah]

can you confirm if the certs on the KIBANA pod located under "/etc/kibana/keys" should be the copy of the certs on ES pod located "/etc/elasticsearch/secret/"? I see they a different when i did comparison as below.

[c184241@nagios01 kibana]$ openssl x509 -noout -modulus -in /u/ossadm/c184241/kibana/kibana_certs/cert | openssl md5
(stdin)= 2c445ac59ac4b5bdb72cd741b87d4004
[c184241@nagios01 kibana]$ openssl x509 -noout -modulus -in /u/ossadm/c184241/kibana/es_certs/admin-cert | openssl md5
(stdin)= 9fa3d5d29f34fe4c43ceddfa549e3ce6

If yes, is this may be the reason for the HTTP 404 errors?
Thanks

[acidwars]

I have the same error as @ayyappansubbiah . I've tried to delete the whole openshift-logging project several times and redeployed with Ansible. The result is always the "404 not found" error message:
Config: Error 404 Not Found: [doc][config:5.6.16]: document missing: [document_missing_exception] [doc][config:5.6.16]: document missing, with { index_uuid="0XCBjOihRHCvQyHWI-EAiQ" & shard="0" & …