Fix k8s overview dashboard

[ChrsMark]

What does this PR do?

Fixes k8s dashboard as part of #2159. Porting PR for elastic/beats#30913 (+elastic/beats#30986):

Closes #2906 too.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-03-30T10:10:09.988+0000

• Duration: 29 min 22 sec

Test stats 🧪

Test	Results
Failed	0
Passed	116
Skipped	0
Total	116

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[ChrsMark]

@mtojek any ideas for the failure on https://beats-ci.elastic.co/blue/organizations/jenkins/Ingest-manager%2Fintegrations/detail/PR-2876/3/pipeline#step-793-log-311?

[mtojek]

Did you check Kibana version in the dashboard/visualizations JSON files?

EDIT:

I suspect that you used/dumped using newer Kibana and now testing with 7.16.0.

[ChrsMark]

Did you check Kibana version in the dashboard/visualizations JSON files?

EDIT:

I suspect that you used/dumped using newer Kibana and now testing with 7.16.0.

Yes this is intentional. The dashboards I build are leveraging 8.2 Kibana's features. Is this supported in testing or I have to wait?

[mtojek]

Yes this is intentional. The dashboards I build are leveraging 8.2 Kibana's features. Is this supported in testing or I have to wait?

I guess that it means that the package won't be supported by older Kibanas and you need to update the Kibana version constraint. Dashboards are part of the package.

[ChrsMark]

Isn't https://github.com/elastic/integrations/pull/2876/files#diff-d20d8086b58468701d8a944fc3ead31ddcc098054affdcc335b69464b4c16c54R13 enough? Or I should completely remove the 7.16.0 part, this would lead to making the package backwards incompatible with 7.x from now on?

[mtojek]

Yes, it's a breaking change for dashboards anyway. In this case you need to drop the 7.x part at all.

[ChrsMark]

@elastic/obs-cloudnative-monitoring are we ok with such change, removing the backwards compatibility with 7.x?

[ChrsMark]

@mtojek is there any reference about such changes' sanity? I'm wondering how safe it is to just remove the 7.16 compatibility at this point but on the other hand we cannot just stick not doing the new changes. Do we have a specific plan for such cases so as to be covered in case we need a fix for 7.x?

[mtojek]

There are teams which already decided to support only the 8.x packages and, in case there is a necessity to release a bugfix, they push modified packages directly to Package Storage. Fortunately, these are really rare cases. The idea is to prevent overusing backporting and just design features to be backward compatible.

There are open discussions around those issues and due to low priority, we don't look into them actively. I can give you references to those if you want to take a look and comment.

[ChrsMark]

There are teams which already decided to support only the 8.x packages and, in case there is a necessity to release a bugfix, they push modified packages directly to Package Storage. Fortunately, these are really rare cases. The idea is to prevent overusing backporting and just design features to be backward compatible.

There are open discussions around those issues and due to low priority, we don't look into them actively. I can give you references to those if you want to take a look and comment.

I see, thanks for the extra info.

[ChrsMark]

As discussed internally with @elastic/obs-cloudnative-monitoring we move forward with releasing the package for version ^8.2 and if we need a bugfix for previous version of stack we will use direct branching and ad-hoc patch release of package.

For example at the moment we are on 1.17.3 version of package which is compatible with kibana.version: "^7.16.0 || ^8.0.0". So after this PR we will ship 1.18.0 versions of package with kibana.version: "^8.2.0" and if we need a bugfix compatible with previous versions we would branch out manually from 1.17.3 and ship 1.17.4 directly.

[mtojek]

[ChrsMark]

@tetianakravchenko @MichaelKatsoulis this PR is ready for review :).

[tetianakravchenko]

LGTM!

[ChrsMark]

CI is failing because of #2906. I added the missing field here so this PR close #2906 too.