[Monitoring] "Internal Monitoring" deprecation warning #72020

igoristic · 2020-07-16T09:27:07Z

Resolves: #51718

Detecting the type of .monitoring-* indices in their cluster to assume which type of method of monitoring their using.

If they don't have any -mb- tags in their monitoring indices we show the following message:

It appears you are using "Internal Collection" for Stack Monitoring.
This method of monitoring will no longer be supported in the next major release (8.0.0).
Please follow the steps in setup mode to start monitoring with Metricbeat.

If for some reason they have both types, we instead show:

It appears you are using both Metricbeat and "Internal Collection" for Stack Monitoring.
only Metricbeat type monitoring will be supported in the next major release (8.0.0).
Please follow the steps in setup mode to migrate the rest of the monitoring to Metricbeat.

We don't show anything if they're on Cloud (for now)

elasticmachine · 2020-07-16T09:27:10Z

Pinging @elastic/stack-monitoring (Team:Monitoring)

…nal-monitoring-deprecation

chrisronline

Looking good! Nice work so far! Left a few notes

chrisronline · 2020-07-16T17:40:54Z

x-pack/plugins/monitoring/public/lib/internal_monitoring_toasts.tsx

+  });
+};
+
+export const showInternalMonitoringToast = ({


Just curious, why did you decide to go with a toast instead of a EuiCallout at the top of the page?

I think it's less invasive. Also I feel like EuiCallout is too close to the enter setup button which makes it appear as if they're somehow associated.

x-pack/plugins/monitoring/public/lib/internal_monitoring_toasts.tsx

chrisronline · 2020-07-16T17:56:26Z

x-pack/plugins/monitoring/public/lib/internal_monitoring_toasts.tsx

+
+        <EuiSpacer />
+        <EuiLink
+          href={`${ELASTIC_WEBSITE_URL}blog/external-collection-for-elastic-stack-monitoring-is-now-available-via-metricbeat`}


x-pack/plugins/monitoring/public/lib/internal_monitoring_toasts.tsx

.../plugins/monitoring/server/routes/api/v1/elasticsearch_settings/check/internal_monitoring.ts

…nal-monitoring-deprecation

apmmachine · 2020-07-21T13:07:07Z

💚 Build Succeeded

Expand to view the summary

Build stats

Build Cause: [Pull request #72020 updated]
Start Time: 2020-07-21T13:02:25.897+0000
Duration: 5 min 38 sec

.../plugins/monitoring/server/routes/api/v1/elasticsearch_settings/check/internal_monitoring.ts

chrisronline · 2020-07-21T13:54:42Z

.../plugins/monitoring/server/routes/api/v1/elasticsearch_settings/check/internal_monitoring.ts

+    _index.includes('-mb-')
+  ).length;
+
+  counts.legacyIndicesCount = hits.length - counts.mbIndicesCount;


These counts seem off to me.

I added this debug code:
console.log(hits.map((h) => h._index));

which results in:

[ '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21', '.monitoring-kibana-7-2020.07.21' ]

Maybe you can just do a query like:

POST .monitoring-es-*/_search { "size": 0, "aggs": { "types": { "terms": { "field": "_index", "size": 10 } } } }

…nal-monitoring-deprecation

igoristic · 2020-07-21T19:07:04Z

@elasticmachine merge upstream

igoristic · 2020-07-22T14:44:06Z

@elasticmachine merge upstream

chrisronline · 2020-07-22T18:57:49Z

@igoristic It looks like the failing tests are from monitoring. Maybe something with these changes broke them?

igoristic · 2020-07-22T22:10:11Z

@igoristic It looks like the failing tests are from monitoring. Maybe something with these changes broke them?

@chrisronline Thats what I thought, but I ran them locally and they all pass. Not sure what the problem is, since CI always shows a different failed test

igoristic · 2020-07-22T23:00:56Z

@elasticmachine merge upstream

chrisronline · 2020-07-23T13:19:38Z

x-pack/plugins/monitoring/public/lib/internal_monitoring_toasts.tsx

+        <p>
+          {i18n.translate('xpack.monitoring.internalAndMetricbeatMonitoringToast.description', {
+            defaultMessage: `It appears you are using both Metricbeat and "Legacy Collection" for Stack Monitoring. 
+            only Metricbeat type monitoring will be supported in the next major release (8.0.0). 


What if we said: Monitoring will require Metricbeat collection in the next major version (8.0.0).

or maybe @gchaps can help here

How about:

In 8.0.0, you must use Metricbeat to collect monitoring data.

…nal-monitoring-deprecation

kibanamachine · 2020-07-27T11:42:17Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 9c7d7b7

Build metrics

async chunks size

id	value	diff	baseline
monitoring	1.1MB	+6.2KB	1.1MB

History

💔 Build #63823 failed 2216464
💔 Build #63796 failed ddbdad0
💔 Build #63618 failed 60daec3
💔 Build #63392 failed 2c45f1c
💔 Build #63370 failed a3171f9

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

chrisronline

LGTM! Nice work!

igoristic · 2020-07-27T21:46:28Z

Backport:
7.x: c630055

* Internal Monitoring deprecation * Fixed type * Added if cloud logic * Fixed i18n test * Addressed code review feedback * Fixed types * Changed query * Added delay to fix a test * Fixed tests * Fixed text Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

bhavyarm · 2020-07-27T22:09:34Z

Also resolves this double alert issue -

sgrodzicki · 2020-07-30T16:04:51Z

Also resolves this double alert issue

@igoristic we would need to backport just the bugfix to make it work for 7.9

igoristic added 3 commits July 16, 2020 05:03

Internal Monitoring deprecation

978c636

Fixed type

df51458

Added if cloud logic

6750d52

igoristic added release_note:deprecation Team:Monitoring Stack Monitoring team v8.0.0 v7.10.0 labels Jul 16, 2020

igoristic requested a review from a team July 16, 2020 09:27

igoristic added 2 commits July 16, 2020 10:00

Merge branch 'master' of https://github.com/elastic/kibana into inter…

654e485

…nal-monitoring-deprecation

Fixed i18n test

6ef83f1

chrisronline suggested changes Jul 16, 2020

View reviewed changes

igoristic added 3 commits July 20, 2020 14:45

Merge branch 'master' of https://github.com/elastic/kibana into inter…

8797eab

…nal-monitoring-deprecation

Merge branch 'master' of https://github.com/elastic/kibana into inter…

4582b2a

…nal-monitoring-deprecation

Addressed code review feedback

cbb0fb4

igoristic requested a review from chrisronline July 20, 2020 22:37

igoristic added 2 commits July 21, 2020 09:00

Merge branch 'master' of https://github.com/elastic/kibana into inter…

d6b31c2

…nal-monitoring-deprecation

Fixed types

b9e7db8

chrisronline suggested changes Jul 21, 2020

View reviewed changes

igoristic added 3 commits July 21, 2020 12:01

Merge branch 'master' of https://github.com/elastic/kibana into inter…

28f4093

…nal-monitoring-deprecation

Merge branch 'master' of https://github.com/elastic/kibana into inter…

57cbbfc

…nal-monitoring-deprecation

Changed query

a3171f9

igoristic requested a review from chrisronline July 21, 2020 17:33

Merge branch 'master' into internal-monitoring-deprecation

2c45f1c

Merge branch 'master' into internal-monitoring-deprecation

60daec3

elasticmachine and others added 2 commits July 22, 2020 17:01

Merge branch 'master' into internal-monitoring-deprecation

ddbdad0

Added delay to fix a test

2216464

chrisronline suggested changes Jul 23, 2020

View reviewed changes

igoristic added 3 commits July 27, 2020 03:41

Merge branch 'master' of https://github.com/elastic/kibana into inter…

24f8817

…nal-monitoring-deprecation

Fixed tests

c44908f

Fixed text

9c7d7b7

igoristic requested review from chrisronline and gchaps July 27, 2020 10:02

chrisronline approved these changes Jul 27, 2020

View reviewed changes

igoristic merged commit 9aa5e17 into elastic:master Jul 27, 2020

igoristic deleted the internal-monitoring-deprecation branch July 27, 2020 15:16

igoristic mentioned this pull request Jul 27, 2020

[7.x] [Monitoring] "Internal Monitoring" deprecation warning (#72020) #73318

Merged

igoristic added the backported label Jul 27, 2020

LeeDr mentioned this pull request Jan 14, 2021

[Monitoring] deprecation dialog link doesn't enter setup mode #88404

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Monitoring] "Internal Monitoring" deprecation warning #72020

[Monitoring] "Internal Monitoring" deprecation warning #72020

igoristic commented Jul 16, 2020

elasticmachine commented Jul 16, 2020

chrisronline left a comment

chrisronline Jul 16, 2020

igoristic Jul 20, 2020 •

edited

Loading

chrisronline Jul 16, 2020

apmmachine commented Jul 21, 2020

Build stats

chrisronline Jul 21, 2020

igoristic commented Jul 21, 2020

igoristic commented Jul 22, 2020

chrisronline commented Jul 22, 2020

igoristic commented Jul 22, 2020

igoristic commented Jul 22, 2020

chrisronline Jul 23, 2020

gchaps Jul 23, 2020

kibanamachine commented Jul 27, 2020

chrisronline left a comment

igoristic commented Jul 27, 2020

bhavyarm commented Jul 27, 2020

sgrodzicki commented Jul 30, 2020

[Monitoring] "Internal Monitoring" deprecation warning #72020

[Monitoring] "Internal Monitoring" deprecation warning #72020

Conversation

igoristic commented Jul 16, 2020

elasticmachine commented Jul 16, 2020

chrisronline left a comment

Choose a reason for hiding this comment

chrisronline Jul 16, 2020

Choose a reason for hiding this comment

igoristic Jul 20, 2020 • edited Loading

Choose a reason for hiding this comment

chrisronline Jul 16, 2020

Choose a reason for hiding this comment

apmmachine commented Jul 21, 2020

💚 Build Succeeded

Build stats

chrisronline Jul 21, 2020

Choose a reason for hiding this comment

igoristic commented Jul 21, 2020

igoristic commented Jul 22, 2020

chrisronline commented Jul 22, 2020

igoristic commented Jul 22, 2020

igoristic commented Jul 22, 2020

chrisronline Jul 23, 2020

Choose a reason for hiding this comment

gchaps Jul 23, 2020

Choose a reason for hiding this comment

kibanamachine commented Jul 27, 2020

💚 Build Succeeded

Build metrics

async chunks size

History

chrisronline left a comment

Choose a reason for hiding this comment

igoristic commented Jul 27, 2020

bhavyarm commented Jul 27, 2020

sgrodzicki commented Jul 30, 2020

igoristic Jul 20, 2020 •

edited

Loading