[Observability] Add docs on schema selection for host data

[mdbirnstiehl]

This PR closes #2084

[github-actions]

🔍 Preview links for changed docs

• reference/data-analysis/observability/observability-host-metrics.md
• solutions/observability/incident-management/create-an-inventory-rule.md
• solutions/observability/infra-and-hosts/analyze-compare-hosts.md
• solutions/observability/infra-and-hosts/detect-metric-anomalies.md
• solutions/observability/infra-and-hosts/view-infrastructure-metrics-by-resource-type.md

[mdbirnstiehl]

@roshan-elastic I've started a draft here for the schema selector docs so we can iterate on what we need. Previews are in the comment above titled Preview links for changed docs.

Let me know what you think about the updates, and if we need to change anything or add additional information anywhere.

I still need to update the metrics reference page, is there a PR or documentation I can look at that has the metric definitions that need to be added?

[roshan-elastic]

@mdbirnstiehl - One thing we're doing is rendering the host view based on it schema.

For example, if it's an OTel host:

• We'll remove the anomaly detection tab as it doesn't work
• We add a little icon to show it is observed via OTel
• The metric formulae in the Lens charts will all change to use OTel formulae - we link to the metrics reference docs in various places in the UI to communicate the formulae
• For processes for OTel hosts, we've alsofixed the UI to explain that it is dependent on the process scraper. We should probably mention this in the host detail docs too.

In short, I'm thinking we should update the docs to reflect this:

Thoughts

• Would it make sense to explain that the host view adapts based on schema in the docs?
• Can we update the host metrics reference to call out the specific formulae for OTel Hosts? I can update* this doc to give the updated logic.

*We'll need to be able to refer to an 'OpenTelemetry' #heading within the Hosts section of the metrics reference so that we can point users here directly from the UI (I have a short-link of https://ela.st/docs-infra-host-metrics-otel that I can point towards the docs)

[roshan-elastic]

Hosts in Infra Inventory
One other thing @mdbirnstiehl, the Infra Inventory will show the schema selector for hosts too so we'll need to update the infra inventory docs to reflect this too:

• [Infra] Infra inventory UI schema selector kibana#230528

FYI it uses the same schema selector and behaves in the same way as hosts.

We'll be adding a link from this selector to the docs to explain how it works (if we wanted to simplify the docs we could possibly just refer to the schema selector in the host docs from the inventory docs for more info?)

FYI - there's a possibility we remove schema selection from the infra inventory in a follow up issue but we haven't decided yet. Any work we do in these infra inventory docs should probably be minimal.

[roshan-elastic]

FYI - @mdbirnstiehl - I have a build available that you can test - I'll DM you

[mdbirnstiehl]

@roshan-elastic I've added the following:

• Added notes to the Processes and Anomalies sections under view hosts.
• Added a list noting the main differences when selecting otel schema under view hosts. I just added the main ones, but we can add more if you think they add value.
• Added a note to the Inventory conditions section about schema dropdown. Is the paragraph before this enough generic information about an inventory rule, or do you think we should add more?
• Added schema note to the Infrastructure Inventory docs
• Created a section in the host metrics reference. Haven't added the actual formulas yet. Is the google doc up to date or does it still need to be?

[roshan-elastic]

Hey @mdbirnstiehl

Added notes to the Processes and Anomalies sections under view hosts.

Looks perfect for Hosts. I think we just need to update the inventory one too but it might be worth just removing that and linking through to the same content in the hosts (your call).

Added a list noting the main differences when selecting otel schema under view hosts. I just added the main ones, but we can add more if you think they add value.

Looks good!

Added a note to the Inventory conditions section about schema dropdown. Is the paragraph before this enough generic information about an inventory rule, or do you think we should add more?

Yeah, that looks good.

Added schema note to the Infrastructure Inventory docs

Looks good!

Created a section in the host metrics reference. Haven't added the actual formulas yet. Is the google doc up to date or does it still need to be?

Yeah I think something like should work. I think OpenTelemetry should probably be a higher-level heading as there will be all the same kinds of metrics underneath it (e.g. CPU, disk, memory etc).

I'll get the doc updated now so you can populate this and re-jig as necessary?

I've updated the doc so you can make the changes now - thanks!

[mdbirnstiehl]

Created a section in the host metrics reference. Haven't added the actual formulas yet. Is the google doc up to date or does it still need to be?

Yeah I think something like should work. I think OpenTelemetry should probably be a higher-level heading as there will be all the same kinds of metrics underneath it (e.g. CPU, disk, memory etc).

I'll get the doc updated now so you can populate this and re-jig as necessary?

I've updated the doc so you can make the changes now - thanks!

Got it, makes sense after seeing the updated doc. I'll work on that today.

[mdbirnstiehl]

@roshan-elastic I've update the host metrics page, and added the notes to the Infrastructure Inventory regard anomalies and processes. One last thing to verify, the OTel formulas are only relevant for stack from 9.2 forward, correct?

[roshan-elastic]

@roshan-elastic I've update the host metrics page, and added the notes to the Infrastructure Inventory regard anomalies and processes.

Awesome @mdbirnstiehl.

One last thing to verify, the OTel formulas are only relevant for stack from 9.2 forward, correct?

That will be the next major release they go into yes. They will likely go out in serverless before then though - I can let you know when it's merged into main to give you a better idea of exactly when it'll be available?

[roshan-elastic]

Hey @mdbirnstiehl,

FYI we'll be imminently merging the Hosts schema work into main:

• [ObsUX][HostsOtel] Remove hostsOtelEnabled feature flag kibana#232497

Feel free to merge in your work any time.

[nastasha-solomon]

Just a few minor comments. Otherwise, lgtm!

[nastasha-solomon]

Not sure if you need to add the {applies_to} tag here. Flagging in case you do.

Suggested change

	If you haven’t added data yet, click **Add data → Host** and select how you want to monitor your host—OpenTelemetry or Elastic Agent.
	`{applies_to}`{stack: "ga 9.2", serverless: "ga"}`
	If you haven’t added data yet, click **Add data → Host** and select how you want to monitor your host—OpenTelemetry or Elastic Agent.

[mdbirnstiehl]

The Add data steps and OTel option did exist in 9.0, but this integration language was out of date. OTel data has been available, it's just the schema selection that is new.