Data is visible with host name under metrics-* index temporarily after upgrade, if fqdn type is set before upgrade.

[amolnater-qasource]

Kibana Build details:

VERSION: 8.8 BC3 kibana cloud environment
BUILD: 62994
COMMIT: 85b22d307ab93fca95c1698ede4cb61d85f3d314

Host OS and Browser version: Windows, MAC and Linux, All

Preconditions:

1. 8.8 BC3 kibana cloud environment should be available.
2. 8.7.1 Agents should be installed.

Steps to reproduce:

1. Set FQDN host name type for all the 8.7.1 agents.
2. Wait for sometime, until only FQDN hostname data is visible.
3. Upgrade the agents and observe post upgrade, data is visible with host name temporarily under Discover tab.

Expected:
Data should be visible with fqdn type only under metrics-* index after upgrade too, if fqdn type is set before upgrade.

Screen Recording:

Agents.-.Fleet.-.Elastic.-.Google.Chrome.2023-05-15.15-30-06.mp4

[amolnater-qasource]

@manishgupta-qasource Please review.

[manishgupta-qasource]

Secondary review for this ticket is Done

[amolnater-qasource]

FYI @ycombinator

[ycombinator]

Upgrade the agents and observe post upgrade, data is visible with host name temporarily under Discover tab:

Just to clarify, you mean that post-upgrade, you can see some documents in metrics-* still containing the short hostname (as opposed to the longer FQDN)?

[amolnater-qasource]

Hi @ycombinator

Thank you for looking into this, yes post-upgrade we observed some entries with metrics-* index.
Please let us know if anything else is required from our end.

Thanks!

[cmacknz]

@amolnater-qasource does running elastic-agent restart produce the same result? Wondering if the agent restart done as part of the upgrade causes this. There will be a period of time where the agent doesn't have the feature flag configuration from the agent yet, and it could perform writes to Elasticsearch in that period.

[amolnater-qasource]

Hi @cmacknz

Thank you for looking into this issue.
We have revalidated this issue on latest 8.8 BC5 Kibana cloud environment and found it reproducible with agent restart too.

Observations:

• Both logs-* and metrics-* are available for the upgraded agents with host name if fqdn type is set.
• Even on restart logs-* and metrics-* are available for the agents with host name.

Screen Recording:
Post upgrade:

Discover.-.Elastic.-.Google.Chrome.2023-05-18.16-46-55.mp4

On Restart:

Discover.-.Elastic.-.Google.Chrome.2023-05-18.16-47-30.mp4

Discover.-.Elastic.-.Google.Chrome.2023-05-18.16-47-59.mp4

Build details:

VERSION: 8.8.0 BC5 Kibana cloud environment
BUILD: 63092
COMMIT: ba9b182a3422f5e162c1ca74baafb91833eca059

Please let us know if anything else is required from our end.

Thanks!

[jlind23]

Closed by elastic/beats#35736

[amolnater-qasource]

Hi Team,

We have revalidated this issue on latest 8.9.0 BC2 Kibana cloud environment and found it fixed now.

Observations:

• Data is visible with fqdn type only under metrics-* index after upgrade too, if fqdn type is set before upgrade.

Screen Recording:

Discover.-.Elastic.-.Google.Chrome.2023-07-06.13-24-48.mp4

Build details:
BUILD: 64459
COMMIT: 6950a2b8207d8388ee8c842d6c0e2b1e1031fd36
Artifact Link: https://staging.elastic.co/8.9.0-826fd88d/downloads/beats/elastic-agent/elastic-agent-8.9.0-windows-x86_64.zip

Hence, we are marking this issue as QA:Validated.

Thanks