[Flaky Test]: TestContainerCMDAgentMonitoringRuntimeExperimental and TestContainerCMDAgentMonitoringRuntimeExperimentalPolicy– expected correct runtime name for monitoring component

[pchila]

Failing test case

TestContainerCMDAgentMonitoringRuntimeExperimental/var_set_to_process

Error message

test expects "process" as runtime for component "http/metrics-monitoring", got empty instead

Build

https://buildkite.com/elastic/elastic-agent/builds/30267#019a7ac1-de2a-4ba3-931f-c4aba51f0436/4371-4731

OS

Linux

Stacktrace and notes

container_cmd_test.go:551: Component ID: beat/metrics-monitoring, version info: beat-v2-client, runtime: process
    container_cmd_test.go:551: Component ID: filestream-monitoring, version info: beat-v2-client, runtime: process
    container_cmd_test.go:551: Component ID: http/metrics-monitoring, version info: , runtime:
    container_cmd_test.go:555:
        	Error Trace:	/opt/buildkite-agent/builds/bk-agent-prod-gcp-1762996122194778860/elastic/elastic-agent/testing/integration/ess/container_cmd_test.go:555
        	            				/opt/buildkite-agent/.asdf/installs/golang/1.24.10/go/src/runtime/asm_amd64.s:1700
        	Error:      	Not equal:
        	            	expected: "process"
        	            	actual  : ""
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -1 +1 @@
        	            	-process
        	            	+
        	Test:       	TestContainerCMDAgentMonitoringRuntimeExperimental/var_set_to_process
        	Messages:   	expected correct runtime name for monitoring component http/metrics with id http/metrics-monitoring
    container_cmd_test.go:551: Component ID: system/metrics-default, version info: beat-v2-client, runtime: process
    container_cmd_test.go:101: >> cleaning up: killing the Elastic-Agent process
    fixture.go:1451: Temporary directory "/tmp/TestContainerCMDAgentMonitoringRuntimeExperimental-var_set_to_process3292396216" preserved for investigation/debugging
    --- FAIL: TestContainerCMDAgentMonitoringRuntimeExperimental/var_set_to_process (9.97s)

[elasticmachine]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[pchila]

@mauri870 @swiatekm this test has been added recently with PR #11018
Could you please take a look and check why it would be flaky ?

[mauri870]

@pchila I'll look into it.

[ebeahan]

Another instance: https://buildkite.com/elastic/elastic-agent-extended-testing/builds/11135#019a7e48-e0b3-4efc-90f4-7e92178d4e0a

[mauri870]

Another failure: https://buildkite.com/elastic/elastic-agent/builds/30336#019a81ee-28e1-46df-a231-7d56990a50f4/4386-4758

[mauri870]

According to the test output, the http/metrics component is reporting an empty version info name from the status --json output. Since this field is normally populated for all other components, and for this one most of the time as well and the command returned success, this suggests an issue with either the test fixture or the command itself.

[swiatekm]

@mauri870 this is set here and I'm not sure how it can be empty when using the otel runtime. I can see the beats manager always set it as well.

[mauri870]

@mauri870 this is set here and I'm not sure how it can be empty when using the otel runtime. I can see the beats manager always set it as well.

Thanks, yeah, it does not make much sense for it to be empty. It is strange that it is the only component that sometimes has an empty value:

    container_cmd_test.go:551: Component ID: beat/metrics-monitoring, version info: beat-v2-client, runtime: process
    container_cmd_test.go:551: Component ID: filestream-monitoring, version info: beat-v2-client, runtime: process
    container_cmd_test.go:551: Component ID: http/metrics-monitoring, version info: , runtime:
    container_cmd_test.go:551: Component ID: log-default, version info: beat-v2-client, runtime: process