runtime_experimental: otel does not gather elastic_agent.filebeat_input datastream

[leehinman]

In the http-metrics-monitoriing component there should be an input for the elastic-agent.filebeat_input datastream. An example from the beat-rendered-config.yml in a diagnostic would be below. but when agent.monitoring._runtime_experimental: otel is used. The filebeat_input does not appear in the otel-final.yml from a diagnostic and no data appears in the data_stream.

Relates #8031

    - data_stream:
        dataset: elastic_agent.filebeat_input
        namespace: default
        type: metrics
      failure_threshold: 5
      hosts:
        - http+unix:///opt/Elastic/Agent/data/tmp/xTEtpJ7117ppc6OYvJCaYHbDW8mLjXGe.sock
      id: metrics-monitoring-filebeat-1
      index: metrics-elastic_agent.filebeat_input-default
      json:
        is_array: true
      metricsets:
        - json
      module: http
      namespace: filebeat_input
      path: /inputs/
      period: 1m0s
      processors:
        - add_fields:
            fields:
                input_id: metrics-monitoring-agent
            target: '@metadata'
        - add_fields:
            fields:
                dataset: elastic_agent.filebeat_input
                namespace: default
                type: metrics
            target: data_stream
        - add_fields:
            fields:
                dataset: elastic_agent.filebeat_input
            target: event
        - add_fields:
            fields:
                stream_id: metrics-monitoring-filebeat-1
            target: '@metadata'
        - add_fields:
            fields:
                id: 6d6c9275-f7d3-4381-bde0-f1ea8cf1bc53
                snapshot: false
                version: 9.1.0
            target: elastic_agent
        - add_fields:
            fields:
                id: 6d6c9275-f7d3-4381-bde0-f1ea8cf1bc53
            target: agent
        - add_fields:
            fields:
                dataset: elastic_agent.filebeat_input
            target: event
        - add_fields:
            fields:
                id: 6d6c9275-f7d3-4381-bde0-f1ea8cf1bc53
                process: filebeat
                snapshot: false
                version: 9.1.0
            target: elastic_agent
        - add_fields:
            fields:
                id: 6d6c9275-f7d3-4381-bde0-f1ea8cf1bc53
            target: agent
        - copy_fields:
            fail_on_error: false
            fields:
                - from: http.agent.beat.cpu
                  to: system.process.cpu
                - from: http.agent.beat.memstats.memory_sys
                  to: system.process.memory.size
                - from: http.agent.beat.handles
                  to: system.process.fd
                - from: http.agent.beat.cgroup
                  to: system.process.cgroup
                - from: http.agent.apm-server
                  to: apm-server
                - from: http.filebeat_input
                  to: filebeat_input
            ignore_missing: true
        - drop_fields:
            fields:
                - http
            ignore_missing: true
        - add_fields:
            fields:
                binary: filebeat
                id: filestream-monitoring
            target: component

[elasticmachine]

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

[swiatekm]

This is intentionally disabled so we don't have a degraded input with beats receivers self-monitoring:

elastic-agent/internal/pkg/agent/application/monitoring/v1_monitor.go

Line 694 in 32ec749

// disabled for filebeat receiver until https://github.com/elastic/beats/issues/43418 is resolved

. If this is now fixed in beats, then we can also re-enable it in agent.

[leehinman]

waiting on elastic/beats#44231 to fix