[elastic_agent] Fix otelcol.* mapping conflicts in logs data streams

[samuelvl]

The OTel Collector service package annotates internal component logs with fields like otelcol.signal and otelcol.signal.output in the same document:

{
  "message": "Retrying failed request",
  "otelcol.signal": "logs",
  "otelcol.signal.output": "failed",
  "otelcol.component.id": "elasticsearch/default",
  "otelcol.component.kind": "exporter"
}

Without an explicit mapping, Elasticsearch treats the dot as a path separator and interprets otelcol.signal as both a leaf value and a parent object — causing a mapping conflict and silently dropping these fields from the index.

Adding otelcol.* as an object with subobjects: false tells Elasticsearch to treat dots as literal characters, so each field is indexed independently as a keyword.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.
• [ ] I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

cd packages/elastic_agent

ELASTIC_PACKAGE_ELASTICSEARCH_HOST=http://127.0.0.1:9200 \
ELASTIC_PACKAGE_ELASTICSEARCH_USERNAME=elastic \
ELASTIC_PACKAGE_ELASTICSEARCH_PASSWORD=changeme \
elastic-package test pipeline --data-streams elastic_agent_logs -v

All pipeline tests should pass, including test-otelcol-dotted-fields.json which exercises documents containing both otelcol.signal and otelcol.signal.output.

Related issues

• Closes [elastic_agent]: OTel monitoring should allow dotted field names #18129

[infra-vault-gh-plugin-prod]

Pinging @elastic/elastic-agent (Team:Elastic-Agent)

[github-actions]

TL;DR

The Buildkite failure is a merge conflict during the repository post-checkout hook, not a test/runtime regression. Rebase pr-19070 onto current main and resolve packages/elastic_agent/{manifest.yml,changelog.yml} conflicts.

Remediation

• Rebase branch samuelvl:elastic-agent-template-subobjects on latest main and resolve conflicts in:
  • packages/elastic_agent/manifest.yml
  • packages/elastic_agent/changelog.yml
• Keep a single coherent version/changelog sequence after rebase (current main already introduced 2.9.0 in elastic_agent; this PR currently introduces 2.8.1 from an older base).
• Push the rebased branch and re-run CI.

Investigation details

Root Cause

Buildkite fails in :pipeline::arrow_up: Upload Pipeline: .buildkite/pipeline.yml because the repo hook attempts to merge PR head with main and hits conflicts before pipeline generation.

Conflicting files are both modified in PR and in main after the branch point:

• PR commit 13b3d83aef... updates:
  • packages/elastic_agent/manifest.yml (version: 2.8.0 -> 2.8.1)
  • packages/elastic_agent/changelog.yml (adds 2.8.1 entry)
• main already includes newer elastic_agent package bump/changelog (2.9.0).

Evidence

• Build: https://buildkite.com/elastic/integrations/builds/43093
• Job/step: :pipeline::arrow_up: Upload Pipeline: .buildkite/pipeline.yml
• Log excerpt (/tmp/gh-aw/buildkite-logs/integrations-pipelinearrow_up-upload-pipeline-buildkitepipelineyml.txt):
  • CONFLICT (content): Merge conflict in packages/elastic_agent/changelog.yml
  • CONFLICT (content): Merge conflict in packages/elastic_agent/manifest.yml
  • Automatic merge failed; fix conflicts and then commit the result.
  • Error: running "repository post-checkout" shell hook ... exited with status 1

I also reproduced the same failure locally by attempting git merge origin/main from the fetched PR head (merge_exit=1 with the same two conflict files).

Verification

• Not run beyond conflict reproduction, because failure occurs before test/build stages.

Follow-up

After rebase conflict resolution, CI should proceed past pipeline upload and then expose any downstream issues (if any).

Note

🔒 Integrity filter blocked 3 items

The following items were blocked because they don't meet the GitHub integrity level.

• #19070 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [elastic_agent] Fix otelcol.* mapping conflicts in logs data streams #19070 pull_request_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".
• [elastic_agent] Fix otelcol.* mapping conflicts in logs data streams #19070 issue_read: has lower integrity than agent requires. The agent cannot read data with integrity below "approved".

To allow these resources, lower min-integrity in your GitHub frontmatter:

tools:
  github:
    min-integrity: approved  # merged | approved | unapproved | none

---

What is this? | From workflow: PR Buildkite Detective

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.

[elastic-vault-github-plugin-prod]

🚀 Benchmarks report

Package elastic_agent 👍(0) 💚(1) 💔(1)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
status_change_logs	500000	250000	-250000 (-50%)	💔

To see the full report comment with /test benchmark fullreport

[macdewee]

LGTM

[elasticmachine]

💚 Build Succeeded

• Buildkite Build
• Commit: 77b0e2b

History

• 💚 Build #43151 succeeded 2e78ea9
• 💔 Build #43093 failed 13b3d83

[elastic-vault-github-plugin-prod]

Package elastic_agent - 2.9.2 containing this change is available at https://epr.elastic.co/package/elastic_agent/2.9.2/