Add `OTEL_COMPONENT=true` to packaging to allow building a seperate OTEL EDOT binary #11237

blakerouse · 2025-11-18T21:54:19Z

What does this PR do?

Adds a new OTEL_COMPONENT flag that allows building the Elastic Agent with a seperate OTEL EDOT collector binary, instead of it being compiled into the Elastic Agent binary. The OTEL_COMPONENT flag is only temporary and will become the default with no way to turn it off in the future.

The binary is currently named otelcol-component to allow it to live next to the otelcol scripts. Because the OTEL_COMPONENT flag is optional and the packages.yml doesn't really allow that optionality this provides a way for both to exist at the same time.

Once this becomes the only way to build then it will just be otelcol and the scripts will either just be replaced with that actual binary or a symlink if the script is not in the correct place.

NOTE: This does not yet include the agentbeat into the single binary, that will happen in the next PR. This just confirms that building in this mode does work and runs correctly.

Why is it important?

Part of the journey to separate the control plane and data plane components into separate binaries.

Checklist

I have read and understood the pull request guidelines of this project.
My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
~~[ ] I have made corresponding changes to the documentation~~
~~[ ] I have made corresponding change to the default configuration files~~
~~[ ] I have added tests that prove my fix is effective or that my feature works~~
~~[ ] I have added an entry in ./changelog/fragments using the changelog tool~~
~~[ ] I have added an integration test or an E2E test~~

Disruptive User Impact

None

How to test this PR locally

Run packaging as you normally would on your system to build an Elastic Agent but add the OTEL_COMPONENT="true" option before the mage package. Example below for building on my system is as follows:

PLATFORMS="darwin/arm64" SNAPSHOT="true" EXTERNAL="true" OTEL_COMPONENT="true" PACKAGES="targz" mage package

Once built running elastic-agent otel will spawn the seperate binary and running with elastic-agent run and a configuration that will use the OTEL runtime will also work.

…uild.

elasticmachine · 2025-11-18T21:54:22Z

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

blakerouse · 2025-11-19T00:33:01Z

Failing only because it cannot install junit2html. Fixes for that is here - #11238

Once merged I will rebase and then should go green.

internal/edot/main.go

internal/pkg/agent/cmd/otel_external_unix.go

magefile.go

blakerouse · 2025-11-19T17:07:38Z

@pchila Thanks for the review. I have updated all your the suggestions.

magefile.go

cmacknz · 2025-11-20T14:08:38Z

The binary is currently named otelcol-component to allow it to live next to the otelcol scripts.

The binary should probably be called elastic-otel-collector to follow the convention of our Docker image. This is similar to what other vendors name their collectors e.g. https://github.com/aws-observability/aws-otel-collector or https://github.com/observIQ/bindplane-otel-collector

blakerouse · 2025-11-20T15:16:47Z

@cmacknz I updated the binary name to elastic-otel-collector.

swiatekm · 2025-11-20T16:49:48Z

When I try to build this locally, I get an error about VCS stamping:

elastic-agent git:(edot-binary-buildtag) ✗ OTEL_COMPONENT=true EXTERNAL=true SNAPSHOT=true PACKAGES="tar.gz" PLATFORMS="linux/amd64" mage package
(...)
--- CrossBuild elastic-otel-collector
--- CrossBuild Elastic-Agent
>> otel:golangCrossBuild: Building for linux/amd64
>> golangCrossBuild: Building for linux/amd64
>> Building using: cmd='build/mage-linux-amd64 golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, GOTOOLCHAIN=local, PLATFORM_ID=linux-amd64]
>> Building using: cmd='build/mage-linux-amd64 otel:golangCrossBuild', env=[CC=gcc, CXX=g++, GOARCH=amd64, GOARM=, GOOS=linux, GOTOOLCHAIN=local, PLATFORM_ID=linux-amd64]
error obtaining VCS status: exit status 128
        Use -buildvcs=false to disable VCS stamping.
Error: finding repository root: running "go list -f {{.Root}}" failed with exit code 1
Error: failed building for linux/amd64: exit status 1
failed building for linux/amd64: exit status 1
package ran for 30.323247319s
Error: multiple failures: running "docker run --ulimit nofile=262144:262144 --env EXEC_UID=1000 --env EXEC_GID=1001 -v /home/mikolaj/go/pkg/mod:/go/pkg/mod:ro -v elastic-agent-crossbuild-build-cache:/tmp/.cache/go-build --rm --env GOFLAGS=-mod=readonly --env GOCACHE=/tmp/.cache/go-build --env MAGEFILE_VERBOSE= --env MAGEFILE_TIMEOUT= --env SNAPSHOT=true --env DEV=false --env EXTERNAL=true --env FIPS=false --env OTEL_COMPONENT=true -v /home/mikolaj/Projects/elastic-agent:/go/src/github.com/elastic/elastic-agent -w /go/src/github.com/elastic/elastic-agent docker.elastic.co/beats-dev/golang-crossbuild:1.24.10-main-debian11 --build-cmd build/mage-linux-amd64 otel:golangCrossBuild --platforms linux/amd64" failed with exit code 1

cmacknz · 2025-11-20T20:36:56Z

The two related issues for vcs stamping issue are but that error doesn't immediately look like either of these two:

cmacknz · 2025-11-20T20:38:16Z

We likely need a temporary CI step to build this way to make sure it doesn't accidentally break in a future PR.

This reverts commit 329de79.

blakerouse · 2025-11-25T02:43:36Z

I fixed the VCS issue and I added the packaging steps in the CI to ensure that the build keeps working as this feature is iterated on before it comes the default.

Its ready for another review. Thanks.

@cmacknz @swiatekm @pchila

v1v

I've reviewed the changes in the BK pipeline - LGTM, although I wonder if using a buildkite matrix could help here. FIPS was enabled in the past and duplicated the code, so I should avoid duplicating steps and use a dynamic approach instead?

.buildkite/integration.pipeline.yml

elasticmachine · 2025-11-25T14:31:12Z

💚 Build Succeeded

Buildkite Build
Commit: 100b101

History

💔 Build #30966 failed 6d58eb2
💔 Build #30942 failed 329de79
💔 Build #30940 failed 8784aaf
💛 Build #30710 was flaky 1691a5f
💔 Build #30566 failed 809f742

cc @blakerouse

github-actions · 2025-11-25T14:51:01Z

@Mergifyio backport 8.19 9.1 9.2

mergify · 2025-11-25T14:51:08Z

backport 8.19 9.1 9.2

✅ Backports have been created

#11415 [8.19] (backport #11237) Add OTEL_COMPONENT=true to packaging to allow building a seperate OTEL EDOT binary has been created for branch 8.19 but encountered conflicts
#11416 [9.1] (backport #11237) Add OTEL_COMPONENT=true to packaging to allow building a seperate OTEL EDOT binary has been created for branch 9.1 but encountered conflicts
#11417 [9.2] (backport #11237) Add OTEL_COMPONENT=true to packaging to allow building a seperate OTEL EDOT binary has been created for branch 9.2

…TEL EDOT binary (#11237) * Add ability to build seperate edot binary. Add build flag to change build. * Get this working. * Fix crossBuild log. * Fix imports. * fixes from code review. * Revert the change for packageAgent. * Update binary name. * Fix docker container. Add CI steps for build. * Change to serial deps. * Try not using go list. * Revert "Change to serial deps." This reverts commit 329de79. * Fix BK to build windows OTEL_COMPONENT. (cherry picked from commit 9f1c464) # Conflicts: # internal/pkg/agent/cmd/common.go # magefile.go

…TEL EDOT binary (#11237) * Add ability to build seperate edot binary. Add build flag to change build. * Get this working. * Fix crossBuild log. * Fix imports. * fixes from code review. * Revert the change for packageAgent. * Update binary name. * Fix docker container. Add CI steps for build. * Change to serial deps. * Try not using go list. * Revert "Change to serial deps." This reverts commit 329de79. * Fix BK to build windows OTEL_COMPONENT. (cherry picked from commit 9f1c464)

…TEL EDOT binary (#11237) * Add ability to build seperate edot binary. Add build flag to change build. * Get this working. * Fix crossBuild log. * Fix imports. * fixes from code review. * Revert the change for packageAgent. * Update binary name. * Fix docker container. Add CI steps for build. * Change to serial deps. * Try not using go list. * Revert "Change to serial deps." This reverts commit 329de79. * Fix BK to build windows OTEL_COMPONENT. (cherry picked from commit 9f1c464) # Conflicts: # internal/edot/cmd/otel.go # internal/pkg/agent/cmd/validate.go # internal/pkg/agent/main.go # magefile.go

…low building a seperate OTEL EDOT binary (#11415) * Add `OTEL_COMPONENT=true` to packaging to allow building a seperate OTEL EDOT binary (#11237) * Add ability to build seperate edot binary. Add build flag to change build. * Get this working. * Fix crossBuild log. * Fix imports. * fixes from code review. * Revert the change for packageAgent. * Update binary name. * Fix docker container. Add CI steps for build. * Change to serial deps. * Try not using go list. * Revert "Change to serial deps." This reverts commit 329de79. * Fix BK to build windows OTEL_COMPONENT. (cherry picked from commit 9f1c464) # Conflicts: # internal/pkg/agent/cmd/common.go # magefile.go * Fix conflicts. * No windows/arm64 in 8.18. --------- Co-authored-by: Blake Rouse <blake.rouse@elastic.co>

…ow building a seperate OTEL EDOT binary (#11417) * Add `OTEL_COMPONENT=true` to packaging to allow building a seperate OTEL EDOT binary (#11237) * Add ability to build seperate edot binary. Add build flag to change build. * Get this working. * Fix crossBuild log. * Fix imports. * fixes from code review. * Revert the change for packageAgent. * Update binary name. * Fix docker container. Add CI steps for build. * Change to serial deps. * Try not using go list. * Revert "Change to serial deps." This reverts commit 329de79. * Fix BK to build windows OTEL_COMPONENT. (cherry picked from commit 9f1c464) * Change to SerialDeps. --------- Co-authored-by: Blake Rouse <blake.rouse@elastic.co>

blakerouse added 3 commits November 13, 2025 16:46

Add ability to build seperate edot binary. Add build flag to change b…

208dbe3

…uild.

Get this working.

7e99972

Merge remote-tracking branch 'upstream/main' into edot-binary-buildtag

c0548e8

blakerouse self-assigned this Nov 18, 2025

blakerouse requested a review from a team as a code owner November 18, 2025 21:54

blakerouse requested a review from ycombinator November 18, 2025 21:54

blakerouse added the Team:Elastic-Agent-Control-Plane Label for the Agent Control Plane team label Nov 18, 2025

blakerouse requested a review from pchila November 18, 2025 21:54

blakerouse added skip-changelog opentelemetry Related to the Elastic Distribution of the OpenTelemetry Collector backport-active-all Automated backport with mergify to all the active branches labels Nov 18, 2025

blakerouse added 2 commits November 18, 2025 17:13

Fix crossBuild log.

c5d0123

Fix imports.

ee0696e

pierrehilbert requested a review from swiatekm November 19, 2025 07:51

blakerouse added 2 commits November 19, 2025 08:21

Merge branch 'main' into edot-binary-buildtag

700d1d2

Merge branch 'main' into edot-binary-buildtag

f0ed082

pchila reviewed Nov 19, 2025

View reviewed changes

internal/edot/main.go Outdated Show resolved Hide resolved

internal/pkg/agent/cmd/otel_external_unix.go Outdated Show resolved Hide resolved

magefile.go Show resolved Hide resolved

fixes from code review.

809f742

Merge branch 'main' into edot-binary-buildtag

23c5f3f

pchila reviewed Nov 20, 2025

View reviewed changes

magefile.go Outdated Show resolved Hide resolved

blakerouse added 2 commits November 20, 2025 10:15

Revert the change for packageAgent.

a09b252

Update binary name.

4c49fca

Merge branch 'main' into edot-binary-buildtag

1691a5f

blakerouse added 2 commits November 24, 2025 14:47

Merge remote-tracking branch 'upstream/main' into edot-binary-buildtag

b185a25

Fix docker container. Add CI steps for build.

8784aaf

blakerouse requested a review from a team as a code owner November 24, 2025 20:23

blakerouse added 4 commits November 24, 2025 16:16

Change to serial deps.

329de79

Try not using go list.

47e4e77

Revert "Change to serial deps."

31950ed

This reverts commit 329de79.

Merge branch 'main' into edot-binary-buildtag

6d58eb2

v1v previously approved these changes Nov 25, 2025

View reviewed changes

.buildkite/integration.pipeline.yml Outdated Show resolved Hide resolved

.buildkite/integration.pipeline.yml Outdated Show resolved Hide resolved

.buildkite/integration.pipeline.yml Show resolved Hide resolved

.buildkite/integration.pipeline.yml Show resolved Hide resolved

blakerouse added 2 commits November 25, 2025 08:04

Fix BK to build windows OTEL_COMPONENT.

b88ac70

Merge remote-tracking branch 'upstream/main' into edot-binary-buildtag

100b101

blakerouse dismissed v1v’s stale review via 100b101 November 25, 2025 13:05

swiatekm approved these changes Nov 25, 2025

View reviewed changes

blakerouse merged commit 9f1c464 into elastic:main Nov 25, 2025
21 checks passed

mergify bot mentioned this pull request Nov 25, 2025

[8.19] (backport #11237) Add OTEL_COMPONENT=true to packaging to allow building a seperate OTEL EDOT binary #11415

Merged

3 tasks

This was referenced Nov 25, 2025

[9.1] (backport #11237) Add OTEL_COMPONENT=true to packaging to allow building a seperate OTEL EDOT binary #11416

Closed

[9.2] (backport #11237) Add OTEL_COMPONENT=true to packaging to allow building a seperate OTEL EDOT binary #11417

Merged

cmacknz mentioned this pull request Dec 10, 2025

set path.home for beat receivers to be components dir #11726

Merged

8 tasks

Add OTEL_COMPONENT=true to packaging to allow building a seperate OTEL EDOT binary #11237

Add OTEL_COMPONENT=true to packaging to allow building a seperate OTEL EDOT binary #11237

Uh oh!

Conversation

blakerouse commented Nov 18, 2025

What does this PR do?

Why is it important?

Checklist

Disruptive User Impact

How to test this PR locally

Uh oh!

elasticmachine commented Nov 18, 2025

Uh oh!

blakerouse commented Nov 19, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

blakerouse commented Nov 19, 2025

Uh oh!

Uh oh!

cmacknz commented Nov 20, 2025

Uh oh!

blakerouse commented Nov 20, 2025

Uh oh!

swiatekm commented Nov 20, 2025

Uh oh!

cmacknz commented Nov 20, 2025

Uh oh!

cmacknz commented Nov 20, 2025

Uh oh!

blakerouse commented Nov 25, 2025

Uh oh!

v1v left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

elasticmachine commented Nov 25, 2025

💚 Build Succeeded

History

Uh oh!

Uh oh!

github-actions bot commented Nov 25, 2025

Uh oh!

mergify bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Add `OTEL_COMPONENT=true` to packaging to allow building a seperate OTEL EDOT binary #11237

Add `OTEL_COMPONENT=true` to packaging to allow building a seperate OTEL EDOT binary #11237

mergify bot commented Nov 25, 2025 •

edited

Loading