Docker scratch container failed to start with collecor built by ocb tool

[lesovsky]

Describe the bug
Container with collector (built with ocb) and based on scratch is failed to start with error: exec /otelcol: no such file or directory.

Steps to reproduce
Prepare builder.yaml config for ocb:

dist:
  name: otelcol
  version: "0.0.1-alpha.1"
  output_path: ./build

exporters:
  - gomod: go.opentelemetry.io/collector v0.62.1
    import: go.opentelemetry.io/collector/exporter/loggingexporter

Build the collector

ocb --config builder.yaml

Also put otel-collector from this repo into ./build directory.

Check both binaries, it works:

$ ./build/otelcol --version
otelcol version 0.0.1-alpha.1
$ ./build/otelcol_origin --version
2022/10/22 15:41:22 proto: duplicate proto type registered: jaeger.api_v2.PostSpansRequest
2022/10/22 15:41:22 proto: duplicate proto type registered: jaeger.api_v2.PostSpansResponse
otelcontribcol version v0.61.0-82-gda2df2664

Prepare Dockerfile (puth both binaries into image)

FROM scratch
COPY ./build/otelcol /
COPY ./build/otelcol_origin /
CMD ["./otelcol"]

Build it with docker build otel-collector:dev .

Run containers and try to exec both binaries - official collector works as intended.

docker run --rm otel-collector:dev /otelcol_origin --version
2022/10/22 09:42:03 proto: duplicate proto type registered: jaeger.api_v2.PostSpansRequest
2022/10/22 09:42:03 proto: duplicate proto type registered: jaeger.api_v2.PostSpansResponse
otelcontribcol version v0.61.0-82-gda2df2664

Collector build manually is not executed:

$ docker run --rm otel-collector:dev /otelcol --version
exec /otelcol: no such file or directory

At the same time, both binaries are present in the image

$ docker create --name test otel-collector:dev
b307a2bbd62fae5ed2d55838899cbe86b1d38dc1ae3adbd21bf474c0f6b0666d
$ docker export test |tar t |grep otel
otelcol
otelcol_origin

Unpacking image show that both binaries have proper permissions (+x) and could be executed succesfully.

I checked configs many times, tried to reproduce that from scratch, tried to use ENTRYPOINT instead of CMD, but had no success.

But if i use FROM golang instead of scratch - it's working fine. I don't understand what the root cause of the issue and why it isn't working with scratch only.

What did you expect to see?
I expect that binary created by ocb could be executed in docker container created from scratch

What did you see instead?
Container failed to start and return error about executable not found

What version did you use?
ocb: 0.62.1 (installed by go install ...)

Environment
OS: Ubuntu 20.04
Golang: 1.18.7
Docker version 20.10.17, build 100c701

[lesovsky]

The same problem with golang alpine images.

[adam-kiss-sg]

I ran into the same issue, so I checked how the official otel collector images are built: https://github.com/open-telemetry/opentelemetry-collector-releases/blob/main/.goreleaser.yaml

The trick seems to be setting CGO_ENABLED=0

Here is a Dockerfile that works for me (you might need to enable buildkit, see: https://docs.docker.com/build/buildkit/#getting-started).
It uses alpine, not sure about scratch, but you can look at the above .goreleaser.yaml config to see what else you might need.

# syntax=docker/dockerfile:1.3-labs

FROM golang:1.18 AS builder

RUN <<EOF
go install go.opentelemetry.io/collector/cmd/builder@latest
export CGO_ENABLED=0
cat > otelcol-builder.yaml <<CEOF
dist:
  name: otelcol-custom
exporters:
  - gomod: github.com/open-telemetry/opentelemetry-collector-contrib/exporter/prometheusremotewriteexporter v0.64.0
CEOF
builder --config=otelcol-builder.yaml --output-path=/tmp/dist
EOF

FROM alpine

RUN apk add --no-cache ca-certificates

COPY --from=builder /tmp/dist/otelcol-custom /usr/bin/otelcol-custom

COPY /config/otel-collector-config.yaml /etc/otel-collector-config.yaml
COPY /config/ucm-otel-collector-config.yaml /etc/ucm-otel-collector-config.yaml

CMD [ "otelcol-custom", "--config", "/etc/otel-collector-config.yaml" ]

[jpkrohling]

@lesovsky, can you confirm that setting export CGO_ENABLED=0 results in a binary that can be used from a container?

[lesovsky]

yes, it works with CGO_ENBALED=0, thanks!