Skip to content

UPSTREAM: <carry>: Add support for optional prebuilt Go binaries in Dockerfiles #183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

hbelmiro
Copy link

@hbelmiro hbelmiro commented Jun 17, 2025

Description of your changes:
This commit adds support for optional prebuilt Go binaries in Dockerfiles using a PREBUILT_BINARY argument. It allows developers to skip source builds and use prebuilt binaries, serving as a workaround for issues like cross-platform builds. The default behavior remains unchanged for compatibility.

For example, one can build the binary with:

CC=x86_64-unknown-linux-gnu-gcc GO111MODULE=on CGO_ENABLED=1 GOOS=linux GOARCH=amd64 go build -o controller backend/src/crd/controller/scheduledworkflow/*.go

Then build the Docker image using the prebuilt binary:

docker build --platform linux/amd64 --build-arg PREBUILT_BINARY=controller -t "dsp-scheduled-workflow:latest" -f backend/Dockerfile.scheduledworkflow .

Checklist:

Summary by CodeRabbit

  • Chores
    • Enhanced Docker build process for backend components to support using either prebuilt binaries or building from source, providing greater flexibility during image creation. This affects the apiserver, driver, launcher, persistence agent, and scheduled workflow controller images. No changes to runtime features or user-facing behavior.

Copy link

coderabbitai bot commented Jun 17, 2025

Walkthrough

Conditional logic was added to several backend Dockerfiles to allow the use of a prebuilt binary via a new PREBUILT_BINARY build argument. If this argument is unset, the Dockerfiles build the binaries from source as before; otherwise, they copy the provided binary into the image.

Changes

Files Change Summary
backend/Dockerfile, backend/Dockerfile.driver, backend/Dockerfile.launcher,
backend/Dockerfile.persistenceagent, backend/Dockerfile.scheduledworkflow
Added PREBUILT_BINARY build argument and conditional logic to either build from source or copy a prebuilt binary into the image.

Sequence Diagram(s)

sequenceDiagram
    participant Builder as Docker Build
    participant Source as Source Code
    participant Prebuilt as Prebuilt Binary (optional)
    participant Image as Docker Image

    Builder->>Builder: Check PREBUILT_BINARY argument
    alt PREBUILT_BINARY is unset
        Builder->>Source: Build binary from source
        Source->>Image: Provide built binary
    else PREBUILT_BINARY is set
        Builder->>Prebuilt: Copy prebuilt binary
        Prebuilt->>Image: Provide prebuilt binary
    end
    Image->>Builder: Finalize image with binary
Loading

Suggested labels

approved, lgtm

Suggested reviewers

  • rimolive

Poem

🐇
In Docker builds, a choice appears,
Build from source or skip the gears.
Prebuilt binaries hop right in,
Saving time, a clever win.
With flags to guide, the path is clear,
The rabbit cheers, the build is near!
🥕🚀


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between be65398 and a7f37ba.

📒 Files selected for processing (1)
  • backend/Dockerfile (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • backend/Dockerfile
⏰ Context from checks skipped due to timeout of 90000ms (3)
  • GitHub Check: KFP SDK Execution Tests - K8s v1.31.0
  • GitHub Check: KFP SDK Execution Tests - K8s v1.29.2
  • GitHub Check: check_ci_status

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@dsp-developers
Copy link

Change to PR detected. A new PR build was completed.
A set of new images have been built to help with testing out this PR:
API Server: quay.io/opendatahub/ds-pipelines-api-server:pr-183
DSP DRIVER: quay.io/opendatahub/ds-pipelines-driver:pr-183
DSP LAUNCHER: quay.io/opendatahub/ds-pipelines-launcher:pr-183
Persistence Agent: quay.io/opendatahub/ds-pipelines-persistenceagent:pr-183
Scheduled Workflow Manager: quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-183
MLMD Server: quay.io/opendatahub/mlmd-grpc-server:latest
MLMD Envoy Proxy: registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2
UI: quay.io/opendatahub/ds-pipelines-frontend:pr-183

@hbelmiro hbelmiro changed the title WIP - UPSTREAM: <carry>: Add support for optional prebuilt Go binaries in Dockerfiles UPSTREAM: <carry>: Add support for optional prebuilt Go binaries in Dockerfiles Jun 17, 2025
@hbelmiro hbelmiro marked this pull request as ready for review June 17, 2025 20:57
@openshift-ci openshift-ci bot requested review from mprahl and rimolive June 17, 2025 20:57
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

♻️ Duplicate comments (4)
backend/Dockerfile.scheduledworkflow (1)

44-45: Duplicate: default sentinel value for PREBUILT_BINARY
See the note in backend/Dockerfile.launcher about using ARG PREBUILT_BINARY="" and checking -n "$PREBUILT_BINARY" rather than "unset".

backend/Dockerfile (1)

34-36: Duplicate: default sentinel value for PREBUILT_BINARY
Align with other components by using ARG PREBUILT_BINARY="" and testing -n "$PREBUILT_BINARY" to avoid literal collisions with "unset".

backend/Dockerfile.driver (1)

37-38: Duplicate: default sentinel value for PREBUILT_BINARY
As above, default to "" and use -n "$PREBUILT_BINARY" instead of the literal "unset".

backend/Dockerfile.persistenceagent (1)

39-40: Duplicate: default sentinel value for PREBUILT_BINARY
Consistent with other Dockerfiles, switch to ARG PREBUILT_BINARY="" and check -n "$PREBUILT_BINARY" to avoid sentinel collisions.

🧹 Nitpick comments (1)
backend/Dockerfile.launcher (1)

39-41: Consider using an empty default for PREBUILT_BINARY
Using the literal "unset" risks colliding with a binary actually named unset. It’s more robust to default to "" and detect with -n, e.g.:

ARG PREBUILT_BINARY=""RUN if [ -n "$PREBUILT_BINARY" ]; then …; else …; fi
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between dca9085 and 9411f58.

📒 Files selected for processing (5)
  • backend/Dockerfile (1 hunks)
  • backend/Dockerfile.driver (1 hunks)
  • backend/Dockerfile.launcher (1 hunks)
  • backend/Dockerfile.persistenceagent (1 hunks)
  • backend/Dockerfile.scheduledworkflow (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (8)
  • GitHub Check: API integration tests v2 - K8s with kubernetes v1.29.2
  • GitHub Check: API integration tests v2 - K8s with database v1.29.2
  • GitHub Check: Basic Sample Tests - K8s v1.29.2
  • GitHub Check: API integration tests v2 - K8s with kubernetes v1.31.0
  • GitHub Check: Basic Sample Tests - K8s v1.31.0
  • GitHub Check: API integration tests v2 - K8s with database v1.31.0
  • GitHub Check: API integration tests v2 with proxy - K8s v1.31.0
  • GitHub Check: check_ci_status

Comment on lines +42 to +48
RUN if [ "$PREBUILT_BINARY" = "unset" ]; then \
GO111MODULE=on CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -tags netgo -ldflags '-extldflags "-static"' -o /bin/launcher-v2 ./backend/src/v2/cmd/launcher-v2/*.go && \
GO111MODULE=on CGO_ENABLED=1 GOOS=linux GOARCH=amd64 GOEXPERIMENT=strictfipsruntime go build -tags 'netgo strictfipsruntime' -o /bin/launcher-v2-fips ./backend/src/v2/cmd/launcher-v2/*.go ;\
else \
cp "${PREBUILT_BINARY}" /bin/launcher-v2 && \
cp "${PREBUILT_BINARY}" /bin/launcher-v2-fips ;\
fi
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Add explicit existence check and clear error for the prebuilt binary
Relying on cp alone yields a generic failure if the path is wrong. To improve diagnostics, wrap the copy in a file‐test, for example:

if [ -f "$PREBUILT_BINARY" ]; then \
  cp "$PREBUILT_BINARY" /bin/launcher-v2 && cp "$PREBUILT_BINARY" /bin/launcher-v2-fips; \
else \
  echo "Error: PREBUILT_BINARY '$PREBUILT_BINARY' not found" >&2; exit 1; \
fi
🤖 Prompt for AI Agents
In backend/Dockerfile.launcher around lines 42 to 48, the current script copies
the prebuilt binary without checking if the file exists, causing unclear errors
if the path is wrong. Modify the else block to first check if the file at
PREBUILT_BINARY exists using a file test. If it exists, proceed with copying to
/bin/launcher-v2 and /bin/launcher-v2-fips; if not, output a clear error message
to stderr and exit with status 1 to improve diagnostics.

Comment on lines +47 to 52
RUN if [ "$PREBUILT_BINARY" = "unset" ]; then \
GO111MODULE=on CGO_ENABLED=1 GOEXPERIMENT=strictfipsruntime go build -tags strictfipsruntime -o /bin/controller backend/src/crd/controller/scheduledworkflow/*.go; \
else \
cp ${PREBUILT_BINARY} /bin/controller; \
fi

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Strengthen the prebuilt-binary copy step

  • Quote the path to handle spaces:
    cp "${PREBUILT_BINARY}" /bin/controller
  • Add a [ -f ] test before cp to surface a clear “file not found” error instead of a cryptic cp failure.
🤖 Prompt for AI Agents
In backend/Dockerfile.scheduledworkflow around lines 47 to 52, the prebuilt
binary copy step should be improved by quoting the PREBUILT_BINARY variable to
handle paths with spaces and adding a file existence check before copying.
Modify the else branch to first test if the file exists using [ -f
"${PREBUILT_BINARY}" ] and if not, output a clear error message and exit; if it
exists, then copy it with cp "${PREBUILT_BINARY}" /bin/controller.

Comment on lines +39 to 45
# If PREBUILT_BINARY is set, use it; otherwise, build from source
RUN if [ "$PREBUILT_BINARY" = "unset" ]; then \
GO111MODULE=on CGO_ENABLED=1 GOOS=linux GOARCH=amd64 GOEXPERIMENT=strictfipsruntime go build -tags 'netgo strictfipsruntime' -o /bin/driver ./backend/src/v2/cmd/driver/*.go; \
else \
cp ${PREBUILT_BINARY} /bin/driver; \
fi

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Improve prebuilt-binary copy logic

  • Wrap the path in quotes: cp "${PREBUILT_BINARY}" /bin/driver
  • Pre-check with [ -f "$PREBUILT_BINARY" ] to fail fast with a clear error if missing.
🤖 Prompt for AI Agents
In backend/Dockerfile.driver around lines 39 to 45, the copy command for the
prebuilt binary should be improved by wrapping the PREBUILT_BINARY variable in
quotes to handle paths with spaces, and adding a pre-check using [ -f
"$PREBUILT_BINARY" ] to verify the file exists before copying. If the file is
missing, the build should fail immediately with a clear error message. Update
the RUN command to include these checks and quoted variable usage.

Comment on lines +41 to 47
# If PREBUILT_BINARY is set, use it; otherwise, build from source
RUN if [ "$PREBUILT_BINARY" = "unset" ]; then \
GO111MODULE=on CGO_ENABLED=1 GOEXPERIMENT=strictfipsruntime go build -tags strictfipsruntime -o /bin/persistence_agent backend/src/agent/persistence/*.go; \
else \
cp ${PREBUILT_BINARY} /bin/persistence_agent; \
fi

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Robustify the prebuilt-binary copy step

  • Quote the argument: cp "${PREBUILT_BINARY}" /bin/persistence_agent
  • Add an existence check to emit a clear “file not found” error before attempting the copy.
🤖 Prompt for AI Agents
In backend/Dockerfile.persistenceagent around lines 41 to 47, the cp command
copying the prebuilt binary should quote the variable to handle paths with
spaces by using cp "${PREBUILT_BINARY}" /bin/persistence_agent. Additionally,
add a check before the copy to verify the file exists, and if not, output a
clear error message and exit to prevent silent failures.

…ockerfiles

Signed-off-by: Helber Belmiro <helber.belmiro@gmail.com>
@hbelmiro hbelmiro changed the title UPSTREAM: <carry>: Add support for optional prebuilt Go binaries in Dockerfiles WIP - UPSTREAM: <carry>: Add support for optional prebuilt Go binaries in Dockerfiles Jun 17, 2025
@hbelmiro hbelmiro changed the title WIP - UPSTREAM: <carry>: Add support for optional prebuilt Go binaries in Dockerfiles UPSTREAM: <carry>: Add support for optional prebuilt Go binaries in Dockerfiles Jun 17, 2025
@dsp-developers
Copy link

Change to PR detected. A new PR build was completed.
A set of new images have been built to help with testing out this PR:
API Server: quay.io/opendatahub/ds-pipelines-api-server:pr-183
DSP DRIVER: quay.io/opendatahub/ds-pipelines-driver:pr-183
DSP LAUNCHER: quay.io/opendatahub/ds-pipelines-launcher:pr-183
Persistence Agent: quay.io/opendatahub/ds-pipelines-persistenceagent:pr-183
Scheduled Workflow Manager: quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-183
MLMD Server: quay.io/opendatahub/mlmd-grpc-server:latest
MLMD Envoy Proxy: registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2
UI: quay.io/opendatahub/ds-pipelines-frontend:pr-183

Signed-off-by: Helber Belmiro <helber.belmiro@gmail.com>
@hbelmiro hbelmiro marked this pull request as draft June 23, 2025 13:17
@hbelmiro hbelmiro marked this pull request as ready for review June 23, 2025 13:26
@openshift-ci openshift-ci bot requested a review from gmfrasca June 23, 2025 13:26
@dsp-developers
Copy link

Change to PR detected. A new PR build was completed.
A set of new images have been built to help with testing out this PR:
API Server: quay.io/opendatahub/ds-pipelines-api-server:pr-183
DSP DRIVER: quay.io/opendatahub/ds-pipelines-driver:pr-183
DSP LAUNCHER: quay.io/opendatahub/ds-pipelines-launcher:pr-183
Persistence Agent: quay.io/opendatahub/ds-pipelines-persistenceagent:pr-183
Scheduled Workflow Manager: quay.io/opendatahub/ds-pipelines-scheduledworkflow:pr-183
MLMD Server: quay.io/opendatahub/mlmd-grpc-server:latest
MLMD Envoy Proxy: registry.redhat.io/openshift-service-mesh/proxyv2-rhel8:2.3.9-2
UI: quay.io/opendatahub/ds-pipelines-frontend:pr-183

@VaniHaripriya
Copy link

/lgtm

Copy link

openshift-ci bot commented Jun 25, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: alyssacgoins
Once this PR has been reviewed and has the lgtm label, please assign mprahl for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants