Skip to content

Conversation

@leodido
Copy link
Contributor

@leodido leodido commented Nov 19, 2025

Description

Export SOURCE_DATE_EPOCH environment variable for all build commands to enable reproducible builds without requiring .git directory.

Fixes https://linear.app/ona-team/issue/CLC-2097/improve-builds-determinism

Problem

Build commands that want deterministic timestamps currently need to call git directly:

COMMIT_TIME=$(git show -s --format=%cI ${__git_commit})

This fails in CI environments with:

  • Shallow clones (git clone --depth=1)
  • Incomplete .git history
  • No .git directory at all

Solution

Export SOURCE_DATE_EPOCH environment variable for all build commands.

Build commands can now use:

BUILD_TIME=$SOURCE_DATE_EPOCH

Why This Makes Sense

Leeway computes a deterministic timestamp once (from git commit or external SOURCE_DATE_EPOCH if set), then exports it for build commands to use. This creates a one-way flow:

Scenario 1: Git Available (Normal Case)

Leeway: Gets timestamp from git commit (e.g., 1705329600)
Leeway: Exports SOURCE_DATE_EPOCH=1705329600
Build commands: Use $SOURCE_DATE_EPOCH (no git needed!)

Scenario 2: No Git, External SOURCE_DATE_EPOCH Set

CI: Sets SOURCE_DATE_EPOCH=1700000000
Leeway: Reads it (no git available)
Leeway: Exports SOURCE_DATE_EPOCH=1700000000 (pass-through)
Build commands: Use $SOURCE_DATE_EPOCH

Scenario 3: No Git, No SOURCE_DATE_EPOCH

Leeway: No git, no external SOURCE_DATE_EPOCH
Leeway: Fails fast (prevents cache pollution)

The flow is: External Environment → Leeway → Build Commands (one-way, not circular).

The Benefit

Before this change:

buildCommand:
  - sh
  - -c
  - |
    # Build command needs git
    COMMIT_TIME=$(git show -s --format=%cI ${__git_commit})
    go build -ldflags "-X main.BuildTime=$COMMIT_TIME"
    # Fails if .git is incomplete or missing!

After this change:

buildCommand:
  - sh
  - -c
  - |
    # SOURCE_DATE_EPOCH is automatically set by leeway
    go build -ldflags "-X main.BuildTime=$SOURCE_DATE_EPOCH" -o app
    # Works even without .git directory!

Benefits

  • ✅ Works without .git directory (CI-friendly)
  • ✅ Standard approach (reproducible-builds.org)
  • ✅ Consistent with Docker builds (already uses SOURCE_DATE_EPOCH)
  • ✅ Same timestamp used for tar archives, Docker images, and build commands
  • ✅ Enables gitpod-next PR #11833 to work correctly

Implementation

Uses existing getDeterministicMtime() function which:

  1. Tries SOURCE_DATE_EPOCH env var first (if set externally)
  2. Falls back to git commit timestamp
  3. Returns a deterministic Unix timestamp

This timestamp is now exported to the environment for all build commands.

Testing

# Create test package:
packages:
  - name: test
    type: generic
    config:
      commands:
        - ["sh", "-c", "echo SOURCE_DATE_EPOCH=$SOURCE_DATE_EPOCH"]

# Build:
leeway build :test

# Expected: SOURCE_DATE_EPOCH=1705329600 (or similar)

Documentation

README.md has been updated with:

  • Description of the SOURCE_DATE_EPOCH environment variable
  • Example usage in BUILD.yaml
  • Benefits for CI environments
  • Link to https://reproducible-builds.org

Co-authored-by: Ona no-reply@ona.com

@leodido leodido force-pushed the leo/export-source-date-epoch branch from c3aadae to f1ae422 Compare November 19, 2025 16:07
@leodido leodido self-assigned this Nov 19, 2025
@leodido leodido requested a review from kylos101 November 19, 2025 16:07
@leodido leodido marked this pull request as ready for review November 19, 2025 16:10
@leodido leodido force-pushed the leo/export-source-date-epoch branch from 43ff444 to 49eefd6 Compare November 19, 2025 16:22
@leodido leodido requested a review from csweichel November 19, 2025 18:06
@leodido leodido force-pushed the leo/export-source-date-epoch branch from 49eefd6 to 6754212 Compare November 19, 2025 18:20
@leodido leodido changed the base branch from main to provenance-outside-tarball November 19, 2025 18:33
leodido added a commit that referenced this pull request Nov 19, 2025
Pass SOURCE_DATE_EPOCH as a Docker build arg to make the timestamp
value available inside Dockerfiles for custom build logic.

This allows Dockerfiles to use:
  ARG SOURCE_DATE_EPOCH
  RUN go build -ldflags "-X main.BuildTime=$SOURCE_DATE_EPOCH" -o app

Use cases:
- Embedding build timestamps in binaries
- Creating build metadata files
- Custom reproducible build logic inside Dockerfiles

Note: Image metadata determinism (layer timestamps, history, OCI
annotations) is already handled by PR #284 through the environment
variable. This only adds Dockerfile-level access to the value.

Co-authored-by: Ona <no-reply@ona.com>
@leodido leodido force-pushed the leo/export-source-date-epoch branch from 6754212 to 994b111 Compare November 19, 2025 18:52
leodido added a commit that referenced this pull request Nov 19, 2025
…mages

Pass SOURCE_DATE_EPOCH as a Docker build arg to enable deterministic
Docker image timestamps.

Dockerfiles MUST declare ARG SOURCE_DATE_EPOCH for BuildKit to use the
timestamp for image metadata:

  FROM alpine:3.18
  ARG SOURCE_DATE_EPOCH

Without this ARG declaration, images will have non-deterministic
timestamps even though the environment variable is set (from PR #284).

With the ARG, BuildKit uses SOURCE_DATE_EPOCH for:
- Image metadata timestamps (created field)
- History timestamps
- OCI annotations

The ARG is also available in RUN commands for custom build logic:
  RUN go build -ldflags "-X main.BuildTime=$SOURCE_DATE_EPOCH" -o app

Co-authored-by: Ona <no-reply@ona.com>
@leodido leodido force-pushed the leo/export-source-date-epoch branch from 994b111 to be57173 Compare November 19, 2025 19:05
leodido added a commit that referenced this pull request Nov 19, 2025
…mages

Pass SOURCE_DATE_EPOCH as a Docker build arg to enable deterministic
Docker image timestamps.

Dockerfiles MUST declare ARG SOURCE_DATE_EPOCH for BuildKit to use the
timestamp for image metadata:

  FROM alpine:3.18
  ARG SOURCE_DATE_EPOCH

Without this ARG declaration, images will have non-deterministic
timestamps even though the environment variable is set (from PR #284).

With the ARG, BuildKit uses SOURCE_DATE_EPOCH for:
- Image metadata timestamps (created field)
- History timestamps
- OCI annotations

The ARG is also available in RUN commands for custom build logic:
  RUN go build -ldflags "-X main.BuildTime=$SOURCE_DATE_EPOCH" -o app

Co-authored-by: Ona <no-reply@ona.com>
leodido added a commit that referenced this pull request Nov 19, 2025
…mages

Pass SOURCE_DATE_EPOCH as a Docker build arg to enable deterministic
Docker image timestamps.

Dockerfiles MUST declare ARG SOURCE_DATE_EPOCH for BuildKit to use the
timestamp for image metadata:

  FROM alpine:3.18
  ARG SOURCE_DATE_EPOCH

Without this ARG declaration, images will have non-deterministic
timestamps even though the environment variable is set (from PR #284).

With the ARG, BuildKit uses SOURCE_DATE_EPOCH for:
- Image metadata timestamps (created field)
- History timestamps
- OCI annotations

The ARG is also available in RUN commands for custom build logic:
  RUN go build -ldflags "-X main.BuildTime=$SOURCE_DATE_EPOCH" -o app

Co-authored-by: Ona <no-reply@ona.com>
leodido added a commit that referenced this pull request Nov 19, 2025
…mages

Pass SOURCE_DATE_EPOCH as a Docker build arg to enable deterministic
Docker image timestamps.

Dockerfiles MUST declare ARG SOURCE_DATE_EPOCH for BuildKit to use the
timestamp for image metadata:

  FROM alpine:3.18
  ARG SOURCE_DATE_EPOCH

Without this ARG declaration, images will have non-deterministic
timestamps even though the environment variable is set (from PR #284).

With the ARG, BuildKit uses SOURCE_DATE_EPOCH for:
- Image metadata timestamps (created field)
- History timestamps
- OCI annotations

The ARG is also available in RUN commands for custom build logic:
  RUN go build -ldflags "-X main.BuildTime=$SOURCE_DATE_EPOCH" -o app

Co-authored-by: Ona <no-reply@ona.com>
leodido added a commit that referenced this pull request Nov 19, 2025
…mages

Pass SOURCE_DATE_EPOCH as a Docker build arg to enable deterministic
Docker image timestamps.

Dockerfiles MUST declare ARG SOURCE_DATE_EPOCH for BuildKit to use the
timestamp for image metadata:

  FROM alpine:3.18
  ARG SOURCE_DATE_EPOCH

Without this ARG declaration, images will have non-deterministic
timestamps even though the environment variable is set (from PR #284).

With the ARG, BuildKit uses SOURCE_DATE_EPOCH for:
- Image metadata timestamps (created field)
- History timestamps
- OCI annotations

The ARG is also available in RUN commands for custom build logic:
  RUN go build -ldflags "-X main.BuildTime=$SOURCE_DATE_EPOCH" -o app

Co-authored-by: Ona <no-reply@ona.com>
Copy link
Member

@geropl geropl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM ✔️

@leodido leodido changed the base branch from provenance-outside-tarball to main November 20, 2025 09:28
Export SOURCE_DATE_EPOCH environment variable for all build commands
to enable reproducible builds without requiring .git directory.

Build commands can use SOURCE_DATE_EPOCH directly for deterministic timestamps.

Benefits:
- Works in CI environments without full .git history
- Standard approach (reproducible-builds.org)
- Same timestamp used for tar archives and Docker images
- No dependency on git being available

Co-authored-by: Ona <no-reply@ona.com>
@leodido leodido force-pushed the leo/export-source-date-epoch branch from be57173 to 015b47f Compare November 20, 2025 09:33
@leodido leodido merged commit 5d71f0e into main Nov 20, 2025
6 checks passed
leodido added a commit that referenced this pull request Nov 20, 2025
…mages

Pass SOURCE_DATE_EPOCH as a Docker build arg to enable deterministic
Docker image timestamps.

Dockerfiles MUST declare ARG SOURCE_DATE_EPOCH for BuildKit to use the
timestamp for image metadata:

  FROM alpine:3.18
  ARG SOURCE_DATE_EPOCH

Without this ARG declaration, images will have non-deterministic
timestamps even though the environment variable is set (from PR #284).

With the ARG, BuildKit uses SOURCE_DATE_EPOCH for:
- Image metadata timestamps (created field)
- History timestamps
- OCI annotations

The ARG is also available in RUN commands for custom build logic:
  RUN go build -ldflags "-X main.BuildTime=$SOURCE_DATE_EPOCH" -o app

Co-authored-by: Ona <no-reply@ona.com>
leodido added a commit that referenced this pull request Nov 20, 2025
…mages

Pass SOURCE_DATE_EPOCH as a Docker build arg to enable deterministic
Docker image timestamps.

Dockerfiles MUST declare ARG SOURCE_DATE_EPOCH for BuildKit to use the
timestamp for image metadata:

  FROM alpine:3.18
  ARG SOURCE_DATE_EPOCH

Without this ARG declaration, images will have non-deterministic
timestamps even though the environment variable is set (from PR #284).

With the ARG, BuildKit uses SOURCE_DATE_EPOCH for:
- Image metadata timestamps (created field)
- History timestamps
- OCI annotations

The ARG is also available in RUN commands for custom build logic:
  RUN go build -ldflags "-X main.BuildTime=$SOURCE_DATE_EPOCH" -o app

Co-authored-by: Ona <no-reply@ona.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants