Skip to content

Release v0.1.0

Latest

Choose a tag to compare

@homer6 homer6 released this 16 Nov 05:53
7bef8ac

The first release of lake-builder: the image a lake in-cluster build Job runs. It fetches a lake repository at a pinned revision, builds it with an ephemeral rootless BuildKit daemon, and pushes the image and its registry cache with a space-scoped token. Tracking issue: #22.

This release was not planned through a release epic, so this page lists everything in the repository.

What it includes

The build Job (contract)

  • One build per invocation. It runs as UID/GID 1000:1000 and refuses UID 0. --version prints lake-builder 0.1.0.
  • Validated input before anything runs:
    • LAKE_URL: HTTP(S), with no credentials, query or fragment.
    • TARGET and CACHE_REF: registry references, with BuildKit option injection rejected. The cache must use the target's registry and space, and must not overwrite the target.
    • CONTEXT and DOCKERFILE: repository-relative paths, or an inline: Dockerfile.
    • BUILD_ARGS: checked against the operator's BUILD_ARG_ALLOW list.
    • BUILD_ID or EVENTS_TOPIC, and XDG_RUNTIME_DIR.
  • Source materialization through lake REST (repository_tree, repository_file), always at the pinned REVISION:
    • Only the chosen context and an explicitly requested Dockerfile are downloaded.
    • Text and binary files are both supported, and file modes 100644/100755 are kept.
    • Traversal, .git, symlinks, special files, and duplicate or conflicting paths are refused.
    • Redirects are refused, proxy variables are ignored, and each response is capped at 64 MiB.
    • The context's .dockerignore and a <Dockerfile>.dockerignore are kept.
  • Credentials:
    • A private per-run DOCKER_CONFIG (mode 0600) holds the push credential (lake:<PUSH_TOKEN>).
    • Base-image pull credentials can be merged in from a mounted .dockerconfigjson (BASE_REGISTRY_AUTH). The lake push entry always wins.
    • No token is passed to BuildKit's environment or arguments.
    • Every token, and every imported credential, is redacted from logs and events.
  • Plain-HTTP in-cluster registry: when LAKE_URL is http://, the exporters get registry.insecure=true. buildkitd also gets a buildkitd.toml that declares the target's registry host http = true.
  • Ephemeral rootless BuildKit:
    • The entrypoint starts rootlesskit buildkitd, waiting with upstream's retry and back-off.
    • It builds through buildctl with the dockerfile.v0 frontend for linux/amd64, pushes the image, and imports and exports the registry cache (mode=max).
    • It stops the daemon on success, failure and SIGTERM.
    • If the daemon never becomes ready, its log is reported.
  • Result:
    • The digest comes from BuildKit's metadata. The pushed manifest is re-read and its SHA-256 checked, following image indexes.
    • size is the unique compressed bytes reachable from the digest.
    • The final stdout line is {"digest":"sha256:…","size":N}. A failure never prints it.
  • Progress events to lake's publish_build_event:
    • Phase events: fetching, building, pushing, done.
    • Log batches of at most 64 KiB, sent at most once a second, with bounded lines.
    • Error events.
    • Delivery is best effort and never changes the build result.

The image (image contents)

  • scratch plus an allow-list of files:
    • static BuildKit v0.33.0 (buildkitd, buildctl, buildkit-runc) and RootlessKit v3.0.1, copied from moby/buildkit:v0.33.0-rootless, pinned by index digest
    • shadow's newuidmap/newgidmap, with their file capabilities, and libbsd and libmd
    • musl and the Mozilla CA bundle
    • CPython 3.12.16 from python-build-standalone, pinned by SHA-256
  • There is no shell, package manager, BusyBox, git, fuse-overlayfs or QEMU.
  • linux/amd64 and linux/arm64 images can be built from the Dockerfile.

Tooling and project files

  • make proof: Ruff lint, the format check and the native contract suite. The suite runs against a fake lake/registry server and fake buildctl, buildkitd and rootlesskit processes.
  • make image-check / tools/image-files.py: fails if a built image holds any file that isn't on the documented component list.
  • tools/spdx-headers.py: adds or checks SPDX-License-Identifier: MIT.
  • tools/sbom.py: writes sbom.spdx.json and sbom.cdx.json.
  • LICENSE (MIT), COPYRIGHT, THIRD_PARTY_LICENSES, SECURITY.md, CONTRIBUTING.md (with the CLA, CLA.md) and CODE_OF_CONDUCT.md.

What it does not include

  • No published image. No image is signed or attested from this tag yet, and the Docker Hub repository stays private.
  • No Dockerfile ADD of a git URL, because the image has no git binary.
  • No cross-platform RUN, because there is no QEMU. Builds target linux/amd64 only.
  • No --secret or --ssh build inputs. Build arguments are not a secret channel.
  • No hosted CI as a release gate. The proof below was run locally.

Known issues

Security-relevant first.

  • The operator must isolate the Job. Builds run untrusted Dockerfiles. Running them safely needs a dedicated Job with:

    • no service-account token, and per-space network isolation
    • the rootless kernel permissions, resource limits and a deadline

    The image does not enable --oci-worker-no-process-sandbox. An operator that sets it, or that runs the pod with an unconfined seccomp profile, must provide that isolation itself. See the contract.

  • The one licence exception, libseccomp (LGPL-2.1). It is statically linked into buildkit-runc so RUN steps keep a seccomp filter. THIRD_PARTY_LICENSES gives its source and how to relink it.

  • The uid-map helpers' licence comes from Alpine's metadata (BSD-3-Clause), not yet from a per-file audit of shadow's sources (#17). #15 stays open until that audit is done.

  • The image has not run end to end: no registry push, cache import, or cluster Job with an out-of-space refusal (#18).

  • test_log_batch_limits_and_redaction is timing-sensitive under full-suite load (#19). It passed in the runs below.

  • Without fuse-overlayfs, fast rootless overlayfs needs Linux 5.11 or newer. On older kernels BuildKit falls back to the slower native snapshotter.

Published image

oramasearch/lake-builder:v0.1.0 and oramasearch/lake-builder:main-7bef8ac name the same index, sha256:170a9363d70d4e3bad695e6f13ddcbc2e741f28c897b95e454efbe415d953ac7. It was built from a detached checkout of the tag:

Platform Manifest
linux/amd64 sha256:43938515c2e24be24272ec19f30afe9b34e667bc6a5a17a85f34250e485a1d69
linux/arm64 sha256:2a7b29adf9d309c113a169fce9c7f9f521ac12120c1d8baa31d6547091646971

The Docker Hub repository is private: an anonymous read returns 404. On the published amd64 image, tools/image-files.py lists 637/637 entries, and --version prints lake-builder 0.1.0. The image is not signed.

Where it has run

  • On a cluster: not yet. The dev-environment run of this commit, and the cluster Job check (#18), were not done before the cut.
  • Locally, with Docker on arm64 macOS:
    • the image built for linux/amd64 (emulated) and linux/arm64
    • a rootless smoke run in the arm64 image: the entrypoint's daemon code started rootlesskit buildkitd
    • a Dockerfile step created uid 4242, so multi-UID mapping works
    • /proc/self/status in the step showed Seccomp: 2
    • the daemon was stopped afterwards

Verification

Proof Result
make proof on the release branch: ruff check, ruff format check, unittest 0 lint errors, 0 files to reformat, 36 tests OK, 0 failed, 0 skipped
docker buildx build --platform linux/amd64 at dev 8f86778 built; both python-build-standalone SHA-256 checks pass (amd64 here, arm64 earlier)
tools/image-files.py on that amd64 image 637 / 637 entries in a listed component, 0 unknown
The same check on the old base image moby/buildkit:v0.33.0-rootless fails with 933 unknown files, BusyBox among them (the check catches GPL userland)
tools/sbom.py --check, tools/spdx-headers.py --check current; 0 source files without the header
gitleaks 8.30.1 (all refs, 21 commits, and the tree); trufflehog 3.99.9 (history) 0 findings
Registry push, cache import, cluster Job, out-of-space refusal not run (#18)