Repository navigation
The first release of lake-builder: the image a lake in-cluster build Job runs. It fetches a lake repository at a pinned revision, builds it with an ephemeral rootless BuildKit daemon, and pushes the image and its registry cache with a space-scoped token. Tracking issue: #22.
This release was not planned through a release epic, so this page lists everything in the repository.
What it includes
The build Job (contract)
- One build per invocation. It runs as UID/GID
1000:1000and refuses UID 0.--versionprintslake-builder 0.1.0. - Validated input before anything runs:
LAKE_URL: HTTP(S), with no credentials, query or fragment.TARGETandCACHE_REF: registry references, with BuildKit option injection rejected. The cache must use the target's registry and space, and must not overwrite the target.CONTEXTandDOCKERFILE: repository-relative paths, or aninline:Dockerfile.BUILD_ARGS: checked against the operator'sBUILD_ARG_ALLOWlist.BUILD_IDorEVENTS_TOPIC, andXDG_RUNTIME_DIR.
- Source materialization through lake REST (
repository_tree,repository_file), always at the pinnedREVISION:- Only the chosen context and an explicitly requested Dockerfile are downloaded.
- Text and binary files are both supported, and file modes
100644/100755are kept. - Traversal,
.git, symlinks, special files, and duplicate or conflicting paths are refused. - Redirects are refused, proxy variables are ignored, and each response is capped at 64 MiB.
- The context's
.dockerignoreand a<Dockerfile>.dockerignoreare kept.
- Credentials:
- A private per-run
DOCKER_CONFIG(mode0600) holds the push credential (lake:<PUSH_TOKEN>). - Base-image pull credentials can be merged in from a mounted
.dockerconfigjson(BASE_REGISTRY_AUTH). The lake push entry always wins. - No token is passed to BuildKit's environment or arguments.
- Every token, and every imported credential, is redacted from logs and events.
- A private per-run
- Plain-HTTP in-cluster registry: when
LAKE_URLishttp://, the exporters getregistry.insecure=true. buildkitd also gets abuildkitd.tomlthat declares the target's registry hosthttp = true. - Ephemeral rootless BuildKit:
- The entrypoint starts
rootlesskit buildkitd, waiting with upstream's retry and back-off. - It builds through
buildctlwith thedockerfile.v0frontend forlinux/amd64, pushes the image, and imports and exports the registry cache (mode=max). - It stops the daemon on success, failure and SIGTERM.
- If the daemon never becomes ready, its log is reported.
- The entrypoint starts
- Result:
- The digest comes from BuildKit's metadata. The pushed manifest is re-read and its SHA-256 checked, following image indexes.
sizeis the unique compressed bytes reachable from the digest.- The final stdout line is
{"digest":"sha256:…","size":N}. A failure never prints it.
- Progress events to lake's
publish_build_event:- Phase events:
fetching,building,pushing,done. - Log batches of at most 64 KiB, sent at most once a second, with bounded lines.
- Error events.
- Delivery is best effort and never changes the build result.
- Phase events:
The image (image contents)
scratchplus an allow-list of files:- static BuildKit v0.33.0 (
buildkitd,buildctl,buildkit-runc) and RootlessKit v3.0.1, copied frommoby/buildkit:v0.33.0-rootless, pinned by index digest - shadow's
newuidmap/newgidmap, with their file capabilities, and libbsd and libmd - musl and the Mozilla CA bundle
- CPython 3.12.16 from python-build-standalone, pinned by SHA-256
- static BuildKit v0.33.0 (
- There is no shell, package manager, BusyBox, git, fuse-overlayfs or QEMU.
linux/amd64andlinux/arm64images can be built from theDockerfile.
Tooling and project files
make proof: Ruff lint, the format check and the native contract suite. The suite runs against a fake lake/registry server and fakebuildctl,buildkitdandrootlesskitprocesses.make image-check/tools/image-files.py: fails if a built image holds any file that isn't on the documented component list.tools/spdx-headers.py: adds or checksSPDX-License-Identifier: MIT.tools/sbom.py: writessbom.spdx.jsonandsbom.cdx.json.LICENSE(MIT),COPYRIGHT,THIRD_PARTY_LICENSES,SECURITY.md,CONTRIBUTING.md(with the CLA,CLA.md) andCODE_OF_CONDUCT.md.
What it does not include
- No published image. No image is signed or attested from this tag yet, and the Docker Hub repository stays private.
- No Dockerfile
ADDof a git URL, because the image has no git binary. - No cross-platform
RUN, because there is no QEMU. Builds targetlinux/amd64only. - No
--secretor--sshbuild inputs. Build arguments are not a secret channel. - No hosted CI as a release gate. The proof below was run locally.
Known issues
Security-relevant first.
-
The operator must isolate the Job. Builds run untrusted Dockerfiles. Running them safely needs a dedicated Job with:
- no service-account token, and per-space network isolation
- the rootless kernel permissions, resource limits and a deadline
The image does not enable
--oci-worker-no-process-sandbox. An operator that sets it, or that runs the pod with an unconfined seccomp profile, must provide that isolation itself. See the contract. -
The one licence exception, libseccomp (LGPL-2.1). It is statically linked into
buildkit-runcsoRUNsteps keep a seccomp filter.THIRD_PARTY_LICENSESgives its source and how to relink it. -
The uid-map helpers' licence comes from Alpine's metadata (BSD-3-Clause), not yet from a per-file audit of shadow's sources (#17). #15 stays open until that audit is done.
-
The image has not run end to end: no registry push, cache import, or cluster Job with an out-of-space refusal (#18).
-
test_log_batch_limits_and_redactionis timing-sensitive under full-suite load (#19). It passed in the runs below. -
Without fuse-overlayfs, fast rootless overlayfs needs Linux 5.11 or newer. On older kernels BuildKit falls back to the slower
nativesnapshotter.
Published image
oramasearch/lake-builder:v0.1.0 and oramasearch/lake-builder:main-7bef8ac name the same index, sha256:170a9363d70d4e3bad695e6f13ddcbc2e741f28c897b95e454efbe415d953ac7. It was built from a detached checkout of the tag:
| Platform | Manifest |
|---|---|
linux/amd64 |
sha256:43938515c2e24be24272ec19f30afe9b34e667bc6a5a17a85f34250e485a1d69 |
linux/arm64 |
sha256:2a7b29adf9d309c113a169fce9c7f9f521ac12120c1d8baa31d6547091646971 |
The Docker Hub repository is private: an anonymous read returns 404. On the published amd64 image, tools/image-files.py lists 637/637 entries, and --version prints lake-builder 0.1.0. The image is not signed.
Where it has run
- On a cluster: not yet. The dev-environment run of this commit, and the cluster Job check (#18), were not done before the cut.
- Locally, with Docker on arm64 macOS:
- the image built for
linux/amd64(emulated) andlinux/arm64 - a rootless smoke run in the arm64 image: the entrypoint's daemon code started
rootlesskit buildkitd - a Dockerfile step created uid 4242, so multi-UID mapping works
/proc/self/statusin the step showedSeccomp: 2- the daemon was stopped afterwards
- the image built for
Verification
| Proof | Result |
|---|---|
make proof on the release branch: ruff check, ruff format check, unittest |
0 lint errors, 0 files to reformat, 36 tests OK, 0 failed, 0 skipped |
docker buildx build --platform linux/amd64 at dev 8f86778 |
built; both python-build-standalone SHA-256 checks pass (amd64 here, arm64 earlier) |
tools/image-files.py on that amd64 image |
637 / 637 entries in a listed component, 0 unknown |
The same check on the old base image moby/buildkit:v0.33.0-rootless |
fails with 933 unknown files, BusyBox among them (the check catches GPL userland) |
tools/sbom.py --check, tools/spdx-headers.py --check |
current; 0 source files without the header |
| gitleaks 8.30.1 (all refs, 21 commits, and the tree); trufflehog 3.99.9 (history) | 0 findings |
| Registry push, cache import, cluster Job, out-of-space refusal | not run (#18) |