Repository navigation
OpenLakeForge 0.2.0-alpha.1
Pre-releaseOpenLakeForge 0.2.0-alpha.1
The small-team adoption release (Milestone 2): OpenLakeForge installs from
PyPI without a checkout, scaffolds a data product without touching shared
platform code, and runs a slim profile with no product allowlist baked into
the tooling.
-
PyPI distribution:
pip install openlakeforgeinstalls theolfconsole
command with a verified, immutable Terraform/Helm/runtime payload embedded
in the wheel and sdist (#128, ADR 0009). -
olf initbootstraps a writablelakehouse_code/project from the
packaged demo in the current directory;olf init --emptycreates a
transitional project with no source, domain, or product yet (#146,
ADR 0009). -
A managed Terraform/Helm/kubectl/kind toolchain:
olfdownloads, verifies,
and privately invokes its own versioned copies underOLF_HOME, so none of
those tools need to be installed on the host.OLF_TOOLCHAIN_MODE=host
opts back into host-installed copies (#127, ADR 0008). -
SDK-managed AWS and Azure authentication:
olf auth login --provider aws|azuregoes through boto3 / the Azure SDK directly; theawsandaz
CLIs are no longer required (#142, ADR 0008). -
Golden-path scaffolding:
olf source new,olf domain new, and
olf product newgenerate a runnable Bronze source, Silver domain, or Gold
product from documented inputs, with no shared-code edit (#40). -
A typed domain inventory built from validated descriptors; every seed-
product allowlist is gone from shared platform code, so an added product
is discovered automatically (#39). -
Persistent Polaris catalog state: a Polaris pod restart no longer loses
table identity or requires a full platform re-apply (#79). -
A slim local profile that omits OpenMetadata and Superset, with e2e
assertions skipped rather than failed when a layer is absent (#78). -
A kind smoke gate (
olf smoke run) on every pull request: one product
pipeline through to a queryable Gold table, within a 45-minute budget
(#81). -
olfis now the only repository orchestration implementation; the
shell-scripted deploy path is gone.Makefiletargets are deprecated
one-line delegates to the equivalentolfcommand (#122-#126, ADR 0008). -
Dagster collapses to one merged code location by default; a per-domain
split is now an explicit configuration choice rather than the default
(#76, ADR 0006). -
lakehouse_code/replacesdomains/as the user-code root: Bronze is
source-owned, Silver is domain-owned, and Gold stays product-owned (#109,
ADR 0004). Theopenlakeforge.io/v1alpha3Lakehouse/Sourcedescriptor
pair replacesv1alpha1/v1alpha2Domaindescriptors; their loader and
validator are removed along withdocs/schema/domain*.jsonand the
v1alpha1->v1alpha2migration guide. -
scripts/release/verify-install.shis replaced byolf release verify-install. -
The decision log is consolidated from 32 ADRs to 10, renumbered
0001-0010
and rewritten to describe what binds today rather than stacking supersessions.
Each new ADR carries a History footer naming the records it absorbs. ADR
numbers referenced in earlier releases, commits, and closed pull requests
refer to the old numbering.
Consumers upgrading from v0.1.0-alpha.1 should:
- Stop cloning the repository to install:
pip install openlakeforge(or
uv tool install openlakeforge --python 3.12), thenolf initin an
empty project directory. See the README and
local installation guide. - Migrate any
domains/<domain>/domain.yamldescriptor to
lakehouse_code/lakehouse.yamlplus onelakehouse_code/bronze/<source>/ source.yamlper Bronze source (openlakeforge.io/v1alpha3). See
docs/reference/domain-descriptor.md. - Replace direct
scripts/*.shor checkoutmake <env>-*invocations with
the equivalentolfcommand; everyMakefiletarget still works as a
delegate, but is no longer the primary interface. - Expect further breaking changes in the next alpha; this stage carries no
forward compatibility guarantee.
- No stable support window is published before
v1.0; only the latest
alpha tag is maintained. - The Azure POC and AWS POC deployment targets remain proof-of-concept
scope; seedocs/architecture/aws-eks-poc.mdfor the current AWS
compatibility gate.
The first publishable OpenLakeForge alpha: a signed, SBOM'd, provenance-
attested release bundle built from the seed multi-product POC (Sales
order_revenue and customer_health, Supply Chain
inventory_reliability) across the local (kind), Azure POC (AKS), and AWS
POC (EKS) deployment targets.
.github/workflows/release.yml: tag-triggered (v*) and
workflow_dispatch(dry-run capable) release pipeline. Builds and pushes
project-codeandsupersetimages toghcr.io/malon64/openlakeforge/*
by digest, signs them keylessly with cosign (Sigstore OIDC, no long-lived
keys), generates an SPDX SBOM per image attached as both a cosign
attestation and a release asset, attaches SLSA build provenance via
actions/attest-build-provenance, and publishes the GitHub Release with
the changelog, the component manifest, the compatibility matrix, and
checksums.txt.olf releaseCLI group (tools/olf/olf/release.py):manifest,
checksums,compatibility-matrix, andcheck(the release-readiness /
clean-install consistency gate).make release-checkandmake release-bundletargets, and a
release-checkjob in.github/workflows/checks.ymlso release drift is
caught on every pull request, not only at tag time.docs/release/releasing.mdanddocs/release/compatibility-matrix.md.scripts/release/verify-install.sh: the scripted clean-checkout install
verification a consumer (or the maintainer, post-merge) runs against a
published tag.
This is the first tagged release; there is no prior version to migrate
from. Consumers adopting this alpha should:
- Pin to the immutable tag
v0.1.0-alpha.1(or the resolved image
digests recorded in the release'scomponent-manifest.json) rather than
:localormain. - Follow docs/release/releasing.md to verify
signatures and checksums before deploying. - Expect breaking changes in the next alpha; this stage carries no forward
compatibility guarantee (see "Lifecycle Definitions" in
docs/industrialization-roadmap.md).
- No stable support window is published before
v1.0; only the latest
alpha tag is maintained. - The Azure POC and AWS POC deployment targets remain proof-of-concept
scope; seedocs/architecture/aws-eks-poc.mdfor the current AWS
compatibility gate.
Verification
See docs/release/releasing.md for the full
consumer verification walkthrough (cosign signature verification,
checksum verification, and the clean-checkout install check).