Skip to content

v2.5.2 — FinLakehouse deployment baseline

Latest

Choose a tag to compare

@Jiahong-Que-9527 Jiahong-Que-9527 released this 30 May 15:53
40d4f40

Summary

This release is the FinLakehouse deployment baseline for the v2.5.x line.
It includes the all-layer Iceberg migration from ADR-020 and the complete FLH
deployment documentation package.

Use this release when deploying the first independent FinLakehouse VPS runtime.

Release Coordinate

  • Tag: v2.5.2
  • Target commit: 40d4f40bcbc8f2a8a775cb9d5d12e60132cb1a9f
  • Base runtime family: v2.5.1 + ADR-020 all-layer Iceberg
  • Deployment target: FinLakehouse on a dedicated VPS

What Changed Since v2.5.1

  • Migrated Bronze, Silver, and Gold to native Iceberg writes through pyiceberg.
  • Added ingestion/iceberg_io.py as the shared Iceberg append / overwrite / scan layer.
  • Added ingestion/iceberg_schemas.py with the six medallion table schemas.
  • Added scripts/init-iceberg-namespaces.py and wired it into make up.
  • Replaced the former Parquet + Hive staging + Trino CTAS Gold path with iceberg.gold.ecb_dax_features.
  • Added the FinLakehouse .env template and deployment guide.
  • Added the Chinese VPS deployment runbook with end-to-end acceptance steps.
  • Added the Iceberg restore drill template required before 24/7 operation.
  • Added the 2026-05-30 project-state snapshot documenting FLH deployment readiness.

FinLakehouse Deployment Entry Points

  • Main guide: docs/finlakehouse-deployment-guide.md
  • Chinese VPS runbook: docs/vps-deployment-runbook.md
  • FLH env template: docs/finlakehouse-env-template.env
  • Current snapshot: docs/项目现状总览_2026-05-30.md
  • Restore drill template: docs/restore-drills/TEMPLATE-iceberg-restore-drill.md

Required FLH Environment Values

Copy docs/finlakehouse-env-template.env to /opt/finlakehouse/.env and replace
every CHANGE_ME value before starting services.

Required identity and storage values:

PRODUCT_ID=finlakehouse
PRODUCT_DISPLAY_NAME=FinLakehouse
ENVIRONMENT=prod
COMPOSE_PROJECT_NAME=finlakehouse
TRINO_USER=finlakehouse
DATA_BUCKET=finlakehouse-data
AUDIT_BUCKET=finlakehouse-audit
MLFLOW_ARTIFACT_BUCKET=finlakehouse-mlflow
WAREHOUSE_URI=s3a://finlakehouse-data/warehouse/

Mandatory secrets to fill:

  • MINIO_ROOT_PASSWORD
  • POSTGRES_PASSWORD
  • S3_SECRET_KEY
  • AWS_SECRET_ACCESS_KEY
  • SUPERSET_SECRET_KEY
  • SUPERSET_ADMIN_PASSWORD

S3_SECRET_KEY and AWS_SECRET_ACCESS_KEY must match MINIO_ROOT_PASSWORD.

Deployment Acceptance Commands

Run these from /opt/finlakehouse/app after creating /opt/finlakehouse/.env:

ENV_FILE=/opt/finlakehouse/.env make up
ENV_FILE=/opt/finlakehouse/.env make verify
ENV_FILE=/opt/finlakehouse/.env make pipeline
ENV_FILE=/opt/finlakehouse/.env make demo

Expected make demo ending:

Demo check      Rows  Status
--------------- ----- ------
Iceberg Gold    53    PASS

Manual Trino checks:

SELECT count(*) FROM iceberg.bronze.ecb_rates;
SELECT count(*) FROM iceberg.bronze.dax_daily;
SELECT count(*) FROM iceberg.silver.ecb_rates_cleaned;
SELECT count(*) FROM iceberg.silver.dax_daily_cleaned;
SELECT count(*) FROM iceberg.gold.ecb_dax_features;

Verification Before Release

  • GitHub Actions PR #25 CI run #154: success.
  • GitHub Actions PR #26 CI run #157: success.
  • Local verification before snapshot commit:
    • make lint
    • make typecheck
    • make test (69 passed)

Live Operation Gate

This release is ready for FLH deployment validation.

Do not declare FinLakehouse live for 24/7 operation until:

  • make verify passes on the FLH VPS.
  • make pipeline succeeds.
  • make demo returns Iceberg Gold PASS.
  • Trino can query Bronze, Silver, and Gold Iceberg tables.
  • MLflow shows at least one ecb_dax_impact run.
  • The FinLakehouse Iceberg restore drill is completed and recorded as PASS under docs/restore-drills/.

Known Boundaries

  • Runtime remains single-host Docker Compose.
  • Object storage remains MinIO.
  • Secrets are still managed through /opt/finlakehouse/.env.
  • Audit evidence packs are planned for v2.6; the audit bucket exists but is not yet written by the pipeline.
  • DAX data remains the checked-in sample CSV, not a live market API feed.