Skip to content

Add DuckLake with MinIO object storage support#8

Merged
EDsCODE merged 2 commits intomainfrom
feat/ducklake-minio-setup
Dec 5, 2025
Merged

Add DuckLake with MinIO object storage support#8
EDsCODE merged 2 commits intomainfrom
feat/ducklake-minio-setup

Conversation

@EDsCODE
Copy link
Copy Markdown
Contributor

@EDsCODE EDsCODE commented Dec 5, 2025

Summary

  • Add docker-compose.yaml with PostgreSQL (metadata catalog) and MinIO (S3-compatible object storage)
  • Add S3 credential configuration with two providers:
    • config: explicit credentials for MinIO or access keys
    • credential_chain: AWS SDK chain (env, config, sts, sso, instance, process)
  • Add scripts/seed_ducklake.sh to populate DuckLake catalog with sample e-commerce/analytics data
  • Add duckgres.yaml pre-configured for local Docker setup
  • Update README with setup instructions and configuration documentation

New Configuration Options

ducklake:
  metadata_store: "postgres:..."
  object_store: "s3://bucket/path/"
  s3_provider: "config"           # or "credential_chain"
  s3_endpoint: "localhost:9000"   # for MinIO
  s3_access_key: "..."
  s3_secret_key: "..."
  s3_region: "us-east-1"
  s3_use_ssl: false
  s3_url_style: "path"            # or "vhost"
  s3_chain: "env;config"          # for credential_chain
  s3_profile: "my-profile"        # for credential_chain

Test plan

  • Run docker compose up -d to start PostgreSQL and MinIO
  • Run ./duckgres --config duckgres.yaml to start server
  • Run ./scripts/seed_ducklake.sh to seed sample data
  • Verify data in MinIO console at http://localhost:9001
  • Query tables via psql: SELECT * FROM ducklake.main.products LIMIT 5;

🤖 Generated with Claude Code

EDsCODE and others added 2 commits December 4, 2025 16:47
- Add docker-compose.yaml with PostgreSQL (metadata) and MinIO (object storage)
- Add S3 credential configuration with two providers:
  - "config": explicit credentials for MinIO or access keys
  - "credential_chain": AWS SDK chain (env, config, sts, sso, instance, process)
- Add seed script to populate DuckLake catalog with sample data
- Add duckgres.yaml pre-configured for local Docker setup
- Update README with setup instructions and configuration docs

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@EDsCODE EDsCODE requested a review from fuziontech December 5, 2025 00:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants