feat: interactive ColdFront walkthrough (4 demos + Codespaces)#38
feat: interactive ColdFront walkthrough (4 demos + Codespaces)#38AntTheLimey wants to merge 42 commits into
Conversation
…rrect service count in README
…docs, check compose v2
…vcontainer fail-fast
…p pre-menu Phase B
…res, scale-out bridge
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
📝 WalkthroughWalkthroughThis PR introduces an interactive ColdFront walkthrough: a devcontainer configuration and post-create provisioning script, a new CI workflow ( ChangesColdFront Walkthrough Onboarding
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Up to standards ✅🟢 Issues
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (4)
docs/walkthrough.md (1)
32-39: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winPrerequisites list omits
jq.
examples/walkthrough/setup.sh'sREQUIRED_CMDSincludesdocker curl jq psql(plusss/lsof), but the docs list only mentionscurl,bash, andpsql. Readers following the docs without running the one-liner may hit an avoidable missing-dependency error.📝 Proposed fix
-- `curl`, `bash`, and `psql` on the host (the psql commands connect to - the published port). +- `curl`, `bash`, `jq`, and `psql` on the host (the psql commands connect + to the published port).🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/walkthrough.md` around lines 32 - 39, The prerequisites section is missing jq, which is required by examples/walkthrough/setup.sh via REQUIRED_CMDS. Update the walkthrough prerequisites list to include jq alongside curl, bash, and psql, and keep the wording aligned with the setup.sh dependency check so readers don’t miss the needed host tools..github/workflows/ci-walkthrough.yml (3)
54-63: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winExited-container check is a no-op —
docker compose pshides exited containers by default.Per Docker's docs,
docker compose ps"By default, only running containers are shown";--allis required to include stopped/exited ones. As written,grep -E "Exit|exited"will never match anything, since exited containers are already filtered out — this step can never catch a service that crashed and will always report "All services are running."Note: if you add
--all, ensure the filter also excludes services that intentionally complete and exit (e.g.,lakekeeper-migrate, a one-shot migration container perdocker-compose.yml), to avoid a new false-positive.🔧 Proposed fix
- docker compose -f examples/walkthrough/docker-compose.yml ps - if docker compose -f examples/walkthrough/docker-compose.yml ps | grep -E "Exit|exited"; then + docker compose -f examples/walkthrough/docker-compose.yml ps --all + if docker compose -f examples/walkthrough/docker-compose.yml ps --all --status=exited \ + --filter "name=events" | grep -vE "lakekeeper-migrate" | grep -qE "Exit|exited"; then🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci-walkthrough.yml around lines 54 - 63, The exited-container check in the CI walkthrough is ineffective because `docker compose ps` only shows running containers by default, so the `grep` in the service-status step can never detect crashed services. Update the check in the workflow step that uses `docker compose -f examples/walkthrough/docker-compose.yml ps` to include exited containers, and then refine the filter so intentionally one-shot services like `lakekeeper-migrate` are excluded from failure detection. Keep the existing `docker compose ... logs` and failure path, but make the status check reliable for actual crashes.
13-23: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winValidate
examples/walkthrough/docker-compose.mesh.ymltoo. TheValidate Docker Composestep only checksexamples/walkthrough/docker-compose.yml, so the distributed demo compose file can drift without CI catching it.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci-walkthrough.yml around lines 13 - 23, The Validate Docker Compose step only checks the main walkthrough compose file, so update the workflow’s validate job to also run docker compose config for examples/walkthrough/docker-compose.mesh.yml. Use the existing Validate Docker Compose step in ci-walkthrough and extend it so both compose files are validated in CI, preventing the mesh demo config from drifting unnoticed.
1-12: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winAdd
packages: readto the walkthrough job
.github/workflows/ci-walkthrough.yml:24-36needs explicit package scope for the GHCR login step; addpermissions: packages: readso the base-image pull keeps working under restricted default token settings.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/ci-walkthrough.yml around lines 1 - 12, The walkthrough workflow is missing explicit package read permissions, which can break the GHCR base-image pull under restricted token defaults. Update the ci-walkthrough workflow by adding a permissions block with packages: read for the walkthrough job so the GHCR login step continues to work; use the existing workflow/job definition in ci-walkthrough.yml to place the change alongside the job configuration.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.devcontainer/post-create.sh:
- Around line 6-16: The provisioning script is missing the Linux networking
dependency needed by the walkthrough prereq check. Update the post-create flow
in the .devcontainer/post-create.sh script where it installs packages and runs
examples/walkthrough/setup.sh to also install iproute2 alongside
postgresql-client and jq, matching the dependency expected by setup.sh and the
CI walkthrough setup.
---
Outside diff comments:
In @.github/workflows/ci-walkthrough.yml:
- Around line 54-63: The exited-container check in the CI walkthrough is
ineffective because `docker compose ps` only shows running containers by
default, so the `grep` in the service-status step can never detect crashed
services. Update the check in the workflow step that uses `docker compose -f
examples/walkthrough/docker-compose.yml ps` to include exited containers, and
then refine the filter so intentionally one-shot services like
`lakekeeper-migrate` are excluded from failure detection. Keep the existing
`docker compose ... logs` and failure path, but make the status check reliable
for actual crashes.
- Around line 13-23: The Validate Docker Compose step only checks the main
walkthrough compose file, so update the workflow’s validate job to also run
docker compose config for examples/walkthrough/docker-compose.mesh.yml. Use the
existing Validate Docker Compose step in ci-walkthrough and extend it so both
compose files are validated in CI, preventing the mesh demo config from drifting
unnoticed.
- Around line 1-12: The walkthrough workflow is missing explicit package read
permissions, which can break the GHCR base-image pull under restricted token
defaults. Update the ci-walkthrough workflow by adding a permissions block with
packages: read for the walkthrough job so the GHCR login step continues to work;
use the existing workflow/job definition in ci-walkthrough.yml to place the
change alongside the job configuration.
In `@docs/walkthrough.md`:
- Around line 32-39: The prerequisites section is missing jq, which is required
by examples/walkthrough/setup.sh via REQUIRED_CMDS. Update the walkthrough
prerequisites list to include jq alongside curl, bash, and psql, and keep the
wording aligned with the setup.sh dependency check so readers don’t miss the
needed host tools.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 2aec7f1c-f59a-4fc9-aa1c-0c070d453e71
📒 Files selected for processing (5)
.devcontainer/devcontainer.json.devcontainer/post-create.sh.github/workflows/ci-walkthrough.ymldocs/walkthrough.mdexamples/walkthrough/README.md
✅ Files skipped from review due to trivial changes (1)
- examples/walkthrough/README.md
| # Install PostgreSQL client and jq | ||
| echo "Installing jq and the PostgreSQL client..." | ||
| sudo apt-get update -qq | ||
| sudo apt-get install -y -qq postgresql-client jq | ||
|
|
||
| # Run the prerequisites check | ||
| # Note: If the published base image is awkward to pull in CI/Codespaces, | ||
| # set COLDFRONT_BASE to build the base locally: | ||
| # export COLDFRONT_BASE=local && bash examples/walkthrough/setup.sh | ||
| echo "" | ||
| bash examples/walkthrough/setup.sh |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟠 Major | ⚡ Quick win
Missing iproute2 package — setup.sh will fail its prerequisites check.
examples/walkthrough/setup.sh requires ss (from iproute2) on Linux hosts, and the CI workflow (ci-walkthrough.yml) explicitly installs iproute2 for this same check. This script only installs postgresql-client and jq, so if the Codespaces base image doesn't already ship iproute2, the prerequisites check will fail during provisioning.
🔧 Proposed fix
echo "Installing jq and the PostgreSQL client..."
sudo apt-get update -qq
-sudo apt-get install -y -qq postgresql-client jq
+sudo apt-get install -y -qq postgresql-client jq iproute2📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Install PostgreSQL client and jq | |
| echo "Installing jq and the PostgreSQL client..." | |
| sudo apt-get update -qq | |
| sudo apt-get install -y -qq postgresql-client jq | |
| # Run the prerequisites check | |
| # Note: If the published base image is awkward to pull in CI/Codespaces, | |
| # set COLDFRONT_BASE to build the base locally: | |
| # export COLDFRONT_BASE=local && bash examples/walkthrough/setup.sh | |
| echo "" | |
| bash examples/walkthrough/setup.sh | |
| # Install PostgreSQL client and jq | |
| echo "Installing jq and the PostgreSQL client..." | |
| sudo apt-get update -qq | |
| sudo apt-get install -y -qq postgresql-client jq iproute2 | |
| # Run the prerequisites check | |
| # Note: If the published base image is awkward to pull in CI/Codespaces, | |
| # set COLDFRONT_BASE to build the base locally: | |
| # export COLDFRONT_BASE=local && bash examples/walkthrough/setup.sh | |
| echo "" | |
| bash examples/walkthrough/setup.sh |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.devcontainer/post-create.sh around lines 6 - 16, The provisioning script is
missing the Linux networking dependency needed by the walkthrough prereq check.
Update the post-create flow in the .devcontainer/post-create.sh script where it
installs packages and runs examples/walkthrough/setup.sh to also install
iproute2 alongside postgresql-client and jq, matching the dependency expected by
setup.sh and the CI walkthrough setup.
…healthcheck, CI-check, and doc fixes
|
Pushed fixes to the branch (
These are based on current |
2 participants
What this adds
An interactive, self-contained ColdFront walkthrough — a guided demo that
doubles as a manual test harness — plus its user docs and a Codespaces
devcontainer.
examples/walkthrough/guide.sh(+runner.sh): interactive guide with four demosvisibility + concurrent cold-write serialization (the R-A bakery), no 409s
examples/walkthrough/docker-compose.mesh.yml— on-demand 2-node mesh stack forthe Distributed demo (single↔mesh stack switch; a laptop can't hold both)
docs/walkthrough.md— the published, copy-pasteable version of the guide.devcontainer/walkthrough/— Codespaces path (auto-launches the guide)Merges current
main, which the Distributed demo relies on — notably#36 (node-local advisory lock; makes multi-writer-per-node cold writes
conflict-free), plus #31 (lakekeeper healthcheck) and #34.
Testing
Local, on this machine (all green): host preflights (gofmt, go vet,
golangci-lint, unit tests, build, compactor module),
mkdocs build --strict,shellcheck -xon the guide, and a realdemo_distributedNONINTERACTIVEsmoke — 5 concurrent cold writers per node across both nodes, 10/10 landed,
0 conflicts, receipts ledger showing both nodes issuing/acking.
The
ci/matrix.shintegration cell can't run on Docker Desktop for Mac(the harness reaches Lakekeeper/SeaweedFS by container IP, which the Mac host
can't route to), so integration is left to GitHub Actions here. A separate
follow-up will make
run-ci-local.shwork on macOS (publish LK/SW host ports +curl
localhost).Notes
against the branch.