Migrate kolibri-server into platforms/debian-server - #14998
Conversation
…able Nginx configuration from KOLIBRI_HOME @jamalex
User-definable configuration layout
…bot/github_actions/actions/checkout-7 build(deps): Bump actions/checkout from 6 to 7
github-actions
Bot
added
DEV: backend
Build Artifacts
Smoke test screenshot |
rtibbles
left a comment
rtibbles
left a comment
There was a problem hiding this comment.
Why have you opened the PR when more than half of the criteria are incomplete? Finish the work before asking for review.
|
You're right — the PR was opened prematurely with only the code-migration and workspace-integration tasks done. The remaining acceptance criteria are now complete: Publish / release pipeline
Build / validation CI
|
322984a to
e387fe5
Compare
a341ef0 to
3be615f
Compare
rtibbles
left a comment
rtibbles
left a comment
There was a problem hiding this comment.
Some concerns remain. Readme is inaccurate and the raspberry pi build seems to have been setup to no longer allow updates in the future.
| RUN mkdir -p /etc/apt/keyrings | ||
| INSTALL 644 files/learningequality.asc /etc/apt/keyrings/learningequality.asc | ||
| INSTALL 644 files/learningequality-kolibri.sources /etc/apt/sources.list.d/ | ||
| INSTALL 644 files/learningequality-kolibri-server.sources /etc/apt/sources.list.d/ |
There was a problem hiding this comment.
Why has this been removed? We still want it to be updated after installing, the same as the Kolibri base debian file.
There was a problem hiding this comment.
Restored in 473a40e. I had conflated two things: switching the initial install to the PR-built .deb did not require dropping the APT source. The source is back, so the Pi still upgrades kolibri-server from the released repo on later apt updates, exactly as it does for the kolibri deb (whose learningequality-kolibri.sources was always kept). The build-time install of the PR-built .deb is unchanged; only the update path is fixed.
| @@ -1,5 +0,0 @@ | |||
| Types: deb | |||
There was a problem hiding this comment.
Similarly for this file being deleted.
There was a problem hiding this comment.
Restored in 473a40e — recreated platforms/raspberry-pi/files/learningequality-kolibri-server.sources and its INSTALL in base.Pifile, so the released APT source is available for post-install upgrades.
| #. If you have further changes, you can keep editing and invoking ``make dist`` | ||
| #. Finally, commit your changes and open a PR, including your entry in ``debian/changelog`` | ||
|
|
||
| Pre-commit |
There was a problem hiding this comment.
This is out of date and should be removed in favour of the repo prek config.
There was a problem hiding this comment.
Fixed in a7d1903. Removed the standalone Pre-commit section (with its pip install pre-commit / pre-commit install steps) and pointed the dev workflow at the monorepo prek config instead (prek install at the repo root; see AGENTS.md).
| Automated release workflow | ||
| ~~~~~~~~~~~~~~~~~~~~~~~~~~ | ||
|
|
||
| Publishing a GitHub release triggers the ``build_debian.yml`` workflow, which: |
There was a problem hiding this comment.
This is out of date. We should delete and correct information in the README.
There was a problem hiding this comment.
Fixed in a7d1903. Rewrote the release section to match the migrated pipeline: it no longer claims a GitHub-release-triggered build_debian.yml, but describes the workflow_dispatch-only platform-debian-server-release.yml — publishable-version gate (refuses dev/local builds), changelog generated from the Kolibri version rather than validated against a release tag, always-on manual approval, no Pages job. Also corrected the stale gh workflow run build_debian.yml / "Build Debian source package" trigger instructions.
rtibbles
left a comment
rtibbles
left a comment
There was a problem hiding this comment.
This is generally good to go - let's cleanup the commit history that has been added (leaving the historic commit history from the original repository completely intact), there's a lot of flip flopping encoded in the commit history that we don't need.
1b5b3ea to
b769f22
Compare
|
Cleaned up the migration commit history. The 19 flip-flopping commits I had added are now 8 logical commits, one per migration task:
The flip-flopping is gone — the Pi APT-source remove-then-restore, the changelog The imported repository history and the merge commit ( |
rtibbles
left a comment
rtibbles
left a comment
There was a problem hiding this comment.
Sorry, one more thing - let's clean up the matrix and just do a single job while it's needed.
Commit history still feels a bit untidy, so let's clean it up again once this has been done.
| KOLIBRI_HOME: .kolibri | ||
| run: uv run python -O -m pytest --reuse-db | ||
| platform_tests: | ||
| name: ${{ matrix.name }} tests |
There was a problem hiding this comment.
This is not getting expanded properly when the job matrix gets skipped.
There was a problem hiding this comment.
Fixed in 22eeaae. The job is now a plain single job (debian_server_tests) with a static name: kolibri-server tests, so the check name no longer depends on ${{ matrix.name }} and resolves correctly even when the job is skipped by the pre_job/stage1 guard.
| needs: [pre_job, stage1_required_checks] | ||
| if: ${{ needs.pre_job.outputs.should_skip != 'true' && needs.stage1_required_checks.result == 'success' }} | ||
| runs-on: ubuntu-latest | ||
| strategy: |
There was a problem hiding this comment.
I am not sure this warrants a matrix for a single entry. Let's simplify it to a single job until its warranted.
There was a problem hiding this comment.
Done in 22eeaae. Collapsed the single-entry matrix into a plain job (debian_server_tests), inlining the one row: Python 3.10 pinned directly and distro-info installed unconditionally. stage2_required_checks.needs and the docs/howtos/python_monorepo.md guidance were updated to match — a future second platforms/ member can reintroduce a matrix when one is warranted.
Register the migrated kolibri-server tree as a virtual uv workspace member: add kolibri as a workspace-resolved dependency, floor requires-python to the root range, and pick it up in `uv sync --all-packages` and ruff. Conform the imported sources to the monorepo ruff config (dropping the member's own ruff block) and remove the dormant, unused vcrpy scaffolding the suite never exercised. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the independent 0.5.x scheme with a `debian/changelog` generated from the Kolibri version at build time. Add `scripts/generate_changelog.py` (with a PEP440->Debian upstream-version conversion so pre-releases sort before the final) and wire it into the Makefile's `changelog` target, reading the version from `KOLIBRI_VERSION` in the environment and falling back to `import kolibri`. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The filter-repo import carried in the standalone repo's own pre-commit config, dependabot config, and pre-commit/python-test CI workflows. All are superseded by the monorepo's repo-wide equivalents, and nested `.github/` is ignored by GitHub Actions, so remove them. Also correct the now out-of-date release and pre-commit sections of the server README to point at the monorepo prek config and the workflow_dispatch release pipeline. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Lift the standalone `release.yml` to `.github/workflows/platform-debian-server-release.yml`: trigger on `workflow_dispatch` only (dropping the release-published auto-trigger), remove the GitHub Pages PPA jobs, keep the full Launchpad DAG (build -> dput -> wait -> copy-to-series -> manual approval -> promote), gate on a publishable (non-dev/local) Kolibri version, and target the changelog at the runner's series directly instead of sed-patching it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a reusable `platform-debian-server-build_deb.yml` (`workflow_call`) and a `server_deb` sibling of the `deb` job in `pr_build_kolibri.yml`, so every PR builds the server .deb from `platforms/debian-server/` versioned to that PR's Kolibri (parsed from the whl job's tar artifact). The build runs `make deb` so the nginx error/loading pages are generated, and excludes the tests directory from the source tarball. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Stage the freshly-built server .deb into the Pi image and install it from file (mirroring how kolibri's own deb is installed), passing it through the PR build DAG and the reusable Pi build workflow. The released kolibri-server APT source is kept in place so the Pi still upgrades kolibri-server from the repo on later apt updates. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Replace the standalone install-only `installtest.yml` matrix with a non-blocking `server_smoke_test` job that installs both PR-built debs, starts kolibri-server, and asserts Kolibri is served through nginx/uwsgi (treating any 5xx as not-yet-ready while polling). The served port is sourced once from nginx.conf. It stays out of the blocking `smoke_tests` gate, matching the Pi smoke test precedent. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add a `platform_tests` matrix job in tox.yml that runs the member's `pytest tests/` on its pinned Python with the `distro-info` system dep, and list it in `stage2_required_checks.needs` so it blocks merge. Note the new `platforms/*` workspace glob and the shared matrix in the python-monorepo how-to. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bb182a9 to
22eeaae
Compare
10 participants
Summary
kolibri-server(nginx + uwsgi front end for Kolibri) has lived in a standalone repo, versioned independently at0.5.xand built against a released Kolibri. This imports its full history underplatforms/debian-server/as a uv workspace member so it resolveskolibrifrom the workspace, and binds its.debversion to the Kolibri version via a generateddebian/changelog(0.5.x→0.19.x, monotonic so apt upgrades cleanly). It is the first publishingplatforms/member under the monorepo consolidation, following the Pi image's co-location pattern.The code migration, workspace/version wiring, and the full CI/release-pipeline integration have all landed.
References
Part of #14964 (tracked under #13720). Source repository: https://github.com/learningequality/kolibri-server. Follows the
platforms/pattern from #14967.Changes:
.debversion to Kolibri via a generated changelog.debinpr_build_kolibri.ymlagainst the PR's Kolibri.debinto the Pi imageinstalltest.yml)pytest tests/as a Stage-2 platform test matrix rowReviewer guidance
pyproject.toml[tool.uv.workspace]— theplatforms/*glob carries anexcludeforraspberry-pi(nopyproject.toml); confirmuv sync --group dev --all-packagesresolves cleanly and doesn't narrow the lock's Python floor via the member'srequires-python.platforms/debian-server/scripts/generate_changelog.pydebian_upstream_version— verify PEP440 pre-release/dev versions get a~prefix sodpkg --compare-versionssorts them before the final (the upstreamsedthis replaces was a no-op); finals must munge to identity.platforms/debian-server/pyproject.toml— the member is virtual (package = false, no build-system); confirm it is never treated as a wheel target and thatkolibri = { workspace = true }is the only kolibri source.AI usage
Used Claude Code to execute the pre-approved migration plan task-by-task (subtree import via
git filter-repo, workspace wiring, changelog generator) under test-driven development. Verified with the member'spytest tests/,uv sync --all-packages,ruff, anddpkg-parsechangelog/dpkg --compare-versionsin an Ubuntu container.@rtibblesbot's comments are generated by an LLM, and should be evaluated accordingly
How was this generated?
🟡 Waiting for feedback
Last updated: 2026-07-15 04:17 UTC