Skip to content

Test Sandbox, resumable LocalKit sync, and installer hardening - #77

Merged
jhd3197 merged 8 commits into
mainfrom
dev
Jul 21, 2026
Merged

Test Sandbox, resumable LocalKit sync, and installer hardening#77
jhd3197 merged 8 commits into
mainfrom
dev

Conversation

@jhd3197

@jhd3197 jhd3197 commented Jul 21, 2026

Copy link
Copy Markdown
Owner

The best way to find out your installer breaks on a bare Debian image is to build a tool that runs it on one — and then watch it break. That's the through-line of this PR. A new Test Sandbox drives ServerKit's installer and script suites across a Linux distro matrix inside Docker, straight from the panel, and it immediately earned its keep: several of the install.sh fixes here are annotated "found via Test Sandbox full mode" — a too-new glibc in the release venv, a python -m venv --help that lies about ensurepip, npm going missing on the source-build fallback. Alongside it, LocalKit's sync grows a chunked, resumable v2 so a transfer that dies at 99% costs one 8 MiB chunk instead of the whole thing, plus the download half of code sync that its "import a remote site locally" flow was missing. Rounding it out: the Marketplace registry flips to its long-planned serverkit.ai default with real logo art, and a one-line-but-fatal deploy-job crash gets fixed.

Highlights

  • Test Sandbox — a new admin page (under System, next to Jobs) that runs ServerKit's installer and script test suites across eight Linux distros in Docker. Pick a quick mode (shell lint + unit suites, ~1–2 min/distro) or a full mode that runs the real install.sh end-to-end in a systemd container and health-probes the result. Live per-distro status, run history, per-distro logs, and cancel.
  • LocalKit sync is now resumable. Transfers move in 8 MiB chunks, so a network drop resumes from the last confirmed chunk instead of restarting, and payloads are no longer capped by the panel's 100 MB request limit. Pulls support HTTP Range, so interrupted downloads resume too.
  • LocalKit can pull code now, not just push it — the download half of the "import this remote site as a new local site" round trip. /pair also advertises a capability list so the desktop app disables buttons an older panel can't serve, instead of failing mid-operation.
  • Marketplace defaults to the serverkit.ai registry and ships real extension logos, so cards show art even before the first successful fetch and even on air-gapped panels.
  • Deploy jobs no longer crash on their first step — a missing property meant every deploy failed with an AttributeError before any work ran.
  • The installer survives minimal and cleaned-down images (missing curl, empty package index, a too-new glibc in the prebuilt venv, missing npm on the source fallback) and now has ARM64 coverage in CI.
Technical changes

Test Sandbox (new full-stack feature)

  • SandboxRun model + migration 075_sandbox_runs (idempotent — guards on live schema since MigrationService runs create_all() before Alembic). results is a per-distro JSON map; full logs live on disk under instance/sandbox-runs/<id>/<distro>.log.
  • TestSandboxService runs a distro matrix in Docker with two modes: quick (a sh runner that bash -n-lints every shipped script and runs the source-able unit suites, mirroring the scripts-ci cross-distro job) and full (a privileged geerlingguy systemd container running real install.sh + a /api/v1/system/health probe). Per-distro work runs on its own thread; containers are named sk-sandbox-<run_id>-<distro> so cancel/stale-sweep can find them; a module-level lock guards the active-container map. Full mode waits for systemctl is-system-running and real DNS+HTTP (probed with python3, since curl isn't in those images) before installing, to dodge the post-boot networkd window that caches empty apt lists.
  • test_sandbox.py blueprint at /api/v1/test-sandbox: GET /distros, GET/POST /runs, GET /runs/<id>, POST /runs/<id>/cancel, GET /runs/<id>/logs/<distro> (plain text). Reads are @viewer_required, mutations @admin_required; a single-run lock returns 409 if one's already going.
  • Frontend TestSandbox.jsx page + _test-sandbox.scss, wired into App.jsx routes/titles, sidebarItems.js (System category), main.scss, and a testSandbox.js API module (the log fetch bypasses the JSON-parsing request() for raw text, same pattern as ai.js).
  • smoke_sandbox.py (manual one-distro smoke) and _dbcheck.py (sqlite integrity helper) added for local verification.

LocalKit sync v2 (chunked + resumable)builtin-extensions/serverkit-localkit/backend/localkit.py, bumped to 0.2.0

  • POST /push/<kind>/init describes a transfer, reaps stale ones, and returns already-confirmed offsets (that array is the resume state). PUT /push/<kind>/chunk verifies a per-chunk sha256, is idempotent on re-send, and writes its marker after the bytes so a crash costs one chunk. POST /push/<kind>/finish verifies the whole-file sha256 before processing — missing chunks 409 with the received set, a hash mismatch discards, so processing only ever runs on verified bytes and an abandoned transfer can't half-apply.
  • v1 push/code and push/db now delegate to _install_code/_import_db, so v1 and v2 share one processing path and the mandatory safe-extract policy.
  • Pulls accept ?session= and serve with conditional=True (Range / If-Range). The session pins one materialized export so ranges never assemble across two different mysqldump/tar runs; no session = the unchanged v1 export-and-delete.
  • A transfer is a temp directory with per-chunk marker files (no shared JSON list, so concurrent chunk writes need no lock); transfer ids are validated against uuid4().hex before touching a path. FEATURES gains sync-v2.

LocalKit pull-code + capabilities

  • GET /pull/code streams a tar.gz of the site's wp-content, built inside the container (docker exec ... tar czf -) to preserve permissions/structure, entries prefixed wp-content/ to match what push/code accepts. GNU tar's exit 1 ("file changed as we read it") is tolerated for live sites.
  • GET /pair returns a features array (append-only capability contract) so clients gate UI on capability. GET /sites gains php_version (regexed off the compose image tag — a file read, not a per-site container shell) and an explicit site_url.

Registry go-liveregistry_service.py, registry_index.json, test_registry.py, docs/EXTENSIONS_REGISTRY.md

  • DEFAULT_REGISTRY_URL flips from the raw-GitHub index to https://serverkit.ai/ext/index.json (proxies the same index with caching + logo URL rewriting); raw GitHub stays documented as the SERVERKIT_REGISTRY_URL manual fallback.
  • Bundled offline index gains logo fields for all extensions so Marketplace cards have art before the first fetch. Tests and docs updated for the new default.

Deploy job crash fixdeployment_job.py

  • DeploymentRunner.run() read self.job.target_server_name, but that value only existed as a to_dict() key, so every deploy crashed with AttributeError on step one. Added a target_server_name property and reused it in to_dict().

Installer / test-harness hardening

  • install.sh: new ensure_bootstrap_tools() (refresh package index + install curl up front) called from main(); py_venv_ok() now also imports ensurepip (Debian's python -m venv --help passes without it and only fails at creation); build_virtualenv() discards a prebuilt release venv whose interpreter won't run (release built on Ubuntu 24.04 hard-requires GLIBC_2.38); sync_source() refreshes the apt index before installing git on empty-index images; a provision_node re-run on the source-build fallback since the release path skips Node.
  • .github/workflows/scripts-ci.yml: new cross-arch-arm64 job on ubuntu-24.04-arm running the unit suites natively (Raspberry Pi / ARM homelab proxy — the VM harness can't virtualize ARM on Windows Hyper-V).
  • scripts/test/full-stack-test.ps1: -Only now selects from the full DistroMap (opt-in alma9/debian11/opensuse15 in addition to fedora) in the order listed; tar pinned to Windows' System32\tar.exe so a Git-Bash PATH doesn't shadow it with GNU tar (which reads C:\... as a remote host).
  • scripts/test/vm-install.sh: overlay now rsync -a --delete (mirror exactly — a stale serverkit-tramo plugin from an older release base was surviving a plain overlay and breaking vite build), protecting .env/instance/ssl/venv/logs; bootstraps Node 22 when the release install skipped it.
  • scripts/test/README.md: distro coverage table split into default vs opt-in, plus ARM-CI notes.

jhd3197 and others added 8 commits July 20, 2026 14:39
The download half of code sync — the missing piece for LocalKit's
"import this remote site as a new local site" flow (LocalKit plan 18).

- GET /pull/code streams a tar.gz of the site's wp-content, built inside
  the container with `docker exec tar czf -` so permissions and structure
  survive. Entries are prefixed `wp-content/`, the same format POST
  /push/code accepts, so both directions share one archive shape. GNU
  tar's exit 1 ("file changed as we read it") is tolerated — a live site
  writing uploads mid-archive is normal.
- GET /pair now reports a `features` array so clients gate their UI on
  capability instead of discovering a 404 mid-operation.
- GET /sites gains `php_version` (regexed off the compose image tag, not
  a per-site container shell) and an explicit `site_url`.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sync v1 POSTs the whole payload in one request, so it is capped by the
panel's 100MB MAX_CONTENT_LENGTH and a drop at 99% means starting over.
v2 makes every request one 8 MiB chunk, which lifts the cap and makes a
retry cost only what was actually lost.

- POST /push/<kind>/init: describes the transfer, reaps stale ones, and
  returns the offsets already confirmed for an identical payload — that
  array IS resume.
- PUT /push/<kind>/chunk: per-chunk sha256, idempotent re-sends, marker
  written after the bytes so a crash costs one chunk, not a corrupt file.
- POST /push/<kind>/finish: verifies the whole-file sha256, THEN runs the
  processing. Missing chunks are a 409 with the received set (resumable);
  a hash mismatch discards. Because processing only ever runs on verified
  bytes, an abandoned transfer cannot leave half-applied state.
- push/code and push/db v1 now delegate to _install_code/_import_db, so
  v1 and v2 share exactly one processing path and the safe-extract policy
  stays mandatory in both.
- Pulls accept ?session= and serve with conditional=True, giving Range /
  If-Range. The session pins one materialized export: pull/db and
  pull/code build their payload per request, and ranges taken from two
  different mysqldump/tar runs would assemble into neither. No session =
  v1 export-and-delete, unchanged.

A transfer is a directory under the temp dir with per-chunk marker files
rather than a shared JSON list, so concurrent chunk writes need no lock.
Transfer ids are validated against uuid4().hex before touching a path.

FEATURES gains 'sync-v2'; older clients keep the v1 routes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Introduces a full-stack Test Sandbox feature to run distro matrix tests in Docker from the UI: new backend API/service, sandbox_runs model + migration, run/log endpoints, and a new frontend page with mode selection, run history, live polling, cancel, and per-distro logs.

Also adds ARM64 scripts CI coverage (ubuntu-24.04-arm) and strengthens install/test automation for minimal images and overlay rebuilds: early curl/bootstrap index refresh, stricter venv validation, release-venv glibc fallback, Node reprovisioning on source fallback, rsync --delete overlay behavior, npm bootstrap in VM install, Windows tar pinning in PowerShell harness, plus updated test docs and distro coverage notes.
DeploymentRunner.run() logged self.job.target_server_name, but the model
only exposed that value as a to_dict() key, so every deploy job crashed
with AttributeError at its first step and was marked failed before any
work ran. Add a target_server_name property and reuse it in to_dict().
Flip DEFAULT_REGISTRY_URL to https://serverkit.ai/ext/index.json (the
long-planned go-live default: cached proxy of the raw index with logo URL
rewriting). Add logo fields to the bundled offline index so Marketplace
cards get real art even before the first successful fetch, and update the
registry tests + docs for the new default.
Copilot AI review requested due to automatic review settings July 21, 2026 03:55

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an admin-facing “Test Sandbox” feature to run installer/script suites across a Docker distro matrix from the panel, while also hardening installer/test harness paths, making LocalKit sync resumable (chunked v2 + pull-code), and switching the Marketplace default registry to serverkit.ai.

Changes:

  • Introduces Test Sandbox backend model/service/API plus a new React page (with SCSS + API client) to run and inspect per-distro Docker-based test runs.
  • Implements LocalKit sync v2 (chunked/resumable uploads + resumable downloads) and adds pull-code + capability advertisement.
  • Hardens install/test workflows (curl/bootstrap tooling, glibc/venv sanity check, Node provisioning) and updates registry default + logos + CI coverage.

Reviewed changes

Copilot reviewed 28 out of 28 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
VERSION Bumps panel version.
scripts/test/vm-install.sh Makes overlay mirroring stricter and bootstraps Node for release-overlay rebuilds.
scripts/test/README.md Updates distro coverage docs and defaults vs opt-in targets.
scripts/test/full-stack-test.ps1 Expands opt-in distro map and pins Windows tar behavior.
install.sh Adds bootstrap tooling, strengthens venv/ensurepip checks, and improves source-fallback robustness.
frontend/src/styles/pages/_test-sandbox.scss Adds Test Sandbox page styling (infra console look).
frontend/src/styles/main.scss Imports the new Test Sandbox page stylesheet.
frontend/src/services/api/testSandbox.js Adds frontend API helpers for Test Sandbox + raw log fetch.
frontend/src/services/api/index.js Registers Test Sandbox API module into ApiService.
frontend/src/pages/TestSandbox.jsx Adds the Test Sandbox admin page (runs, polling, logs, history).
frontend/src/components/sidebarItems.js Adds “Test Sandbox” nav item under System.
frontend/src/App.jsx Wires Test Sandbox route and page title.
docs/EXTENSIONS_REGISTRY.md Updates registry docs to serverkit.ai default behavior.
builtin-extensions/serverkit-localkit/plugin.json Bumps LocalKit extension version to 0.2.0.
builtin-extensions/serverkit-localkit/backend/localkit.py Implements resumable sync v2 + pull-code + capabilities and site metadata enrichment.
backend/tests/test_registry.py Updates tests for new default registry URL + logo normalization expectations.
backend/smoke_sandbox.py Adds a manual smoke runner for the Test Sandbox service.
backend/migrations/versions/075_sandbox_runs.py Adds migration for sandbox run history storage.
backend/app/services/test_sandbox_service.py Implements Docker-backed distro matrix execution + logs + cancellation.
backend/app/services/registry_service.py Switches DEFAULT_REGISTRY_URL to https://serverkit.ai/ext/index.json.
backend/app/models/sandbox_run.py Adds SandboxRun model for persisted run history/results.
backend/app/models/deployment_job.py Fixes deploy crash by adding/reusing target_server_name property.
backend/app/models/init.py Exposes SandboxRun in models package import surface.
backend/app/data/registry_index.json Adds bundled logo fields for Marketplace offline/default art.
backend/app/api/test_sandbox.py Adds REST endpoints for distros/runs/logs (viewer/admin gated).
backend/app/init.py Registers the Test Sandbox blueprint.
backend/_dbcheck.py Adds a local SQLite integrity helper script.
.github/workflows/scripts-ci.yml Adds native ARM64 job for script suite coverage.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +25 to +27
def list_runs():
limit = min(int(request.args.get('limit', 20)), 100)
return jsonify({'runs': TestSandboxService.list_runs(limit=limit)})
Comment on lines +190 to +194
with _active_lock:
names = list(_active_containers.get(run_id, []))
for name in names:
subprocess.run(['docker', 'rm', '-f', name],
capture_output=True, timeout=30)
Comment on lines +278 to +280
finally:
subprocess.run(['docker', 'rm', '-f', container],
capture_output=True, timeout=30)
Comment on lines +285 to +290
cmd = [
'docker', 'run', '--rm', '--name', container,
'-v', f'{REPO_ROOT}:/src',
'-v', f'{work_dir}:/work',
image, 'sh', '/work/run-quick.sh',
]
Comment on lines +320 to +327
from app.models import WordPressSite
for entry in payload.get('sites') or []:
site = WordPressSite.query.get(entry.get('id')) if entry.get('id') else None
if site is None or not site.application:
continue
entry['php_version'] = _php_version(site)
entry['site_url'] = entry.get('url') or _site_url(site)
return jsonify(payload), 200
@jhd3197
jhd3197 merged commit f1c5ff1 into main Jul 21, 2026
1 check passed
jhd3197 added a commit that referenced this pull request Aug 6, 2026
These five workflows triggered on both `push: [dev, main]` and
`pull_request: [dev, main]`, so one change was validated up to three times: on
the dev push, again on the dev->main PR, and a third time on the post-merge
push to main.

Narrow them to `push: [dev]` + `pull_request: [main]`:

- `pull_request: [dev]` was dead config. Every PR this repo has had targets
  main (checked back through #77, dependabot's included), so it never produced
  a run.
- `push: [main]` was redundant for release-smoke: release.yml's build-release
  job runs that same scripts/build-release.sh for real, on the same commit,
  moments later.

Also drops test-system-utils' `unit-tests` job. It ran
`pytest tests/test_utils_system.py` (44 tests) that Backend CI's bare `pytest`
already collects — there is no pytest.ini, addopts or collect_ignore narrowing
collection. The distro matrix and the raw-subprocess audit stay; those are the
parts Backend CI genuinely cannot do.

Coverage of main is unchanged: a pull_request run tests the merge result, and
release.yml still gates itself on the full backend suite.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants