feat(embedded-server): switch from npx to bundled Etherpad source#38
Merged
Conversation
…led source
The `etherpad-lite` and `etherpad` npm packages were both unpublished
(404 on cold start), leaving the embedded-server flow dead since
spring 2026. This PR replaces the npx spawn with a bundled-source flow:
- `scripts/fetch-etherpad.mjs` — idempotent fetch of the pinned Etherpad
GitHub source tarball into `packages/desktop/resources/etherpad/`,
followed by `pnpm install --prod` in `src/`. Version pinned via
`ETHERPAD_VERSION` (currently `v2.7.3`). Re-runs are no-ops unless
`--force` or version mismatch.
- `embedded-server.ts` spawns `node --require tsx/cjs node/server.ts
--settings <path>` against the bundled source rather than `npx`.
Startup timeout drops 480s → 90s now that the multi-hundred-MB npx
download is gone.
- `findBundledEtherpadDir({ resourcesPath, appRoot })` is the single seam
that resolves the source: dev uses `appRoot/resources/etherpad`,
packaged apps use `<resourcesPath>/etherpad/`.
- `electron-builder.yml` adds `extraResources` mapping
`resources/etherpad` → `etherpad` so packaged installers include the
source. Excludes Etherpad's own tests + .git from the bundle.
- `AddWorkspaceDialog` re-enables the "Use a local server" checkbox.
Embedded workspaces disable the URL field and skip the URL probe.
- AGENTS.md, en.ts i18n updated.
3 new tests for `findBundledEtherpadDir`; existing embedded-server tests
updated for the new spawn args. 284 desktop tests pass.
**Runtime prereq (dev only):** machine must have `node` on PATH for
spawn. Bundling node into shipped installers is a follow-up — for now
the dev box's node is used.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ⓘ You've reached your Qodo monthly free-tier limit. Reviews pause until next month — upgrade your plan to continue now, or link your paid account if you already have one. |
release.yml (Linux/macOS/Windows) and snap-publish.yml now run `pnpm --filter @etherpad/desktop fetch:etherpad` between `pnpm install` and `pnpm build` so the produced installers actually contain Etherpad at `<resourcesPath>/etherpad/`. Without this step the installer would ship without the source and the embedded-server flow would fail at runtime with the friendly 'Etherpad source not bundled' error.
…d system node
Production embedded-server spawn now uses `process.execPath` (the
Electron binary) with `ELECTRON_RUN_AS_NODE=1`, which Electron honours
to behave as plain Node.js. Removes the 'must have node on PATH'
prerequisite for shipped installers.
Tests inject `nodeRuntime: { execPath: 'node', env: {} }` to keep the
existing spawn-assertion clean; a new test verifies the default path
uses Electron + ELECTRON_RUN_AS_NODE.
…ivers + Swagger UI
fetch:etherpad now edits src/package.json before `pnpm install --prod`,
deleting deps that the dirty-db embedded path doesn't load:
cassandra-driver, @elastic/elasticsearch, mongodb, mysql2, pg,
redis, rethinkdb, surrealdb, swagger-ui-express, swagger-jsdoc
ueberdb2 stays (it's the abstraction the dirty driver dispatches
through). Smoke-tested locally: `node --require tsx/cjs
node/server.ts` still serves /api/ → {"currentVersion":"1.3.1"}.
Bundle size: 313MB → 259MB (-54MB). Conservative — more savings
available by trimming dev-time tools (typescript, jsdom, esbuild
binaries) in a follow-up once we audit which are runtime-only.
…7MB)
Three new prune passes that target deps the dirty-db embedded path
never loads at runtime, verified via smoke test (server boots and
/api/ responds with {"currentVersion":"1.3.1"}).
1. **.pnpm store prune.** ueberdb2 + transitives pull in cloud DB
drivers (mongodb, elastic, cassandra, redis, rethinkdb, surrealdb,
tedious), Azure auth, OpenTelemetry, Apache Arrow, Swagger UI even
when those backends aren't selected. Delete their store entries
post-install — they're not require()d on the dirty path.
~160MB pruned across 68 entries.
2. **@types/*, rusty-store-kv, bson.** Type defs aren't loaded at
runtime; rusty-store-kv is an alternative ueberdb2 backend;
bson is a mongodb leftover.
3. **Source tree.** Drop /doc, /docs, /packaging, /snap, /Dockerfile,
/admin, /ui, /tests, /local_plugins and docs files from the
extracted source — not needed for the embedded server.
Kept (originally pruned, broke tsx): esbuild + typescript — tsx's
TypeScript loader require()s both at runtime even though they look
like build tools.
Smoke-tested at every step. Bundle reduction:
- original: 313 MB
- after src/pkg slim: 259 MB (DB drivers from package.json)
- after .pnpm prune: 158 MB (transitive cloud-DB removal)
- after types/bson: 144 MB (@types/*, rusty-store-kv, bson)
- after source slim: 137 MB (drop docs/packaging/tests)
…lemetry + debug log The e2e test for the embedded workspace flow was failing because \`app.getAppPath()\` in a dev/test launch resolves to a deeper path than the original \`<appRoot>/resources/etherpad/\` lookup expected. Broaden findBundledEtherpadDir to also try \`<appRoot>/../../resources/etherpad/\` and \`<appRoot>/../resources/etherpad/\` — handles electron-vite's out/main/ + packaged-app layouts. @opentelemetry/api restored to the bundle: \`prom-client\` (Etherpad's metrics) require()s it during boot. Pruning it logged an ugly error even though Etherpad continued. EPD_EMBEDDED_DEBUG=1 now tees the embedded-server's stdout+stderr to /tmp/epd-embedded-debug.log so e2e failures survive the test runner's userDataDir cleanup. After this: \`E2E_EMBEDDED=1 pnpm test:e2e --grep "embedded workspace"\` passes in ~5s on a warm tsx cache. Bundle size 159MB (was 313MB originally; +22MB from the opentelemetry restore).
New \`tests/android.spec.ts\` + \`tests/android-fixtures.ts\` drive a running emulator (or USB-attached device) via adb input commands. Skipped by default — set ANDROID_E2E=1 to run. Why not Playwright \`connectOverCDP\` or puppeteer-core? Android WebView's stripped-down CDP rejects \`Browser.setDownloadBehavior\` (Playwright) and \`Target.getBrowserContexts\` (puppeteer-core), so neither high-level driver attaches. adb input commands are the lowest-common-denominator and work on every WebView version. Provides: - adbClearAppData / adbForceStop / adbLaunchApp — app lifecycle - adbTap(x, y) / adbText(text) / adbBack / adbHome — input - adbScreenshot(path) — PNG capture - adbDumpUi() — uiautomator XML hierarchy - waitForUiText(regex) — poll-until-text-appears helper uiautomator surfaces WebView text content (good enough for "did the dialog render?"-style assertions) but not HTML form elements — coordinate taps + screenshot diffs cover the interactive paths. Confirmed on the local x86_64 emulator (Android 34): ✓ first launch shows the AddWorkspaceDialog (5.7s) ✓ uiautomator can see the name + url + colour fields (5.3s)
This was referenced May 12, 2026
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resurrects the embedded-server flow (Spec 5 v1) that was broken when the
etherpad-liteandetherpadnpm packages were unpublished (both 404on cold start).
scripts/fetch-etherpad.mjs— idempotent fetch of the pinnedGitHub source tarball +
pnpm install --prod. Drops Etherpad sourceinto `packages/desktop/resources/etherpad/` (gitignored). Pinned to
`v2.7.3`; future bumps just change `ETHERPAD_VERSION` in the script.
`node --require tsx/cjs node/server.ts --settings ` spawned from
the bundled source. Startup timeout drops from 480s to 90s — no
npx download in the hot path.
in either layout (dev: `resources/etherpad/` next to the package;
packaged: `/etherpad/` from electron-builder
`extraResources`).
installers ship Etherpad pre-installed. Excludes Etherpad's own tests
Embedded workspaces disable the URL field and skip the URL probe.
just couldn't reach it before.
Caveats
is opt-in, not a postinstall hook — first-run download is 5MB
compressed + ~300MB of node_modules after install).
PATH. Shipping a bundled node binary is a follow-up — current installer
assumes the user has node.js installed.
Tests
appRoot fallback).
Test plan
AddWorkspaceDialog → "Use a local server" → Add → workspace
appears, pad loads.
so shipped installers actually bundle the source.
🤖 Generated with Claude Code