Dev productivity improvements#1289
Merged
Merged
Conversation
Dev productivity rollup: - Portless dev hosts: pnpm console:dev / ee-api:dev now serve at https://console.jitsu.localhost / https://ee.jitsu.localhost. New pnpm ui:dev runs both via turbo. Auto-detects the current branch and appends -<branch> for non-default branches; --no-branch opts out. - dev-scripts workspace package with `pnpm dev <subcommand>` dispatcher. Bare `pnpm dev` runs `turbo run dev` as before. - `pnpm dev copy-db --src URL --dst URL`: full-DB pg_dump | psql clone. Drops+recreates dst (prompts; --clean-dst skips). Skips CREATE EXTENSION so dst doesn't need plv8/vector/etc. Strips top-level SET statements so newer pg_dump can feed older dst. Structure-only by default for large log/audit tables; --all-tables overrides. - AUTH_COOKIE_DOMAIN env var for NextAuth: opt-in. Unset = unchanged (host-only cookie). Set to .jitsu.localhost in dev to share session across console/ee/branch hosts. - Next.js 16.2.3 → 16.2.6. allowedDevOrigins=*.jitsu.localhost so next dev doesn't block portless hosts. turbopack.root pinned to repo root to avoid worktree picking up the main checkout's lockfile. - Drop unused release / release:canary / release:build-scripts root scripts and the monorel + semantic-release* devDeps. Drop obsolete sync-bulker.sh.
777b21f to
b6aff45
Compare
vklimontovich
added a commit
that referenced
this pull request
May 8, 2026
Mirrors the fix from #1289 so the OpenAPI dev work isn't blocked when running console:dev from a git worktree. Turbopack panicked because Next was walking up past the worktree and picking the main checkout's pnpm-workspace.yaml as the workspace root.
Single mechanism replaces five scattered .env loaders (dotenv-cli wrappers, tsx --env-file-if-exists flags, process.loadEnvFile, hand-rolled parser). Layered hierarchy (later wins; existing process.env always wins; missing files skipped): 1. ~/.jitsu/.env.local (shared across worktrees) 2. <repo>/.env.local (per-worktree) Implementation: root .npmrc sets node-options=--require=...preload-env.cjs, so pnpm exports NODE_OPTIONS for every script-spawned node process. The preload walks up from cwd to find pnpm-workspace.yaml and load the repo .env.local. (Node disallows --env-file* in NODE_OPTIONS for security; --require is allowed.) No wrapper means `node --inspect` and the debugger attach to the script's own process directly — no dotenv-cli parent in the tree. Drops dotenv-cli + dotenv direct deps from root, console, ee-api, rotor, dev-scripts. CONTRIBUTING.md gains an "Environment variables" section.
Decouples the .env.local preload from dev-scripts (which is for tools like copy-db). New top-level workspace package @jitsu-internal/env-preload owns the preload module exclusively. .npmrc references it by package name, so resolution from any sub-workspace still walks node_modules up to the root. No behavior change.
Two fixes from PR review: 1. makeSetStripper used to drop every top-level SET, including row_security and check_function_bodies that real restores need. Narrow to a denylist (just transaction_timeout for now — the PG 17→<17 incompatibility we hit). Other SETs pass through. 2. runCopyDb now refuses to start when --src and --dst resolve to the same (host, port, database). Without this, a typo plus --clean-dst would drop the only copy before pg_dump runs.
There was a problem hiding this comment.
Found one correctness regression in the new env preload path.
- currently prevents repo-level from overriding because it checks the live while loading each layer. That reverses the intended precedence documented in the PR (shared first, then repo override).
Please adjust the loader so only pre-existing environment variables are protected, while later file layers can override earlier file layers.
There was a problem hiding this comment.
Found one correctness regression in the new env preload path.
- env-preload/preload-env.cjs currently prevents repo-level .env.local from overriding ~/.jitsu/.env.local because it checks the live process.env while loading each layer. That reverses the intended precedence documented in the PR (shared first, then repo override).
Please adjust the loader so only pre-existing environment variables are protected, while later file layers can override earlier file layers.
The old guard `if (process.env[key] !== undefined) continue` skipped any key that already existed in process.env at the time of the check. After ~/.jitsu/.env.local loaded, every shared key was in process.env, so the repo-level <repo>/.env.local could never override it — backwards from the documented "later wins" precedence. Snapshot the shell's original env keys at startup; only those are untouchable. Keys set by a previous file load are overwritable, so the order is now: shell > <repo>/.env.local > ~/.jitsu/.env.local Verified end-to-end: repo overrides home for shared keys; home-only keys still load; shell still wins over both.
Dismissing stale bot review: duplicate of the previously-resolved host-aliasing concern (#discussion_r3209668650). Author chose to keep the simple host/port/database string compare for this dev-only tool — see thread reply.
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pnpm console:dev/pnpm ee-api:devnow serve athttps://console.jitsu.localhost/https://ee.jitsu.localhostvia portless. Newpnpm ui:devruns both in parallel through turbo.git rev-parse origin/HEAD), the host becomeshttps://console-<branch>.jitsu.localhost/https://ee-<branch>.jitsu.localhost. Branch is sanitized for DNS (lowercased,[^a-z0-9-]→-, capped at 30 chars). Pass--no-branchto opt out.dev-scriptsworkspace package with apnpm dev <subcommand>dispatcher. Barepnpm devrunsturbo run devas before.pnpm dev copy-db --src URL --dst URL— full-database clone viapg_dump | psql.$ENV_VARplaceholders in URLs are resolved againstprocess.env(populated by the layered.env.localloading — see below). Drops and recreates the dst database (prompts first; pass--clean-dstto skip the prompt). Skips allCREATE EXTENSIONstatements so dst doesn't need plv8/vector/etc. installed. Strips top-levelSETstatements from the dump so newer pg_dump versions can feed older dst servers. By default a few large log/audit tables (betteruptime.events,newjitsu.StatusChange,newjitsu.AuditLog,newjitsu.source_task) are structure-only — pass--all-tablesto include their rows..env.localloading, no wrapper. Single mechanism replaces five scattered loaders (dotenv-cli,tsx --env-file-if-exists,process.loadEnvFile, hand-rolled parser, plus duplicates).node --inspectand your debugger attach to the script's own process — nodotenv-cliparent in the tree. See "Env resolution" below.AUTH_COOKIE_DOMAINenv var for NextAuth. Set to.jitsu.localhostin dev to share the session acrossconsole.jitsu.localhost/ee.jitsu.localhostand any branch hosts. Unset in production — when unset, behavior is unchanged (cookie stays host-only, noDomainattribute).allowedDevOrigins: ["*.jitsu.localhost"]in both Next configs sonext devdoesn't block cross-origin_next/*requests from the portless hosts.turbopack.rootpinned to the repo root in both Next configs. Without it, Next walks up looking for a lockfile and on a worktree picks up the main checkout'spnpm-workspace.yaml, which makes Turbopack's module graph panic.release/release:canary/release:build-scripts(thecli/build-scriptspackage they referenced doesn't exist) and themonorel+semantic-release*devDeps that backed them. Also drops the obsoletesync-bulker.sh.Why dash style for branch subdomains
*.jitsu.localhost(single-level wildcard) covers every host in dash style, so a single Firebase Authorized-Domains entry is all that's ever needed. Dot style (console.<branch>.jitsu.localhost) would require either per-branch console clicks or a multi-level wildcard.Env resolution
Two
.env.locallayers, auto-loaded for every node process spawned by a pnpm script (later wins; existingprocess.envalways wins; missing files skipped silently):~/.jitsu/.env.local— shared across all worktrees of all branches (Firebase, Stripe, OIDC, GitHub OAuth — anything that doesn't change per branch).<repo>/.env.local— per-worktree (DATABASE_URL,NEXTAUTH_URL,AUTH_COOKIE_DOMAIN, anything branch-specific).How: root
.npmrcsetsnode-options=--require=env-preload. pnpm exports that asNODE_OPTIONSfor every script-spawned node process. The preload (env-preload/preload-env.cjs, ~50 LOC of plain CommonJS) loads the shared file, then walks up fromprocess.cwd()to findpnpm-workspace.yamland load the repo.env.local. (Why a preload, not--env-file-if-exists: Node disallows--env-file*inNODE_OPTIONSfor security;--requireis allowed.)What this drops:
dotenv-clifrom root + console + rotor;dotenvdirect dep from ee-api + dev-scripts; theloadRepoEnvblock inrun-app.ts; the hand-rolled parser indev-scripts/src/utils/env.ts; thedotenv-clispawn indev-scripts/src/index.ts;--env-file-if-existsflags in rotor + console scripts.See CONTRIBUTING.md → Environment variables for the full story.
Setup
On a feature branch, the suffix is auto-applied: e.g. on
dev-productivity-improvementsyou gethttps://console-dev-productivity-improv.jitsu.localhost.In Firebase Console → Authentication → Settings → Authorized domains, add
*.jitsu.localhostonce.For shared secrets across worktrees:
In
<repo>/.env.localfor shared session across console/ee/branch hosts:Test plan
pnpm installcleanpnpm dev helpshows subcommandspnpm console:devopens athttps://console.jitsu.localhostpnpm console:devopens athttps://console-<branch>.jitsu.localhostpnpm console:dev --no-branchopens athttps://console.jitsu.localhostregardless of branch*.jitsu.localhostauthorizedAUTH_COOKIE_DOMAIN=.jitsu.localhost: log in once on console, ee-api recognizes the sessionpnpm dev copy-db --src '$DATABASE_URL' --dst 'postgres://...local-dev'prompts to drop dst, then clonespnpm dev copy-db --src ... --dst ... --clean-dstskips the promptpnpm dev copy-db --src ... --dst ... --all-tablesincludes data for the otherwise structure-only tablespnpm exec node -e 'console.log(process.env.DATABASE_URL ? "SET" : "UNSET")'printsSETif<repo>/.env.localdefines it; same from inside any workspace dirpnpm console:devboots; attach Node debugger; confirm nodotenv-cliparent in the process tree andprocess.envshows the layered values