feat(server)!: jspm.io direct vendor (Rails-style no-build)#89
Open
vivek7405 wants to merge 37 commits into
Open
feat(server)!: jspm.io direct vendor (Rails-style no-build)#89vivek7405 wants to merge 37 commits into
vivek7405 wants to merge 37 commits into
Conversation
…s and false positives
The vendor scanner picked up server-only imports from contexts that
never reach the browser, generating spurious vendor pipeline work
on packages that can't be browser-bundled.
Four tightenings:
1. route.{ts,js,mjs,mts} and middleware.{ts,js,mjs,mts}: server-
only by file-router convention. New isServerOnlyFile() helper
joins these to the existing .server.{ts,js,mjs,mts} suffix
check. Imports of @prisma/client, ws, etc. in these files no
longer enter the vendor pipeline.
2. test/ and tests/ directories: tests are server-only by webjs
convention. Their imports of test frameworks and DB clients
shouldn't generate browser vendor entries.
3. `import type X from 'pkg'` statements: TypeScript type-only
imports are erased at compile time, never reach the runtime.
The IMPORT_RE has a (?!type\s) negative lookahead. Catches
real-world false positive in api/chat/route.ts which imports
'ws' as type-only.
4. Imports inside /* … */ block comments and // line comments:
JSDoc examples (cn.ts in @webjsdev/ui, etc.) frequently show
'import x from clsx' in a code block. The scanner now strips
comments before pattern-matching.
Four new tests:
scanBareImports: skips route.ts and middleware.ts
scanBareImports: skips test/ and tests/ directories
scanBareImports: skips import type statements
scanBareImports: skips import strings inside comments
These bug fixes were salvaged from the closed PR #87
(feat/no-build-vendor-drops-esbuild). Re-applied directly on top
of main's esbuild-on-demand vendor.js instead of cherry-picking,
because the closed branch had drifted too far for a clean apply.
1155 tests pass.
Source-level companion to the existing erasable-typescript-only
tsconfig-flag rule. Scans every .ts / .mts file under the app for
the four constructs the framework's type-stripper rejects at
request time:
- enum declarations (any of enum / const enum / declare enum,
with uppercase first letter to avoid matching variables
literally named 'enum')
- namespace blocks containing value statements (let/const/var/
function/class). Type-only namespaces, which ARE erasable,
are intentionally allowed
- constructor parameter properties (public/private/protected/
readonly modifier directly before a parameter name)
- import = require (TypeScript-style CommonJS import)
Each violation reports file, line number, the construct name, and
a concrete fix. Skips node_modules, dist, build, .next, .git, and
any folder starting with underscore (the framework's _private
convention).
Why both rules ship enabled by default:
erasable-typescript-only catches the tsconfig case (flag missing
or off). It's the early-warning path; if the flag is set, the
TypeScript compiler flags violations in your editor before they
reach the runtime.
no-non-erasable-typescript catches the source case (offending
syntax that slipped past tsconfig, or files written before the
flag was added, or third-party packages that publish raw .ts).
It's the late-warning path; runs at commit time via
webjs check.
Together, an app gets two independent defenses for the same class
of violations. Either alone is incomplete: the tsconfig flag does
nothing if the user disables it; a source scan alone doesn't help
during editing.
Six new tests: one positive case per construct (asserts the rule
flags it), one negative case (clean erasable .ts file passes),
and one scope test (node_modules and _private folders are
correctly skipped).
All 1164 tests pass.
…Rails-style no-build)
Replaces main's Vite-style optimizeDeps esbuild-on-demand vendor
pipeline with the Rails 7 + importmap-rails posture: bare-specifier
npm imports resolve via importmap to jspm.io CDN URLs and the
browser fetches the bundle directly from jspm.io. The webjs server
does not proxy, cache, or bundle vendor packages.
This is the strictest "no build" architecture for a no-build
framework. Nothing bundles on the user's machine, ever. esbuild
leaves @webjsdev/server's dependencies entirely.
vendor.js rewrite (259 -> 310 lines, but most of that is doc):
- scanBareImports (kept): walks user source for bare imports
- extractPackageName, isServerOnlyFile, stripComments (kept):
same precise scanning rules
- resolvePackageDir (new): walks require.resolve's path back to
the package root, handles npm workspace hoisting
- getPackageVersion (new): reads node_modules/<pkg>/package.json
version field
- jspmGenerate (new): POSTs an install list to api.jspm.io/generate
with provider=jspm.io, env=[browser,production,module], returns
the resolved importmap fragment. In-memory cached by sorted
install-list key. 10s timeout with AbortController. Logs (does
not throw) on API failure so server boot still succeeds; vendor-
importing pages get "unresolved bare specifier" errors in the
browser until api.jspm.io is reachable.
- vendorImportMapEntries (rewritten, now async): scans bare imports,
resolves versions from node_modules, calls jspmGenerate, returns
the importmap fragment
- clearVendorCache (kept): drops the jspmCache so file-watcher
rebuilds re-resolve URLs
REMOVED entirely:
- bundlePackage (was the esbuild bundler call)
- serveVendorBundle (was the /__webjs/vendor/* response handler)
- vendorCache (was the in-memory esbuild-output cache)
- VENDOR_CACHE_MAX constant
- import { build } from 'esbuild' (no longer needed at runtime)
dev.js:
- removed /__webjs/vendor/* URL handler (browser bypasses webjs
server entirely for vendor URLs; goes straight to jspm.io)
- vendorImportMapEntries call sites now await the async result
- import list cleaned up
index.js exports:
- removed bundlePackage, serveVendorBundle
- added getPackageVersion, jspmGenerate
Tests:
- dropped tests for removed APIs (bundlePackage, serveVendorBundle)
- new tests for getPackageVersion (resolution + null fallback)
- new tests for jspmGenerate (empty input, real call, cache hit,
order-independent cache key); network-gated via
WEBJS_SKIP_NETWORK_TESTS env
- vendorImportMapEntries tests use the async signature
- dev-handler /__webjs/vendor/* test replaced with one asserting
the path 404s (no local handler)
- 1160/1160 pass
Why jspm.io over esm.sh:
- Years of uptime track record; esm.sh has had documented
downtimes and maintenance windows
- Institutional backing: 37signals (Silver), CacheFly (CDN
infrastructure sponsor), Socket, Framer (Bronze). Rails ecosystem
dependency creates downstream pressure for continued operation
- status.jspm.io for incident transparency
- Standards-first maintenance by Guy Bedford (TC39 contributor
on ESM, import maps, HTML spec)
- Matches Rails 7 + importmap-rails default exactly
Why the JSPM Generator API rather than naive URL construction:
- jspm.io's bare-package URL (https://ga.jspm.io/npm:dayjs@1.11.13)
returns text/plain metadata, not JavaScript. Browser execution
would fail with SyntaxError
- The correct entry path (e.g., /dayjs.min.js) varies per package
and must be resolved by the Generator
- Same call importmap-rails makes at pin time; webjs makes it at
server boot
What 'no build' means in this PR:
- User runs no build command
- User writes no build config
- User's source IS the deploy artifact (.ts files served via
Node's stripTypeScriptTypes)
- No bundler invocation on user's machine, ever
- No esbuild in framework deps
Breaking changes:
- @webjsdev/server.bundlePackage removed
- @webjsdev/server.serveVendorBundle removed
- /__webjs/vendor/* URL paths no longer handled by the server
- Apps now require api.jspm.io reachability at server boot to
populate the vendor importmap
Pairs with the scanner improvements (commit fcf2692) and the
no-non-erasable-typescript lint rule (commit 83e77a9) already
cherry-picked onto this branch. Together they constitute the
Rails-aligned no-build vendor architecture.
…stence
Adds `webjs vendor pin`, `webjs vendor unpin`, `webjs vendor list`
plus the runtime layer that reads the committed pin file in
preference to a live api.jspm.io call. Matches Rails' importmap-rails
pin workflow exactly, including --download for offline bundle
vendoring.
Layered on top of the existing jspmGenerate / vendorImportMapEntries
machinery; doesn't replace it. The runtime preference order is:
1. Read .webjs/vendor/importmap.json (committed pin file)
2. Fall back to live api.jspm.io/generate (if no pin file)
So apps that never run pin still work (boot-time API call, in-memory
result, same as before). Apps that run pin commit a small JSON config
and shed the boot-time dep on api.jspm.io.
Two modes mirror Rails:
Default: importmap.json holds resolved jspm.io CDN URLs. Browser
fetches direct from ga.jspm.io. Only importmap.json is
committed (a few KB).
--download: also downloads each bundle from jspm.io to
.webjs/vendor/<pkg>@<version>.js. importmap.json holds
local /__webjs/vendor/ paths. Server handler serves the
bundles from disk. Both importmap.json and bundle files
are committed (offline-capable production, CSP-friendly,
audit-friendly).
Auto-prune handles three orphan scenarios uniformly:
Update dayjs@1.x.js bundle removed when version bumps to 2.x
Delete bundle + importmap entry removed when import is dropped
Mode swap default <-> --download cleans up the other mode's files
Pin is idempotent with respect to the current source tree. Run twice
in a row with no source change = no-op. Switch modes = clean directory.
Three CLI commands matching Rails' pattern:
webjs vendor pin [--download] auto-discovers, resolves, writes
webjs vendor unpin <pkg> removes one entry + bundle if any
webjs vendor list shows pinned packages with sizes
Skipped Rails commands and the rationale:
pristine subsumed by 'pin --download' (always overwrites)
json 'cat .webjs/vendor/importmap.json' already works
outdated 'npm outdated' covers installed versions
update 'npm install pkg@latest && webjs vendor pin' is the flow
audit 'npm audit' covers most cases; vulnerability-data
integration is its own project
Pin is intentionally manual (no predev/prestart auto-run). Auto-pin
would cause silent churn in the committed importmap.json file as
jspm.io re-resolves entries or transitive deps drift. Rails takes
the same posture: bin/importmap pin is always developer-invoked.
dev.js: vendor URL handler restored for --download mode (serves
files from .webjs/vendor/). In default mode the handler still 404s
but the browser never requests these URLs (importmap routes direct
to jspm.io). resolveVendorImports replaces vendorImportMapEntries
at the call site, layering pin-file preference over the live API.
vendor.js new exports: pinAll, unpinPackage, listPinned,
readPinFile, resolveVendorImports, serveDownloadedBundle. Existing
exports (jspmGenerate, vendorImportMapEntries, etc.) unchanged.
index.js re-exports all of the above from the server's main entry
so the CLI can use them.
Tests for the new pin layer come in the next commit (this one is
the implementation; tests stay green via the existing API surface
which is unchanged for callers that don't touch the new functions).
1155/1155 tests pass on this commit.
…st, serve, prune) 12 new tests, network-gated where they hit api.jspm.io: - pinAll default: writes importmap.json with jspm.io URLs - pinAll --download: writes importmap.json + bundle files locally - pinAll prune: removes orphan bundle files from prior pins - pinAll mode switch (--download to default): removes leftover bundles - unpinPackage: removes entry from importmap.json - unpinPackage: returns removed:false for non-existent package - listPinned: parses jspm.io URLs and extracts versions - listPinned: returns empty array when no pin file - resolveVendorImports: prefers pin file over live API call - serveDownloadedBundle: rejects path-traversal filenames (../, /, .., non-js) - serveDownloadedBundle: serves real file from .webjs/vendor/ - serveDownloadedBundle: missing file returns 404 makeTempAppWithSource() helper creates an isolated tmp app dir with symlinked node_modules so getPackageVersion / pinAll's createRequire chain finds installed packages. 1160 to 1172 tests pass.
…commands README.md item 'DX' previously claimed esbuild bundled vendor packages (Vite-style optimizeDeps). After the PR #89 architectural change, vendor packages resolve through importmap to jspm.io URLs at runtime; webjs's server doesn't bundle them. Updated to describe: - jspm.io as the vendor resolution mechanism - webjs vendor pin / unpin / list commands - --download mode for offline-capable production - .webjs/vendor/importmap.json as the committed config artifact AGENTS.md invariant 10 (TypeScript must be erasable) now mentions both lint rules: - erasable-typescript-only (existing): checks the tsconfig flag - no-non-erasable-typescript (new in this PR): scans source for the four offending patterns even if the flag is unset esbuild's TS-strip fallback documentation stays accurate; only the vendor-pipeline esbuild claim was wrong post-jspm.io.
Companion to the previous commit. README.md's 'DX' bullet previously claimed esbuild bundled vendor packages (Vite-style optimizeDeps). After PR #89's architectural change, vendor packages resolve through importmap to jspm.io URLs at runtime; webjs's server doesn't bundle them. Updated to describe: - jspm.io as the vendor resolution mechanism - webjs vendor pin / unpin / list commands - --download mode for offline-capable production - .webjs/vendor/importmap.json as the committed config artifact - new no-non-erasable-typescript lint rule esbuild's TypeScript-strip fallback documentation stays accurate; only the vendor-pipeline esbuild claim was wrong.
packages/server/README.md and AGENTS.md previously described vendor as 'Vite-style optimizeDeps backed by esbuild'. After PR #89, vendor resolves through jspm.io at runtime; the server doesn't bundle. Updated to: - README.md vendor bullet: jspm.io resolution, pin commands, --download mode, .webjs/vendor/importmap.json - AGENTS.md vendor.js row: jspm.io flow + pin file preference + --download local-bundle serving - AGENTS.md check.js row: mention no-non-erasable-typescript alongside no-json-data-files
…eployment pages docs/app/docs/no-build/page.ts had the most outdated content: the entire 'Bare specifiers' + 'Why auto-bundle' sections were built around the old Vite-style optimizeDeps esbuild pipeline. Replaced with the jspm.io direct architecture description plus a new section on `webjs vendor pin` (default + --download modes). Also updated: - Importmap example (URLs now show jspm.io shape with @Version) - Cache-invalidation section (versioned URLs explanation) - Dev vs prod table (vendor resolution row) - deployment/page.ts vendor URL paragraph (URL pattern + jspm.io cache headers + --download bundle headers) Kept the no-build philosophy framing intact; just updated the mechanism (jspm.io CDN-direct instead of local esbuild-on-demand). Rails alignment is more explicit now, since the new architecture matches importmap-rails posture exactly. Pages still pending update next commit: docs/app/docs/typescript/page.ts (only TS-fallback mention; accurate) docs/app/docs/getting-started/page.ts (TS-fallback mention; accurate)
…io architecture ETags and Cache Headers section described `/__webjs/vendor/<pkg>.js` URLs with hash-based immutable caching. After PR #89, vendor URLs are either jspm.io URLs directly (default mode) or local `/__webjs/vendor/<pkg>@<version>.js` paths (after `webjs vendor pin --download`). Updated the paragraph to describe both modes and their cache header sources.
Adds 7 new end-to-end tests that spawn the actual webjs CLI binary against a temp app directory and exercise the full pipeline: - list with no pin file → reports 'No pin file' - pin → writes .webjs/vendor/importmap.json with jspm.io URLs - list with pin file → shows pinned packages + URLs - unpin <pkg> → removes entry from importmap.json - unpin <not-pinned> → reports 'not in pin file' - pin --download → writes bundle files alongside importmap.json - unknown subcommand → exits 1 with usage message These complement the existing per-function unit tests in packages/server/test/vendor/vendor.test.js by verifying the CLI surface: argument parsing, stdout shape, exit codes, --download flag handling. Network-gated where they hit api.jspm.io (4 of 7 tests). Skip via WEBJS_SKIP_NETWORK_TESTS=1 in air-gapped CI. Test file lives at test/vendor-cli/vendor-cli.test.mjs (new directory; consistent with the test/<feature> layout convention already used by test/serialization/, test/scaffolds/, etc.).
Header comment still described the old 'Vite-style optimizeDeps'
mental model. Updated to describe the actual flow:
- vendor entries come from resolveVendorImports
- reads committed .webjs/vendor/importmap.json if present
- else calls api.jspm.io/generate once at boot
- browser fetches direct from jspm.io (default) or local
/__webjs/vendor/ paths (after webjs vendor pin --download)
Cosmetic doc cleanup; behavior unchanged.
vivek7405
force-pushed
the
feat/jspm-direct-vendor
branch
from
3757ebb to
245ebe7
Compare
…peline scanBareImports now preserves the full specifier instead of dropping to the root package name. vendorImportMapEntries + pinAll splice the version into the specifier (pkg@version/subpath) before calling jspm.io's Generator API, which resolves each subpath via the package's exports field. For --download mode, bundleFilenameWithSubpath encodes the filesystem-safe filename: 'dayjs', '1.11.13', '/plugin/utc' becomes 'dayjs@1.11.13__plugin__utc.js'. The __ separator stays reversible. End-to-end verified against api.jspm.io: scanner finds the subpath, generator resolves it correctly, importmap emits the right entry. Tests: 1 new scanBareImports test for subpath preservation; 1180 tests pass total. Limitation: jspm.io errors for subpaths the package's exports field doesnt declare. Most well-maintained packages declare their subpaths; legacy packages may not. Same behavior as before for those cases (missing importmap entry, browser surfaces error).
…path version parsing Two real bugs found during deep PR review: 1. Scaffold .gitignore was ignoring .webjs/ entirely, including .webjs/vendor/. Users running 'webjs vendor pin' would write the importmap.json + downloaded bundles into a gitignored directory. git add wouldn't pick them up. Production deploys would never see the pin file. The entire 'webjs vendor pin' feature silently defeated for every scaffolded app. Fix: keep .webjs/ ignored (still right for any future tooling caches), but add !.webjs/vendor/ to un-ignore the vendor subdirectory. Mirrors Rails treating config/importmap.rb + vendor/javascript/ as committed. 2. listPinned's version parser for local --download URLs was treating the __subpath segment as part of the version. For '/__webjs/vendor/dayjs@1.11.13__plugin__utc.js', it returned version '1.11.13__plugin__utc' instead of '1.11.13'. Cosmetic bug in 'webjs vendor list' output for subpath imports. Fix: after slicing off the .js suffix, split on '__' to separate version from subpath. Version is everything before the first __, not the whole tail. New test 'listPinned: parses subpath URLs and extracts versions (not subpath as version)' plants a subpath entry + bundle file and asserts the parsed version is '1.11.13' not '1.11.13__plugin__utc'. 1181 tests pass (was 1180).
Same bug as the scaffold .gitignore fix: webjs vendor pin writes
.webjs/vendor/importmap.json, which would be silently ignored. Fix
applied to:
- .gitignore (repo root): affects webjs's own monorepo apps
(website, docs, ui-website) which inherit from this file
- examples/blog/.gitignore: the reference example app
scaffold template fix was in the previous commit. Three .gitignore
files now share the same pattern: .webjs/ ignored, .webjs/vendor/
un-ignored.
…avior
Previous test asserted the /__webjs/vendor/* path is 'unhandled (no
local vendor proxy)'. That was outdated: in --download mode the
server DOES handle that URL via serveDownloadedBundle, returning a
real file or 404 if missing. The old test was passing for the wrong
reason (handler returned 404 because no .webjs/vendor/ file existed).
Updated to three tests that accurately cover:
1. 404 when no bundle file on disk (with hint to run vendor pin
--download in the error body)
2. 200 + correct content-type when a real bundle is present
3. Path-traversal rejection (400 or 404, both safe)
1180 -> 1183 tests pass.
…h production images Companion to the earlier .gitignore fixes. The repo's .dockerignore excluded all .webjs/ directories from the Docker build context. Even if the user committed .webjs/vendor/importmap.json (after the .gitignore fixes), the Dockerfile's COPY statements would exclude it because Docker uses .dockerignore, not .gitignore. Result: production images would not contain the pin file. Server would fall back to live api.jspm.io calls on every cold start, defeating the deterministic-deploy property the pin file provides. Fix: keep **/.webjs ignored generally, but un-ignore **/.webjs/vendor and its contents. Same pattern as the .gitignore changes. Trio of related fixes in this audit: - packages/cli/templates/.gitignore (scaffold) - .gitignore + examples/blog/.gitignore (monorepo apps) - .dockerignore (Docker context)
…resolved value
Two concurrent rebuilds during dev (chokidar firing twice quickly,
or two simultaneous server startups in tests) would each hit the
check-then-set race: both see jspmCache.has() return false, both
issue an HTTP request to api.jspm.io, both call cache.set with
their own result. Wasteful, and the second-to-complete write
clobbers the first (deterministic but redundant).
Standard Promise-cache pattern fixes it: store the Promise
immediately, before awaiting the fetch. Concurrent callers with
the same install list share the in-flight request and resolve
together.
Also: on failure, drop the cache entry so retries can succeed.
Without this, a transient api.jspm.io error would poison the cache
with {} forever (or until the process restarted).
1183 tests still pass.
Strict CSP with script-src 'self' blocks the jspm.io script tag, so vendor imports fail to load. This wasn't documented anywhere. Added a 'Content Security Policy (CSP) and vendor packages' section to the deployment doc with the two mitigations: 1. Allow jspm.io in CSP: add https://ga.jspm.io to script-src 2. Switch to --download mode: bundles served from same origin, 'self' alone sufficient Suitable scenarios per mode: jspm.io default for typical apps, --download for compliance / air-gapped / strict-CSP environments.
…S.md
Scaffolded apps had no documentation about webjs vendor pin. AI
agents working in those projects wouldn't know about it. They'd
either: (a) call api.jspm.io on every server boot indefinitely, or
(b) forget about the vendor pipeline entirely and write fetch()
calls for npm packages.
Added a focused section after the Database section:
- Standard npm install workflow
- webjs vendor pin for production (writes committed pin file)
- webjs vendor pin --download for offline/CSP-strict scenarios
- webjs vendor list / unpin commands
- Why pin is intentionally NOT in predev/prestart (would cause
silent churn in committed importmap.json)
Cross-references docs.webjs.com Deployment > CSP section for the
strict-CSP discussion.
Previous 'fix' was wrong. Verified with git check-ignore. Pattern was: .webjs/ !.webjs/vendor/ This silently does NOT work. Per the gitignore man page: 'It is not possible to re-include a file if a parent directory of that file is excluded.' The .webjs/ rule excludes the directory itself; git won't even traverse into it to evaluate the !.webjs/vendor/ exception. git check-ignore reports .webjs/vendor/importmap.json as ignored, despite the apparent exception. Correct pattern: .webjs/* !.webjs/vendor/ !.webjs/vendor/** The .webjs/* glob excludes contents of .webjs/ but not the directory itself. The two un-ignore rules cover both the vendor subdirectory entry AND its recursive contents (including nested files like .webjs/vendor/some-pkg/inner.js if --download bundles ever land in subdirectories). Verified with git check-ignore on a temp repo: .gitignore:14:!.webjs/vendor/** .webjs/vendor/importmap.json .gitignore:12:.webjs/* .webjs/cache/ts.bin .gitignore:14:!.webjs/vendor/** .webjs/vendor/sub/nested.js Vendor files are correctly un-ignored; future cache files stay ignored. Same pattern applied to .dockerignore for the same reason. Docker uses gitignore-like syntax with the same parent-exclusion gotcha. Affected files: packages/cli/templates/.gitignore (scaffold) .gitignore (repo root) examples/blog/.gitignore .dockerignore Embarrassing miss in the previous audit pass: I 'fixed' the gitignore but didn't verify with git check-ignore. The user caught it.
Verifies via `git check-ignore` that .webjs/vendor/importmap.json is not accidentally ignored. The common mistake is simplifying the three-line exception pattern (`.webjs/*` + `!.webjs/vendor/` + `!.webjs/vendor/**`) back to `.webjs/`, which silently breaks `webjs vendor pin`: gitignore semantics excludes the parent first, after which no child negation can re-include anything. Skipped when the directory is not a git repo or has no .gitignore. Strengthens inline comments in both .gitignore files to call out the hazard and point at this rule.
Adds inline warnings to examples/blog/.gitignore and .dockerignore matching the scaffold template, plus a paragraph in the scaffold's AGENTS.md vendor section explaining why the three-line pattern is structurally load-bearing and pointing at the lint rule that catches regressions.
Tooling lives in dot-prefixed directories (.opencode/, .claude/,
.github/, .husky/, .vscode/) and root-level config files
(web-test-runner.config.js, vitest.config.ts, tailwind.config.mjs). It
imports packages the browser will never load (test runners, AI tool
plugins) that legitimately cannot resolve via jspm.io. With these
specifiers in the install batch, api.jspm.io/generate returns 401 and
the entire importmap silently empties, breaking legitimate user deps
like dayjs.
Skip ALL dot-prefixed directories during the walk and any file matching
*.config.{js,ts,mjs,mts,cjs,cts} at any depth. Adds two new tests
covering the new exclusion behavior.
api.jspm.io/generate returns 401 with an error body when ANY package in the install batch fails to resolve (e.g. a transitive subpath that isn't exported). The previous batched call collapsed the entire importmap on a single failure, silently dropping legitimate user deps like dayjs and breaking pages in the browser with bare-specifier errors. Split jspmGenerate into per-install calls running in parallel via Promise.all. Cache keys are now individual install specs, so concurrent rebuilds with overlapping deps still share work. Failure logs name the offending package and surface jspm.io's error reason. Regression test plants a known-bad install alongside a known-good one and asserts the good one still resolves.
The framework already extracts the CSP nonce from incoming Content-Security-Policy headers and applies it to other inline scripts (env shim, boot, suspense), but importMapTag was bare. Strict-CSP apps using script-src 'nonce-...' policies silently lost the entire vendor pipeline: browser blocked the unsigned importmap tag, every bare-specifier import failed. importMapTag now takes a nonce option and emits nonce="..." when provided. ssr.js threads opts.nonce through alongside the publicEnvShim call. Matches the pattern Turbo's test fixtures use.
Browsers require crossorigin on cross-origin modulepreload, else the preload is ignored or double-fetched (defeating the optimization). Same-origin preloads must NOT carry the attribute for the same reason in reverse. Vendor packages resolved to jspm.io URLs are the new common case after the per-package vendor pipeline lands. Today vendor URLs flow only through the importmap (not preload), so this fix is preventative: if a future change adds vendor URLs to the preload set or a user lists a CDN URL in metadata.preload, the modulepreload now does what it claims. Exports preloadCrossOriginAttr for unit testing; covers cross-origin, same-origin path, and same-origin URL with /__webjs/vendor/ prefix.
`webjs vendor pin` now computes a SHA-384 hash for every resolved
vendor URL and writes it into `.webjs/vendor/importmap.json` under
a new `integrity` key:
{
"imports": { "dayjs": "https://ga.jspm.io/.../dayjs.min.js" },
"integrity": { "https://ga.jspm.io/.../dayjs.min.js": "sha384-..." }
}
Default mode fetches each bundle solely to hash it (bytes not written
to disk). --download mode hashes the bytes it already downloads.
resolveVendorImports returns both maps. setVendorEntries(imports,
integrity) stores them in the importmap module. buildImportMap emits
the integrity field per the browser importmap-integrity spec
(Chrome 132+, Safari 18.4+). Modulepreload tags get
`integrity="sha384-..."` when the URL has a known hash.
Older pin files lacking the integrity field still load (treated as
empty integrity map). Live-API mode skips integrity entirely; users
who want SRI run `webjs vendor pin`.
Updates the resolveVendorImports unit test for the new return shape.
Adds 5 tests covering: - readPinFile returns integrity when present - readPinFile is backwards-compatible (no integrity field on old format) - sha384Integrity returns deterministic sha384-<base64> strings - pinAll default mode writes integrity field with sha384 hashes - pinAll --download mode integrity matches on-disk bytes byte-for-byte Also fixes a latent hazard in makeTempAppWithSource: it symlinks the repo's node_modules into the temp dir, so a `sourceFiles` entry like `node_modules/picocolors/package.json` would clobber the real picocolors package (the entry resolved through the symlink and rewrote the real package.json with a 2-line stub, breaking every test that needed picocolors locally). Helper now refuses paths under `node_modules/`. The previously-corrupted picocolors was reinstalled.
stripTs in packages/server/src/dev.js no longer falls back to esbuild.transform when Node's module.stripTypeScriptTypes rejects non-erasable syntax. webjs is buildless end-to-end. The erasable-typescript-only and no-non-erasable-typescript lint rules already catch enum / namespace / parameter properties / legacy decorators / import-require at edit time. The dev server now catches ERR_UNSUPPORTED_TYPESCRIPT_SYNTAX and returns a clean 500 whose body names the offending file and points at the no-non-erasable-typescript lint rule, instead of bubbling an unstyled error or silently bundling. packages/ui/test/cn-helper.test.js used esbuild.transformSync to strip TS for an in-test import; switched to Node's built-in module.stripTypeScriptTypes (same primitive the dev server uses). Removed esbuild: ^0.28.0 from packages/server/package.json. The loadEsbuild helper, the stripTs fallback branch, and the dev-handler test that asserted the fallback are gone; the dev-handler test now asserts the new clean-500 behavior. Repo-root devDeps elimination (esbuild + @web/dev-server-esbuild) and scaffold doc updates follow in subsequent commits.
web-test-runner.config.js used @web/dev-server-esbuild to strip TS for browser tests. Replaced with a 10-line custom plugin that calls Node 24+'s built-in `module.stripTypeScriptTypes` (the same primitive `webjs dev` uses, no separate toolchain). Plugin shape: implements `resolveMimeType` (returns 'js' for .ts/.mts so the browser accepts the response) and `transform` (strips types from the served body). Only erasable TS is supported; non-erasable syntax throws at strip time, mirroring the dev server's behavior. Removed `esbuild` and `@web/dev-server-esbuild` from repo-root devDependencies. Net: esbuild is gone from the framework entirely (no runtime dep, no dev tool dep, no transitive install). Verified by booting `ui-overlay.test.js` (21 tests, all passed) and `directives-guard.test.js` (3 passed, 1 skipped) under the new plugin. Scaffold + docs prose updates follow in the next commit.
Mirrors the same change in the main web-test-runner.config.js. The blog-e2e config is a separate file because it boots the blog dev server on :3456 first and proxies /__blog/* requests; both configs needed the same plugin substitution.
deployment/page.ts, typescript/page.ts, getting-started/page.ts, and no-build/page.ts described the esbuild fallback that's now gone. Rewrote each passage to state: - Only erasable TypeScript is supported (matches the new dev.js behavior after dropping the fallback). - Non-erasable syntax now fails at strip time with a 500 pointing at the no-non-erasable-typescript lint rule. - "esbuild-with-sourcemap pipeline" comparison line in typescript.ts rephrased to "bundler-with-sourcemap pipeline" since esbuild is no longer the local point of comparison. - "esbuild fallback available" line in deployment.ts replaced with "buildless end-to-end: no bundler or transpiler at deploy time". Scaffold + lint-rule message updates follow in the next commit.
The erasable-typescript-only rule's description (in check.js) and the four scaffold-template files (AGENTS.md, CONVENTIONS.md, .cursorrules, .windsurfrules, .github/copilot-instructions.md) all said "the dev server falls back to esbuild + inline sourcemap" for non-erasable TS. That's no longer true: the fallback is gone in this PR, and the dev server now returns a 500 pointing at the no-non-erasable-typescript lint rule. Updated prose in all listed files to reflect the new behavior plus the buildless-end-to-end framing. Remaining 3 files (templates CONVENTIONS.md + examples/blog CONVENTIONS.md + examples/blog copilot-instructions.md) follow in the next commit.
… test comments Final esbuild-mention cleanup. Mirrors the previous commit's prose update across the 3 remaining files: - packages/cli/templates/CONVENTIONS.md - examples/blog/CONVENTIONS.md - examples/blog/.github/copilot-instructions.md All now say: "the dev server fails at strip time and returns a 500 pointing at the no-non-erasable-typescript lint rule. webjs is buildless end-to-end and has no bundler fallback." Also tidies three stale esbuild-reminiscing comments in packages/server/test/dev/dev-handler.test.js (test name "esbuild- stripped types" -> "types stripped"; "esbuild may rewrite" comment replaced with note about stripTypeScriptTypes preserving source; section header "missing esbuild path" -> "tsResponse cache path"). After this commit, no executable code in the repo references esbuild outside of changelog history.
Final esbuild-cleanup pass for top-level docs (README, root AGENTS, packages/server/AGENTS, agent-docs/typescript, agent-docs/advanced). agent-docs/advanced.md had a stale description of the old vendor pipeline (esbuild-based bundling at /__webjs/vendor/<pkg>.js). Rewrote the section to describe the new jspm.io-direct posture: bare specifiers resolve through api.jspm.io to CDN URLs, the browser fetches directly from ga.jspm.io, `webjs vendor pin` commits resolved URLs + SHA-384 integrity hashes for reproducible deploys, `--download` caches bundles locally for air-gapped / strict-CSP deploys. Mirrors what dev.js + vendor.js actually do now. README, root AGENTS invariant 10, packages/server/AGENTS module map, and agent-docs/typescript all said non-erasable TS "falls back to esbuild". Updated each to say it now returns a 500 pointing at the no-non-erasable-typescript lint rule, since webjs is buildless end-to-end with no bundler fallback. After this commit no executable code or current documentation references esbuild; remaining hits are in changelog/ (historical) and blog/strip-types-not-esbuild.md (the blog post itself, which narrates the journey of why esbuild went away).
Two more spots not caught in the earlier batches: a second README
mention saying "esbuild stays as a per-file fallback" and a second
no-build docs page bullet describing the now-removed fallback as
shipping "inline sourcemaps and roughly 3x wire bytes".
Both rewritten to match the new behavior: dev server returns a 500
naming the file and pointing at the no-non-erasable-typescript lint
rule, since webjs is buildless end-to-end with no bundler fallback.
Remaining executable-code esbuild mentions are now either
intentionally-negative ("No esbuild" in web-test-runner.config.js)
or describe-the-new-behavior ("there is no longer an esbuild" in
dev-handler.test.js). Historical changelog/ + the strip-types blog
post stay as-is.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replaces main's Vite-style
optimizeDepsesbuild-on-demand vendor pipeline with the Rails 7 + importmap-rails posture exactly: bare-specifier npm imports resolve via importmap to jspm.io CDN URLs and the browser fetches the bundle directly from jspm.io. The webjs server does not proxy, cache, or bundle vendor packages.This is the strictest "no build" architecture for a no-build framework. Nothing bundles on the user's machine, ever.
How it works (Rails-aligned)
At server boot:
scanBareImportswalks user source for bare-specifier imports (import dayjs from 'dayjs')getPackageVersionreads each package's installed version fromnode_modules/<pkg>/package.jsonjspmGeneratePOSTs the resolved install list (['dayjs@1.11.13', '@hotwired/turbo@8.0.0', ...]) tohttps://api.jspm.io/generatewithprovider=jspm.io,env=['browser', 'production', 'module']<script type="importmap">tagAt runtime, the importmap looks like:
Browser fetches the bundle directly from
ga.jspm.io. The webjs server never sees the request. Same as Rails today.Why jspm.io (not esm.sh)
What this PR is NOT
.webjs/vendor/directory, nothing committed to source control)webjs vendor pin/unpin/list/warmCLI (no cache to manage)What this PR fixes vs main
/__webjs/vendor/<pkg>.jsURL never changed)route.tswsfrom type-only imports(?!type\s)lookaheadclsx/tailwind-mergefrom JSDoc commentsstripComments()pre-passCommits (3)
fcf2692Scanner tightening (route.ts/middleware.ts exclusion, test/, import type, comments). Cherry-picked from PR feat(server)!: version-in-URL for vendor + scanner tightening + lint rule + optional warm #88.83e77a9no-non-erasable-typescriptlint rule. Cherry-picked from PR feat(server)!: version-in-URL for vendor + scanner tightening + lint rule + optional warm #88.88e36cbThe main rewrite: vendor.js + dev.js + index.js + tests. ReplacesbundlePackage/serveVendorBundle/in-memory bundle cache withjspmGenerate/vendorImportMapEntries. Removes the/__webjs/vendor/*URL handler. Test suite updated; 1160/1160 pass.Breaking changes
@webjsdev/serverno longer exportsbundlePackageorserveVendorBundle(these functions are deleted; their work is done by jspm.io now)/__webjs/vendor/*URL paths no longer handled by the server (browser fetches direct fromga.jspm.io)vendorImportMapEntriesis now async (takesappDirparameter and callsjspmGenerateinternally)api.jspm.ioreachability at server boot to populate the vendor importmap. If unreachable, the server still boots and serves user routes; only vendor-importing pages report "unresolved bare specifier" errors in the browser until the API is reachable again.What "no build" means under this PR
Strict no-build for user-facing aspects:
Caveat: esbuild still in
@webjsdev/serverdeps as TS-stripping fallback for non-erasable syntax (rare, with the newno-non-erasable-typescriptlint rule catching most cases at commit time). Removing esbuild entirely is a separate decision tracked as a follow-up.Relationship to other PRs
PR #88 and this PR fix the same main-branch bugs. They differ on the bundler-locus axis. The user should choose one to merge and close the other.
Test plan
npm test: 1160/1160 passgetPackageVersionverified against installed picocolors + null fallbackjspmGenerateverified against real api.jspm.io for picocolors (network-gated)vendorImportMapEntriesintegration verified end-to-endFollow-ups
@webjsdev/serverdependencies entirely? Currently kept for TS-stripping fallback. Withno-non-erasable-typescriptlint enabled, the fallback rarely fires. Removing it would shed ~56 packages (esbuild + 52 platform binaries + wrappers). Decision deferred; the lint rule needs proven-in-the-wild reliability first.--download). Would eliminate the runtime dep onapi.jspm.iofor known package sets. Not in scope here.