Skip to content

Drop the polyfill for native workspace APIs - #33

Merged
TomChv merged 3 commits into
mainfrom
drop-polyfill-native-workspace
Aug 21, 2026
Merged

Drop the polyfill for native workspace APIs#33
TomChv merged 3 commits into
mainfrom
drop-polyfill-native-workspace

Conversation

@TomChv

@TomChv TomChv commented Aug 20, 2026

Copy link
Copy Markdown
Member

Reworks #16.

The TypeScript SDK depended on dagger/polyfill for three things: selecting the modules it manages, forking the workspace to stage edits, and re-rooting the resulting changeset into the client's cwd. All three are now native in the engine (dagger/dagger#13854, #13855), so the dependency goes.

Module selection. currentModule.asSDK(workspace:).modules returns this SDK's registered modules, workspace-root-relative, already narrowed by the engine to the client's cwd — everything at or below it, plus the nearest enclosing module when the cwd is not itself registered. That is exactly what the polyfill's findConfigDirs walk intersected with the workspace's managed-module list produced, so the walk, the intersection, and the cwd-to-root path mapping all go.

Changesets. #13855 set changesetCwdCutover = "v1.0.0-beta.10": past that module engine version the engine measures a Workspace.changes result from the workspace cwd itself, and fails loudly on a change that falls outside it. Declaring the version hands the re-rooting the polyfill's fork used to do back to the engine, and every staging site collapses to the shape #13855 was designed for — keep the baseline you already hold, derive, diff:

ws.withNewDirectory("/" + modPath, templateSource).changes(ws)

Paths are anchored at / because they are workspace-root-relative: a relative workspace path resolves from ws.cwd and would otherwise be prefixed a second time when a function is called directly from a subdirectory rather than driven by the engine.

Module sources now resolve through Workspace.moduleSource, which is the call the polyfill's core field was making. And ModuleSource.generate(workspace:) stages the module's local dependency closure itself, then applies only that module's own generated context to the workspace it was handed — so the explicit generateLocalDependencies staging around each codegen goes with it, in both Mod.generate and generateAllModule.

generateAll* keeps its concurrency, and the merge stays a changeset merge:

let changes = modules(ws)
  .filter { mod => mod.skipGenerate(ws) == false }
  .map { mod => ws.moduleSource("/" + mod.rootPath).generate(ws).changes(ws) }
changes.{{isEmpty}}
ws.changes(ws).withChangesets(changes)

Worth recording why, since the near-miss is easy to walk into: these per-module changesets are cwd-measured now, so they must not be folded by re-applying them to a workspace — Workspace.withChanges overlays at the workspace root, which is one frame off for every cwd but the root, and the engine's nested per-dependency generation runs at the dependency's cwd. Changeset.withChangesets merges the patches directly and never re-enters workspace coordinates, so operands and result are both cwd-measured — the frame the engine applies a returned generator changeset in. The empty fold seed no longer needs the polyfill's fork: ws.changes(ws) compares a workspace against itself, so it stays empty even when that nested workspace already carries a staged closure.

dagger.lock is what the beta.10 CLI writes: the v2 format, the container digests it resolved, and sdk-sdk floated to 00bb067, which picks up dagger/sdk-sdk#20 — the harness now runs its contract checks on the configured CLI release. Before it, a CLI older than beta.10 silently skipped every function taking a Workspace! argument, which is initModule and every generator here. It also brings a monorepo check group that did not run at all on the old pin.

Test

dagger --x-release=v1.0.0-beta.10 check

57/57. That includes sdk-sdk:generation:respects-cwd and sdk-sdk:chain:respects-cwd, which cover generating from a subdirectory — the case e2e:generate:generate-all-scope-check documents it cannot run itself and used to defer to the polyfill's own suite for.

Requires an engine at v1.0.0-beta.10 or newer, which the module now declares.

The TypeScript SDK depended on `dagger/polyfill` for three things: selecting
the modules it manages, forking the workspace to stage edits, and re-rooting
the resulting changeset into the client's cwd. All three are now native in the
engine (dagger/dagger#13854, dagger/dagger#13855), so the dependency goes.

Module selection. `currentModule.asSDK(workspace:).modules` returns this SDK's
registered modules, workspace-root-relative, already narrowed by the engine to
the client's cwd — everything at or below it, plus the nearest enclosing module
when the cwd is not itself registered. That is exactly what the polyfill's
`findConfigDirs` walk intersected with the workspace's managed-module list
produced, so the walk, the intersection, and `moduleRelPath` all go.

Changesets. `#13855` set `changesetCwdCutover = "v1.0.0-beta.10"`: past that
module engine version the engine measures a `Workspace.changes` result from the
workspace cwd itself, and fails loudly on a change that falls outside it.
Declaring the version hands the re-rooting the polyfill's fork used to do back
to the engine, and every staging site collapses to the shape #13855 was designed
for — keep the baseline you already hold, derive, diff:

    ws.withNewDirectory("/" + modPath, templateSource).changes(ws)

Module sources. `Workspace.moduleSource` resolves a path the same way the
polyfill's nested client did — the polyfill's `core` was that call. And
`ModuleSource.generate(workspace:)` stages the module's local dependency
closure itself, then applies only that module's own generated context to the
workspace it was handed, so the explicit `generateLocalDependencies` staging
around each codegen goes with it.

generateAllModule folds by workspace rather than by changeset: each module
generates onto the workspace the previous one produced and one diff at the end
carries them all. Folding per-module changesets is no longer available — past
the cutover each would be measured from the cwd, which is not the frame
`Workspace.withChanges` applies them in. The cost is that per-module codegen no
longer runs concurrently.

dagger.lock is what the beta.10 CLI writes: the v2 format, the container digests
it resolved, and sdk-sdk floated to 00bb067, which picks up dagger/sdk-sdk#20 —
the harness now runs its contract checks on the configured CLI release. Before
it, a CLI older than beta.10 silently skipped every function taking a
`Workspace!` argument.

Signed-off-by: Tom Chauveau <tom@dagger.io>
TomChv added 2 commits August 21, 2026 12:20
Signed-off-by: Tom Chauveau <tom@dagger.io>
Dropping the polyfill does not have to cost the concurrent per-module codegen.
Folding the generated workspaces one into the next serializes them, and the
reason given for it — that per-module changesets can no longer be combined past
the changeset cwd cutover — only holds for combining them through a workspace.

Workspace.withChanges overlays at the workspace root, so handing it a changeset
the engine measured from ws.cwd misplaces every path for any cwd but the root.
Changeset.withChangesets merges the patches directly and never re-enters
workspace coordinates: cwd-measured operands, cwd-measured result, which is the
frame the engine applies a returned generator changeset in. That is the shape
the polyfill had, with its own re-rooting rather than the engine's.

The empty fold seed no longer needs a fork either: ws.changes(ws) compares a
workspace against itself, so it stays empty even under the engine's nested
per-dependency generation, where the incoming workspace already carries a
staged dependency closure.

Signed-off-by: Tom Chauveau <tom@dagger.io>
@TomChv
TomChv merged commit c53a5fb into main Aug 21, 2026
58 checks passed
TomChv added a commit that referenced this pull request Aug 24, 2026
Workspace.withNewDirectory replaces the directory it writes, where the
polyfill's fork.withDirectory merged onto it. Dropping the polyfill (#33)
flipped that semantic under two call sites, so both deleted files they do
not own.

initModule wiped the destination: `dagger module init typescript hello`
over a directory holding a foo.txt removed it, along with the module config
the engine writes there before calling the SDK. It now layers the rendered
starter onto existingDir(ws, modPath), as dagger/go-sdk#30 and
dagger/python-sdk#14 did.

generateClient / generateAllClient wiped hand-written files in the client
package — the bug #9 fixed with an overlay, lost when the polyfill went
away. Clients keep the user's files and still drop the *.gen.ts bindings of
a module that has left the closure: the SDK owns that set, the user owns
the rest. go-sdk and python-sdk keep plain replace for clients; this repo
does not, given #9.

Pruning has to happen twice, because withNewDirectory is not one operation.
On a local-directory workspace it replaces what it writes; on a synthetic
one — the shape a git-loaded workspace has, and how the Cloud checks runner
loads this repo — it merges. Isolated, same engine, same call:

  local removed:     main.ts, package.json, stale-dep.gen.ts
  synthetic removed: (nothing)

So the bindings come out of the baseline, covering replace, and off the
workspace with withoutFile, covering merge. The baseline still reads from
the untouched workspace: reading it back out of the pruned one comes up
empty on a local directory, which drops the user's files. Reported as
dagger/dagger#13955.

Three checks guard this, none of which pass without the fix:

  - init:init-over-existing-check inits over a fixture module and asserts
    removedPaths is empty. Before: dagger.json, index.ts, nested/.
  - client:generate-client-respects-existing-check gains main.ts (must
    survive) and stale-dep.gen.ts (must be pruned), so it fails under
    replace and under a plain overlay alike.
  - client:generate-client-on-synthetic-workspace-check covers the other
    half of the split, asserting the resulting tree rather than the
    removals, since removals are what diverge. It builds the workspace with
    ws.directory("/").asWorkspace, so the CI shape is reachable locally
    with no git pin.

A config-file fixture cannot catch any of it: config-updator merges those
files, so they survive a replace and the check passes anyway.

Verified with `dagger check 'e-2-e*'` (31/31) on a v1.0.0-beta.10 engine
and 60/60 in CI, plus an engine-driven `dagger module init typescript hello
-y` over a directory holding an unrelated file, which now keeps it.

Signed-off-by: Tom Chauveau <tom@dagger.io>
TomChv added a commit that referenced this pull request Aug 24, 2026
Workspace.withNewDirectory replaces the directory it writes, where the
polyfill's fork.withDirectory merged onto it. Dropping the polyfill (#33)
flipped that semantic under two call sites, so both deleted files they do
not own.

initModule wiped the destination: `dagger module init typescript hello`
over a directory holding a foo.txt removed it, along with the module config
the engine writes there before calling the SDK. It now layers the rendered
starter onto existingDir(ws, modPath), as dagger/go-sdk#30 and
dagger/python-sdk#14 did.

generateClient / generateAllClient wiped hand-written files in the client
package — the bug #9 fixed with an overlay, lost when the polyfill went
away. Clients keep the user's files and still drop the *.gen.ts bindings of
a module that has left the closure: the SDK owns that set, the user owns
the rest. go-sdk and python-sdk keep plain replace for clients; this repo
does not, given #9.

Pruning has to happen twice, because withNewDirectory is not one operation.
On a local-directory workspace it replaces what it writes; on a synthetic
one — the shape a git-loaded workspace has, and how the Cloud checks runner
loads this repo — it merges. Isolated, same engine, same call:

  local removed:     main.ts, package.json, stale-dep.gen.ts
  synthetic removed: (nothing)

So the bindings come out of the baseline, covering replace, and off the
workspace with withoutFile, covering merge. The baseline still reads from
the untouched workspace: reading it back out of the pruned one comes up
empty on a local directory, which drops the user's files. Reported as
dagger/dagger#13955.

Three checks guard this, none of which pass without the fix:

  - init:init-over-existing-check inits over a fixture module and asserts
    removedPaths is empty. Before: dagger.json, index.ts, nested/.
  - client:generate-client-respects-existing-check gains main.ts (must
    survive) and stale-dep.gen.ts (must be pruned), so it fails under
    replace and under a plain overlay alike.
  - client:generate-client-on-synthetic-workspace-check covers the other
    half of the split, asserting the resulting tree rather than the
    removals, since removals are what diverge. It builds the workspace with
    ws.directory("/").asWorkspace, so the CI shape is reachable locally
    with no git pin.

A config-file fixture cannot catch any of it: config-updator merges those
files, so they survive a replace and the check passes anyway.

Verified with `dagger check 'e-2-e*'` (31/31) on a v1.0.0-beta.10 engine
and 60/60 in CI, plus an engine-driven `dagger module init typescript hello
-y` over a directory holding an unrelated file, which now keeps it.

Signed-off-by: Tom Chauveau <tom@dagger.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants