Skip to content

WP Build: avoid infinite worker build loop#80361

Merged
jsnajdr merged 2 commits into
trunkfrom
update/vips-rebuild-fix
Jul 22, 2026
Merged

WP Build: avoid infinite worker build loop#80361
jsnajdr merged 2 commits into
trunkfrom
update/vips-rebuild-fix

Conversation

@simison

@simison simison commented Jul 16, 2026

Copy link
Copy Markdown
Member

What?

Stop wp-build --watch from rebuild-looping on generated src/worker-code.ts for worker packages (@wordpress/vips, @wordpress/video-conversion).

Why?

Those packages write bundled worker/WASM output back into src/worker-code.ts on every build. The watcher treated that as a source change and rebuilt again (~3–5s each), spamming ✅ vips during npm run dev / storybook:dev / npm run storybook:e2e:dev.

Screenshot 2026-07-16 at 18 20 26

How?

Ignore **/worker-code.ts in the package watcher, and skip it in isPackageSourceFile as a second guard.

Path is hardcoded and used in WP Build initially here:

const workerCodeFile = path.join( packageDir, 'src', 'worker-code.ts' );

Path is also already ignored by ESLint and Gitignore files around repo:

'packages/video-conversion/src/worker-code.ts',
'packages/vips/src/worker-code.ts',

# Auto-generated worker code for inline Blob URL creation.
# The build process generates a placeholder if needed, then overwrites it with real content.
src/worker-code.ts

# Auto-generated worker code for inline Blob URL creation.
# The build process generates a placeholder if needed, then overwrites it with real content.
src/worker-code.ts

Testing Instructions

Not sure other than running npm run storybook:dev or npm run storybook:e2e:dev for a while. :-)

Testing Instructions for Keyboard

Use of AI Tools

yes

@simison
simison requested a review from adamsilverstein July 16, 2026 15:28
@simison simison added the [Package] wp-build /packages/wp-build label Jul 16, 2026
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the props-bot label.

If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message.

Co-authored-by: simison <simison@git.wordpress.org>
Co-authored-by: youknowriad <youknowriad@git.wordpress.org>
Co-authored-by: jsnajdr <jsnajdr@git.wordpress.org>
Co-authored-by: adamsilverstein <adamsilverstein@git.wordpress.org>
Co-authored-by: ramonjd <ramonopoly@git.wordpress.org>

To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook.

@simison
simison requested a review from youknowriad July 16, 2026 15:30
@simison simison added [Type] Bug An existing feature does not function as intended [Type] Build Tooling Issues or PRs related to build tooling labels Jul 16, 2026
@github-actions

github-actions Bot commented Jul 16, 2026

Copy link
Copy Markdown

Size Change: 0 B

Total Size: 7.75 MB

compressed-size-action

@simison
simison force-pushed the update/vips-rebuild-fix branch from 99046be to 48e42ce Compare July 16, 2026 17:01
@simison
simison requested a review from jsnajdr July 16, 2026 18:47
@simison simison removed the [Type] Bug An existing feature does not function as intended label Jul 17, 2026
}

// Skip build-generated worker bundles written back into src/.
if ( relativePath.endsWith( '/src/worker-code.ts' ) ) {

@youknowriad youknowriad Jul 17, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How/where is this defined as an output?

(My adjacent thinking here, is that it feels that we're hard coding a lot of stuff here, in what is supposed to be a generic tool)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The worker-code.ts file is produced during build when processing the wpWorkers key in package.json. The vips and video-conversion packages currently define a worker.

It's part of a pipeline that:

  1. Bundles the wpWorkers entrypoint as a separate JS bundle.
  2. Produces a worker-code.ts file that exports the JS bundle as a big string.
  3. Imports the big string and creates a blob URL from it.
  4. Spawns a new web-worker with new Worker( blobURL ).
  5. Talks to the worker using a proxy object that implements methods as async message senders/receivers.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assumed it was one of those things where WP Build is just highly opinionated about things, but I'm not familiar with the Worker builder implementation or decisions; maybe @adamsilverstein is?

Probably a bit out of scope for this fix to change it, tho.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the produced worker-code.ts in "src"?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's been like that since #74785 and I see @ramonjd had a PR to move away from src/ to fix this exact same issue, but they closed it: #76118

Can't trace from those PRs the why.

@simison simison Jul 17, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could meanwhile put the path to a shared constant, but a proper fix would indeed be moving out from src but I'm not really familiar with this setup to touch it too much now. :-)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have code in src that imports the worker-code.ts module and uses it to load the worker:

import { workerCode } from './worker-code';
const blobUrl = URL.createObjectURL( new Blob( [ workerCode ] ) );
new Worker( blobUrl );

This ./worker-code import must resolve to something in all relevant build tasks. During TypeScript typechecking (tsc --build) and during esbuild build. If the worker-code.ts was outside src, how else would we import it? It would have to be a separate package or something.

We have a generate-worker-placeholders.mjs script that runs before TypeScript to generate a worker-code.ts placeholder file so that the tsc build that follows doesn't fail with unresolved modules. And then another build task generates the real files.

What @ramonjd tried in #76118 seems to be:

  1. Fool TypeScript with a mere worker-code.d.ts file that declares types and no code.
  2. Adding an esbuild plugin that externalizes the worker-code.ts import, postponing resolution until both the importer and importee are both in build.

For some reason that didn't work, maybe TypeScript wasn't fooled? The CI logs for that March 20 build are expired, it can't be immediately checked.

For the record, this wpWorkers pipeline is not particularly elegant, we should simplify it. But at the moment I don't know why.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have much to add here other that some of the complexity/needing to encode the has to do with our limited control of the runtime environment (eg. we can't rely on setting headers).

For the record, this wpWorkers pipeline is not particularly elegant, we should simplify it.

I am open to reworking how we approach this. For now, adding an explicit path here to avoid the build recursion seems fine.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason that didn't work, maybe TypeScript wasn't fooled?

Good summary! And thanks for getting this PR up.

Yeah the general thrust of my tests was to make sure imports of ./worker-code were resolved without writing generated code into src/.

I think the CI was borking pretty hard so I closed it without comment (or my attention was diverted). I'll fire #76118 up again, rebase and just let it run to document it.

@simison
simison force-pushed the update/vips-rebuild-fix branch from 48e42ce to 3bd1d59 Compare July 17, 2026 13:27
@github-actions

github-actions Bot commented Jul 17, 2026

Copy link
Copy Markdown

Flaky tests detected in c8a1116.
Some tests passed with failed attempts. The failures may not be related to this commit but are still reported for visibility. See the documentation for more information.

🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/29913425586
📝 Reported issues:

@simison
simison requested a review from ramonjd July 17, 2026 16:30
@simison
simison force-pushed the update/vips-rebuild-fix branch from 3bd1d59 to ba0103f Compare July 17, 2026 16:30
@simison
simison requested a review from andrewserong July 17, 2026 16:31
@jsnajdr
jsnajdr force-pushed the update/vips-rebuild-fix branch from ba0103f to c8a1116 Compare July 22, 2026 10:50

@jsnajdr jsnajdr left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even if we decide to improve the wasm/worker build system in the future, this is still a good fix to have.

@jsnajdr
jsnajdr enabled auto-merge (squash) July 22, 2026 10:51
@jsnajdr
jsnajdr merged commit ff229a4 into trunk Jul 22, 2026
60 checks passed
@jsnajdr
jsnajdr deleted the update/vips-rebuild-fix branch July 22, 2026 11:25
@github-actions github-actions Bot added this to the Gutenberg 23.7 milestone Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

[Package] wp-build /packages/wp-build [Type] Build Tooling Issues or PRs related to build tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants