Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent unpredictable dependency hosting by explicitly resolve module path on webpack.ProvidePlugin #20971

Merged
merged 1 commit into from Jan 11, 2021

Conversation

simnalamburt
Copy link
Contributor

@simnalamburt simnalamburt commented Jan 11, 2021

ProvidePlugin replaces certain identifiers with another modules. As a result, 'buffer' and 'process' modules are added as implicit dependencies to all Next.js plugins. This can be problematic. With "Plug'n'Play" strategy, it don't work at all since they fail-fast with implicit dependencies. With "node_modules" strategy, it might seem OK but actually it can be result into unpredictable behavior since in uses dependency hoisting.

For example, currently users cannot use next-auth plugin with Plug'n'Play strategy:

image

This patch let Next.js properly handles such cases with require.resolve.

Closes #20955

References:

ProvidePlugin replaces certain identifiers with another modules. As a
result, 'buffer' and 'process' modules are added as implicit
dependencies to all Next.js plugins. Which is OK for "node_modules"
strategy, but problematic with "Plug'n'Play" strategy.

This patch let Next.js properly handles such cases with Yarn PnP mode
and PNPM's PnP mode.

References:
  nextauthjs/next-auth#1034
@ijjk
Copy link
Member

ijjk commented Jan 11, 2021

Stats from current PR

Default Server Mode (Decrease detected ✓)
General Overall decrease ✓
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
buildDuration 10.6s 10.6s ⚠️ +83ms
nodeModulesSize 80.9 MB 76.2 MB -4.68 MB
Page Load Tests Overall decrease ⚠️
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
/ failed reqs 0 0
/ total time (seconds) 2.031 2.077 ⚠️ +0.05
/ avg req/sec 1230.91 1203.74 ⚠️ -27.17
/error-in-render failed reqs 0 0
/error-in-render total time (seconds) 1.255 1.253 0
/error-in-render avg req/sec 1991.97 1995.56 +3.59
Client Bundles (main, webpack, commons)
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
677f882d2ed8..89aa.js gzip 13.1 kB 13.1 kB
framework.HASH.js gzip 39 kB 39 kB
main-dba4ec1..27c0.js gzip 6.63 kB 6.63 kB
webpack-50be..df5b.js gzip 751 B 751 B
Overall change 59.4 kB 59.4 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
polyfills-81..14d7.js gzip 31.2 kB 31.2 kB
Overall change 31.2 kB 31.2 kB
Client Pages
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
_app-b6fc6bc..222c.js gzip 1.28 kB 1.28 kB
_error-e2ffa..0f3f.js gzip 3.46 kB 3.46 kB
hooks-010c20..8411.js gzip 887 B 887 B
index-bbee2f..528b.js gzip 227 B 227 B
link-d979103..c5ff.js gzip 1.64 kB 1.64 kB
routerDirect..bf84.js gzip 303 B 303 B
withRouter-a..5826.js gzip 302 B 302 B
Overall change 8.09 kB 8.09 kB
Client Build Manifests
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
_buildManifest.js gzip 323 B 323 B
Overall change 323 B 323 B
Rendered Page Sizes
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
index.html gzip 614 B 614 B
link.html gzip 619 B 619 B
withRouter.html gzip 606 B 606 B
Overall change 1.84 kB 1.84 kB

Serverless Mode (Decrease detected ✓)
General Overall decrease ✓
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
buildDuration 12.3s 12.1s -104ms
nodeModulesSize 80.9 MB 80.9 MB -210 B
Client Bundles (main, webpack, commons)
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
677f882d2ed8..89aa.js gzip 13.1 kB 13.1 kB
framework.HASH.js gzip 39 kB 39 kB
main-dba4ec1..27c0.js gzip 6.63 kB 6.63 kB
webpack-50be..df5b.js gzip 751 B 751 B
Overall change 59.4 kB 59.4 kB
Legacy Client Bundles (polyfills)
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
polyfills-81..14d7.js gzip 31.2 kB 31.2 kB
Overall change 31.2 kB 31.2 kB
Client Pages
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
_app-b6fc6bc..222c.js gzip 1.28 kB 1.28 kB
_error-e2ffa..0f3f.js gzip 3.46 kB 3.46 kB
hooks-010c20..8411.js gzip 887 B 887 B
index-bbee2f..528b.js gzip 227 B 227 B
link-d979103..c5ff.js gzip 1.64 kB 1.64 kB
routerDirect..bf84.js gzip 303 B 303 B
withRouter-a..5826.js gzip 302 B 302 B
Overall change 8.09 kB 8.09 kB
Client Build Manifests
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
_buildManifest.js gzip 323 B 323 B
Overall change 323 B 323 B
Serverless bundles
vercel/next.js canary simnalamburt/next.js pnp-plugin Change
_error.js 1 MB 1 MB
404.html 2.67 kB 2.67 kB
hooks.html 1.92 kB 1.92 kB
index.js 1 MB 1 MB
link.js 1.06 MB 1.06 MB
routerDirect.js 1.05 MB 1.05 MB
withRouter.js 1.05 MB 1.05 MB
Overall change 5.18 MB 5.18 MB
Commit: 06c8ec8

@merceyz
Copy link
Contributor

merceyz commented Jan 11, 2021

Again, as I said in #20955 (comment), this is not a PnP issue, please don't label it as such.

It's an issue that PnP just so happens to highlight, this is an issue for node_modules as well (Yarn 1, Yarn 2 w/ node_modules, and pnpm) see https://yarnpkg.com/advanced/rulebook#packages-should-only-ever-require-what-they-formally-list-in-their-dependencies for an explainer on hoisting

@simnalamburt
Copy link
Contributor Author

simnalamburt commented Jan 11, 2021

Again, as I said in #20955 (comment), this is not a PnP issue, please don't label it as such.

Oops I didn't know a discussion regarding this issue was going on at there. Yeah actually this is an issue even for node_modules since requiring implicit dependencies results into unpredictable behavior as you said.

@simnalamburt simnalamburt marked this pull request as ready for review Jan 11, 2021
@simnalamburt simnalamburt changed the title Handle Plug'n'Play properly while using webpack.ProvidePlugin Prevent unpredictable dependency hosting by explicitly resolve module path on webpack.ProvidePlugin Jan 11, 2021
@simnalamburt
Copy link
Contributor Author

simnalamburt commented Jan 11, 2021

@merceyz Thanks! I updated the title and description of this PR. Actually I knew that this is not an issue of only PnP but.. you know.. many node.js developers don't think relying on hoisting is a problem. So saying "it doesn't work with Yarn 2" was the best way for me to let maintainers know the situation. :)

@simnalamburt
Copy link
Contributor Author

simnalamburt commented Jan 11, 2021

Let me investigate the failed CI, but I'm not sure if it was really due to this patch.

@kodiakhq kodiakhq bot merged commit 552aa9d into vercel:canary Jan 11, 2021
30 of 33 checks passed
@timneutkens
Copy link
Member

timneutkens commented Jan 11, 2021

Thanks @simnalamburt, great PR 👍

@JakubKoralewski
Copy link

JakubKoralewski commented Feb 12, 2021

@simnalamburt, The error that you mention with Module not found: can't resolve 'process' is still here for me in a project with next-auth and next.js. Were you able to build a next.js app with yarn2 pnp?

Also while I write this, please paste the error in the issue next time cause I couldn't find it while googling 😅

@simnalamburt
Copy link
Contributor Author

simnalamburt commented Feb 12, 2021

Hi @JakubKoralewski! To use next-auth with Yarn 2 PnP:

  • Before this PR: Need to add "process" dependency to the next-auth and manually patch next-auth.
  • After this PR: Still need to add "process" dependency to the next-auth. Just no need to patch.

This is how it is. 🤦

Yes this PR has been merged and we don't need to manually patch the next-auth but still we need to fix the dependency list of next-auth like this:

# .yarnrc.yml
packageExtensions:
  next-auth@*:
    dependencies:
      process: "^0.11.10"

Actually I sent a PR to the next-auth to fix this issue, but they rejected it. Contrary to what they think, your issue did originate from next-auth but looks like they don't understand the full situation.

I don't have the time or energy to persuade them, so I'm just using next-auth like this, but if you're confident you'll change their minds, try send them a PR.

@JakubKoralewski
Copy link

JakubKoralewski commented Feb 12, 2021

Thanks for the snippet! Looks like I got a different error, so I guess it works :D

I don't have the time or energy to persuade them, so I'm just using next-auth like this, but if you're confident you'll change their minds, try send them a PR.

Not confident at all, but let's give them some time. Whether it will be Yarn or another package manager, plug and play is the future and once they start using it themselves they'll fix it 🤣

simnalamburt added a commit to simnalamburt/berry that referenced this issue Feb 20, 2021
I tried to patch NextAuth.js first but they rejected it. So I'm fixing
Next.js and Yarn instead. I already have fixed the Next.js side and this
is the remaining piece.

References:
  nextauthjs/next-auth#1034
  vercel/next.js#20971
simnalamburt added a commit to simnalamburt/berry that referenced this issue Feb 20, 2021
I tried to patch NextAuth.js first but they rejected it. So I'm fixing
Next.js and Yarn instead. I already have fixed the Next.js side and this
is the remaining piece.

References:
  nextauthjs/next-auth#1034
  vercel/next.js#20971
simnalamburt added a commit to simnalamburt/berry that referenced this issue Feb 20, 2021
I tried to patch NextAuth.js first but they rejected it. So I'm fixing
Next.js and Yarn instead. I already have fixed the Next.js side and this
is the remaining piece.

References:
  nextauthjs/next-auth#1034
  vercel/next.js#20971
@simnalamburt
Copy link
Contributor Author

simnalamburt commented Feb 20, 2021

Good news for @JakubKoralewski! I've opened a PR to fix next-auth's dependency by default! Take a look at yarnpkg/berry#2493 :)

@vercel vercel locked as resolved and limited conversation to collaborators Jan 29, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants