Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Usage of polyfill configuration throws error #10010

Closed
2 of 7 tasks
jeluard opened this issue Apr 3, 2024 · 6 comments
Closed
2 of 7 tasks

Usage of polyfill configuration throws error #10010

jeluard opened this issue Apr 3, 2024 · 6 comments
Labels
bug An error in the Docusaurus core causing instability or issues with its execution closed: can't repro This bug is because of some local setup that can't be reproduced.

Comments

@jeluard
Copy link

jeluard commented Apr 3, 2024

Have you read the Contributing Guidelines on issues?

Prerequisites

  • I'm using the latest version of Docusaurus.
  • I have tried the npm run clear or yarn clear command.
  • I have tried rm -rf node_modules yarn.lock package-lock.json and re-installing packages.
  • I have tried creating a repro with https://new.docusaurus.io.
  • I have read the console error message carefully (if applicable).

Description

When writing mdx components using external dependencies, it might be necessary to use polyfills. One way to do that is to create a docusaurus plugin leveraging node-polyfill-webpack-plugin.
After migrating to docusaurus 3.2 an exception is thrown when building the final website.

Note: it's not clear to me if this is the recommended approach to polyfill node.

Reproducible demo

jeluard/lodestar@4128951

Steps to reproduce

  1. cd docs
  2. Run yarn
  3. Run yarn build

Expected behavior

Compilation should finish and node specifics should be polyfilled.

Actual behavior

The following error is thrown:

[ERROR] Error: Unable to build website for locale en.
    at tryToBuildLocale (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/commands/build.js:53:19)
    at async /Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/commands/build.js:64:9
    at async mapAsyncSequential (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/utils/lib/jsUtils.js:20:24)
    at async Command.build (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/commands/build.js:62:5) {
  [cause]: Error: Docusaurus static site generation failed for 1 paths:
  - "/lodestar/"
      at generateStaticFiles (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/ssg.js:85:15)
      at async executeSSG (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/commands/build.js:146:23)
      ... 4 lines matching cause stack trace ...
      at async Command.build (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/commands/build.js:62:5) {
    [cause]: AggregateError
        at generateStaticFiles (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/ssg.js:86:20)
        at async executeSSG (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/commands/build.js:146:23)
        at async buildLocale (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/commands/build.js:126:31)
        at async tryToBuildLocale (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/commands/build.js:46:13)
        at async /Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/commands/build.js:64:9
        at async mapAsyncSequential (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/utils/lib/jsUtils.js:20:24)
        at async Command.build (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/commands/build.js:62:5) {
      [errors]: [
        Error: Can't render static file for pathname "/lodestar/"
            at generateStaticFile (/Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/@docusaurus/core/lib/ssg.js:119:15)
            at processTicksAndRejections (node:internal/process/task_queues:95:5)
            at runNextTicks (node:internal/process/task_queues:64:3)
            at process.processImmediate (node:internal/timers:449:9)
            at async /Users/julien/Documents/Projects/jeluard/lodestar/docs/node_modules/p-map/index.js:57:22 {
          [cause]: TypeError: Cannot read properties of undefined (reading 'id')
              at DocItem (server.bundle.js:20550:89)
              at Uc (server.bundle.js:32879:44)
              at Xc (server.bundle.js:32881:253)
              at Z (server.bundle.js:32887:89)
              at Xc (server.bundle.js:32885:231)
              at Z (server.bundle.js:32887:89)
              at Xc (server.bundle.js:32881:481)
              at Z (server.bundle.js:32887:89)
              at Vc (server.bundle.js:32879:473)
              at Xc (server.bundle.js:32881:210)
        }
      ]
    }
  }
}

Note that with docusaurus 3.1 a different error is thrown.

Your environment

  • Public source code: https://github.com/jeluard/lodestar/
  • Docusaurus version used: 3.2
  • Environment name and version (e.g. Chrome 89, Node.js 16.4): node 20
  • Operating system and version (e.g. Ubuntu 20.04.2 LTS): OSX

Self-service

  • I'd be willing to fix this bug myself.
@jeluard jeluard added bug An error in the Docusaurus core causing instability or issues with its execution status: needs triage This issue has not been triaged by maintainers labels Apr 3, 2024
@slorber
Copy link
Collaborator

slorber commented Apr 4, 2024

Hi

When writing mdx components using external dependencies, it might be necessary to use polyfills.

I'm not sure to follow, why exactly? How is this different from using external dependencies in React page components? How are external dependencies requiring a node polyfill different from first-party code you write that requires the exact same polyfill?

After migrating to docusaurus 3.2 an exception is thrown when building the final website.

Note that with docusaurus 3.1 a different error is thrown.

It's not clear from which version you are upgrading exactly. If it didn't work before, then how can we consider the new v3.2 breaks things?

Note: it's not clear to me if this is the recommended approach to polyfill node.

I don't know either. A minimal repro showing exactly the code to polyfill would help. It's not us that recommended usage of node-polyfill-webpack-plugin although it might work I have never used that package before myself. It is not an official Webpack plugin afaik and could also contain bugs.


jeluard/lodestar@jeluard/docusaurus-lc

This repro is not very actionable from my perspective. I can't investigate your monorepo in its entirety to look for problems because it's not minimal and many things on it could be the source of problems (for example, a dependency/hoisting problem related to the package manager)

Please create a minimal repro using https://docusaurus.new/stackblitz

I need to see a minimal project that works on a specific version before the upgrade, despite including a lib requiring polyfill. And I need to see it failing after the upgrade.

@slorber slorber added status: needs more information There is not enough information to take action on the issue. bug An error in the Docusaurus core causing instability or issues with its execution and removed bug An error in the Docusaurus core causing instability or issues with its execution status: needs triage This issue has not been triaged by maintainers labels Apr 4, 2024
@jeluard
Copy link
Author

jeluard commented Apr 4, 2024

I'm not sure to follow, why exactly? How is this different from using external dependencies in React page components? How are external dependencies requiring a node polyfill different from first-party code you write that requires the exact same polyfill?

What is different is the way things are configured in docusaurus. How can I tweak webpack's configuration in docusaurus?

I need to see a minimal project that works on a specific version before the upgrade, despite including a lib requiring polyfill. And I need to see it failing after the upgrade.

I can look at providing a minimal reproducing issue definitely. What is the recommended approach to hook polyfill, if any?
If I don't provide any polyfill, an error very similar to this one is thrown by webpack.

Note that in dev mode (i.e. docusaurus start), everything works as expected with my current approach. It only fail when building (via docusaurus build).

@jeluard jeluard changed the title Updating to 3.2 breaks usage of node-polyfill-webpack-plugin Usage of polyfill configuration throws error Apr 4, 2024
@slorber
Copy link
Collaborator

slorber commented Apr 4, 2024

I'm not sure to follow, why exactly? How is this different from using external dependencies in React page components? How are external dependencies requiring a node polyfill different from first-party code you write that requires the exact same polyfill?

What is different is the way things are configured in docusaurus. How can I tweak webpack's configuration in docusaurus?

https://docusaurus.io/docs/api/plugin-methods/lifecycle-apis#configureWebpack

I need to see a minimal project that works on a specific version before the upgrade, despite including a lib requiring polyfill. And I need to see it failing after the upgrade.

I can look at providing a minimal reproducing issue definitely.

That would help me help you 👍

What is the recommended approach to hook polyfill, if any? If I don't provide any polyfill, an error very similar to this one is thrown by webpack.

"very similar" does not help. If you have an error, please share it exactly as is.

We don't have a recommended approach for now, because most sites don't need it and this is not different from any other Webpack app. If you can make it work outside Docusaurus, you can probably make it work similarly inside.

I still don't even know what contains your code to require such polyfill.

None of the 2 reported errors in this issue (v3.1 or v3.2) look related to polyfills.

Note that in dev mode (i.e. docusaurus start), everything works as expected with my current approach. It only fail when building (via docusaurus build).

Probably because the problem is due to using browser APIs in Node.js, which is likely not a polyfill issue. I will only be able to help if I see a minimal repro.

@jeluard
Copy link
Author

jeluard commented Apr 4, 2024

You can find a minimal repository here. First commit just uses the lib w/o polyfill, then the second adds a polyfill.

https://docusaurus.io/docs/api/plugin-methods/lifecycle-apis#configureWebpack

This is what I did here by providing a plugin, so I would assume that this is the right approach.

I still don't even know what contains your code to require such polyfill.

Some libraries relies on node code (e.g. path, Buffer or crypto) and expect polyfills to run in a browser.

Probably because the problem is due to using browser APIs in Node.js, which is likely not a polyfill issue. I will only be able to help if I see a minimal repro.

That's the other way around, those libs use node APIs e.f. crypto or path. Historically browserify or others have been used to address that.

I've seen a number of workaround in the wild, so looks like others people have those needs:

@slorber slorber removed the status: needs more information There is not enough information to take action on the issue. label Apr 4, 2024
@slorber
Copy link
Collaborator

slorber commented Apr 4, 2024

I'm sorry but this repro is still not minimal to me. You don't even mention "the lib" that requires polyfills.

The minimal repro should have the code requiring to polyfill inlined, so that we can focus on that, and solely that. Otherwise you are asking me to retro-engineer your @lodestar/light-client package, which I don't plan to do.

Reading the error message, it's still not clear to me how you could be 100% convinced the problem is related to polyfills and nodejs APIs. That's why I want to see very simple code. Just add require("crypto") in your homepage for example and show me that polyfilling it can't work, and we'll be sure it's a polyfill issue in the first place. Otherwise it can be anything coming from your lib.

It's not clear to me why you use both resolve aliases/fallbacks in addition to the polyfill plugin you mentioned above. Why both at the same time? Is this how you are supposed to polyfill things?

Also, do we finally agree that this is not particularly related to v3.2? Because you changed the title.

BTW you should rather do this, otherwise, you risk polyfilling Node APIs that are already provided and override them.

configureWebpack(config, isServer, utils) {
  if (isServer) {
    return;
  }

@slorber
Copy link
Collaborator

slorber commented Apr 4, 2024

You are using this component inside your homepage:

<LatestOptimisticHeader>

import {useEffect, useState} from "react";
import {getClient} from "@lodestar/api";
import {createChainForkConfig} from "@lodestar/config";
import {networksChainConfig} from "@lodestar/config/networks";

import {Lightclient, LightclientEvent} from "@lodestar/light-client";
import {LightClientRestTransport} from "@lodestar/light-client/transport";
import {getFinalizedSyncCheckpoint, getGenesisData, getLcLoggerConsole} from "@lodestar/light-client/utils";

const config = createChainForkConfig(networksChainConfig.mainnet);
const logger = getLcLoggerConsole({logDebug: true});
const api = getClient({urls: ["https://lodestar-mainnet.chainsafe.io"]}, {config});

const lightclient = await Lightclient.initializeFromCheckpointRoot({
  config,
  logger,
  transport: new LightClientRestTransport(api),
  genesisData: await getGenesisData(api),
  checkpointRoot: await getFinalizedSyncCheckpoint(api),
  opts: {
    allowForcedUpdates: true,
    updateHeadersOnForcedUpdate: true,
  },
});

await lightclient.start();

export function LatestOptimisticHeader(): React.JSX.Element {
  const [head, setHead] = useState<string | undefined>();
  const [updated, setUpdated] = useState<boolean>(false);
  useEffect(() => {
    lightclient.emitter.on(LightclientEvent.lightClientOptimisticHeader, (optimisticUpdate) => {
      setHead(optimisticUpdate.beacon.slot.toString());
      setUpdated(true);
      setTimeout(() => setUpdated(false), 1000);
    });
  }, []);
  return (
    <span>
      Latest optimistic header: <span className={updated ? "highlight" : ""}>{head}</span>
    </span>
  );
}

Again, this is not minimal at all. You are expecting me to retro-engineer your code.

The only thing I can tell you is that this has never worked in Docusaurus, not in v3.2 or any previous versions, simply because we don't support ESM and top-level await. As soon as you remove usage of this component, your Docusaurus site can build, so this code is the problem.

I'm closing because I doubt there is a bug in Docusaurus. We'll re-open when a truly minimal repro is provided, not involving the complexity of your custom dependencies and the code above which can't work.

@slorber slorber closed this as not planned Won't fix, can't repro, duplicate, stale Apr 4, 2024
@slorber slorber added the closed: can't repro This bug is because of some local setup that can't be reproduced. label Apr 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug An error in the Docusaurus core causing instability or issues with its execution closed: can't repro This bug is because of some local setup that can't be reproduced.
Projects
None yet
Development

No branches or pull requests

2 participants