ipfs-http-client breaks Webpack builds

[dapplion]

With 48.1.2 a server webpack breaks with (error message below). See CI run https://github.com/dappnode/DNP_DAPPMANAGER/pull/516/checks?check_run_id=1563460710#step:3:279

---

webpack.config.js

const path = require("path");
const { NODE_ENV = "production" } = process.env;
module.exports = {
  entry: "./src/index.ts",
  mode: NODE_ENV,
  target: "node",
  output: {
    path: path.resolve(__dirname, "build"),
    filename: "index.js"
  },
  // externals: [/node_modules/, "bufferutil", "utf-8-validate"],
  resolve: {
    extensions: [".ts", ".js"]
  },
  module: {
    rules: [
      {
        test: /\.ts$/,
        use: ["ts-loader"]
      }
    ]
  },
  optimization: {
    // Minimization does not provide great disk space savings, but reduces debug capacity
    minimize: false
  },
  devtool: "source-map"
};

---

webpack error

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts
[tsl] ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts(69,26)
      TS1005: ',' expected.

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts
[tsl] ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts(69,61)
      TS1005: ',' expected.

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts
[tsl] ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts(69,151)
      TS1005: ',' expected.

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts
[tsl] ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts(69,180)
      TS1005: ',' expected.

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts
[tsl] ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/dist/src/index.d.ts(69,215)
      TS1005: ',' expected.

+ many more lines with the same error

[dapplion]

It also breaks with a different error

ERROR in /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/electron-fetch/lib/index.es.js
Module not found: Error: Can't resolve 'electron' in '/home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/electron-fetch/lib'
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/electron-fetch/lib/index.es.js 1262:13-32
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-utils/src/fetch.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-utils/src/http/fetch.browser.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-utils/src/http/fetch.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-utils/src/http.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/src/lib/core.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/src/lib/configure.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/src/version.js
 @ /home/lion/Code/dappnode/DNP_DAPPMANAGER/node_modules/ipfs-http-client/src/index.js
 @ ./src/modules/ipfs/ipfsSetup.ts
 @ ./src/modules/ipfs/methods/pinAdd.ts
 @ ./src/ethForward/index.ts
 @ ./src/index.ts

[achingbrain]

So we can replicate the issue, what version of webpack is this?

[dapplion]

https://github.com/dappnode/DNP_DAPPMANAGER/blob/60ac93345872a9b8f66bd1ac63b7b620dddabe65/packages/dappmanager/package.json#L91

"webpack": "^4.43.0",
"webpack-cli": "^3.3.11"

[TheoXD]

I'm in the same boat, building with ipfs-http-client just spams the console with "',' expected" errors.

[perfectmak]

I just experienced this now. Is there any update on how to fix it?

I was using ipfs-http-client version 49.0.2. I had to downgrade to 47.0.0 and I stopped getting the error.

[NickBeukema]

I've also experienced this issue when deploying a function into AWS Lambda, Webpack errors with the following issue:

ERROR in /build/node_modules/electron-fetch/lib/index.es.js
Module not found: Error: Can't resolve 'electron' in '/build/node_modules/electron-fetch/lib'
 @ /build/node_modules/electron-fetch/lib/index.es.js 1262:13-32
 @ /build/node_modules/ipfs-utils/src/fetch.js
 @ /build/node_modules/ipfs-utils/src/http/fetch.browser.js
 @ /build/node_modules/ipfs-utils/src/http/fetch.js
 @ /build/node_modules/ipfs-utils/src/http.js
...

I'm somewhat concerned that ipfs-http-client ends up pulling in Electron through ipfs-utils, am I interpreting that correctly?

Is the suggested fix for this to downgrade?

[jthomerson]

I have the same problem. Then when I installed a couple different versions (47 and 55), I got "Module "stream" has been externalized for browser compatibility and cannot be accessed in client code."

[achingbrain]

@jthomerson stream is a node js module and webpack doesn't polyfill them since version 5. If your project build is pulling it in you need to make it available somehow.

There are examples of bundling ipfs and ipfs-http-client using webpack here: https://github.com/ipfs-examples?q=webpack

[jthomerson]

@achingbrain thanks. I probably should've clarified a bit. I'm aware of stream being for node. What I don't understand is why it's being pulled in. I assumed it was a problem with ipfs-http-client itself, especially since I had the same Electron-centric error originally posted in this bug. But the problem may be the bundler itself. I'm basically following this React-based tutorial, but instead of using React, I started the project with Vitesse - so Vue for the framework, Vite for the bundler.

If anyone happens to have any ideas on why Vite might be pulling in Stream, I'd be happy to hear them. All I'm doing in the code is this:

import { create as ipfsHttpClient } from 'ipfs-http-client'

const client = ipfsHttpClient('https://ipfs.infura.io:5001/api/v0')

const added = await client.add(
  file,
  {
    progress: prog => console.log(`received: ${prog}`),
  },
)
const url = `https://ipfs.infura.io/ipfs/${added.path}`
item.value.fileURL = url

[achingbrain]

I'm unfamiliar with Vite, but it looks like it uses Rollup under the hood and not webpack so I don't think it's related to this issue.

[ShravanSunder]

This is an additional problem with any bundler that requires esm modules. It looks like ipfs-http-client is requiring electron and electron-fetch for some reason in versions after 47.

[ShravanSunder]

You should be able to work around this by marking electron and electron-fetch as external in webpack

I've managed to replicate and work around this in vite using this plugin
and naming electron and electron-fetch as externals

[milahu]

I started the project with Vitesse - so Vue for the framework, Vite for the bundler.

related, since vite is similar to rollup

• Getting IPFS to work with ES Modules and Rollup #1927
  • https://github.com/DougAnderson444/ipfs-vite-svelte-kit

[SgtPooki]

js-ipfs is being deprecated in favor of Helia. You can #4336 and read the migration guide.

Please feel to reopen with any comments by 2023-06-02. We will do a final pass on reopened issues afterward (see #4336).

Unfortunately, this won't be resolved in js-ipfs repos, but Helia should not run into these problems. If you're still on CJS, you will likely be between a rock and a hard place, but I highly recommend updating your package to ESM and using vite/rollup bundling to solve cjs/esm pains.