Buffer is not defined

[konsumer]

What version of Remix are you using?

1.3.5

Steps to Reproduce

• setup a remix cloudflare-worker project with mantine or any other library that uses Buffer. I followed the remix directions on the mantine site.
• remix build: works fine
• wrangler publish : fails

I can make a demo project that fails, if you want to try to deploy it to your own cloudflare.

Expected Behavior

Should output code that can be deployed on cloudflare.

Actual Behavior

It was reported at #2625 but said to be fixed in 1.3.5, so it may be some combination of remix/mantine/etc. I can build in remix fine, but on deploy (via wrangler), I get this:

✘ [ERROR] Received a bad response from the API

  Uncaught ReferenceError: Buffer is not defined
    at line 4699
    at line 33
    at line 5472
    at line 33
    at line 6626
    at line 33
    at line 6704
    at line 33
    at line 94195
   [code: 10021]


  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/wrangler2/issues/new

Normally, in other builders, a polyfill can be used, either in esbuild config, which we can't edit in remix, or by importing buffer. If I install buffer module & and this in server entry-point:

/* global addEventListener  */

import buffer from 'buffer'

import { createEventHandler } from '@remix-run/cloudflare-workers'
import * as build from '@remix-run/dev/server-build'

globalThis.Buffer = buffer

addEventListener(
  'fetch',
  createEventHandler({ build, mode: process.env.NODE_ENV })
)

I get this error:

Building Remix app in production mode...
It appears you're using a module that is built in to node, but you installed it as a dependency which could cause problems. Please remove buffer before continuing.
Error
    at Object.onBuildFailure (/Users/konsumer/Documents/work/PROJECT/node_modules/@remix-run/dev/cli/commands.js:150:13)
    at buildEverything (/Users/konsumer/Documents/work/PROJECT/node_modules/@remix-run/dev/compiler.js:280:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Object.build (/Users/konsumer/Documents/work/PROJECT/node_modules/@remix-run/dev/compiler.js:105:3)
    at async Object.build (/Users/konsumer/Documents/work/PROJECT/node_modules/@remix-run/dev/cli/commands.js:145:3)
    at async Object.run (/Users/konsumer/Documents/work/PROJECT/node_modules/@remix-run/dev/cli/run.js:467:7)

[konsumer]

Here is my related mantine issue. I am having a bit of trouble figuring out who to report it to, or where I could even work on a PR to fix the problem.

I made a demo project here, but couldn't even get it started using everyone's "getting started" directions, and remix-esbuild-override stopped working (which used to be how I solved this.)

[konsumer]

Related Discussion

[3x071c]

Cloudflare runs on V8 natively, without the additional Node.js abstraction. This means certain APIs, such as Buffer, aren't available for your dependencies to use in Cloudflare Workers/Pages. Try this, it allows you to override Remix's esbuild configuration to inject/alias polyfills. Here's an example on how to replace the Buffer API. If you want a full production usage example, check out this repository. I agree that Remix should be polyfilling Buffer by default on CF though, it's definitely a hurdle for adoption. (I think there already is an issue about that)

[konsumer]

Cloudflare runs on V8 natively, without the additional Node.js abstraction.

Yep, I'm aware. The way I generally solve this, in non-remix worker projects (I have quite a few) is to polyfill it in my esbuild (or whatever bundler I'm using) config, as I said above.

Try this, it allows you to override Remix's esbuild configuration to inject/alias polyfills.

I was originally using remix-esbuild-override but it stopped working in my project (as I mentioned above.)

I made this issue because I was told in discord that custom esbuild config would not be supported, but we could add issues for specific things I am having trouble with, in a case-by-case fashion. I'm not really here for helping/troubleshooting remix, or whatever, this is more for future-devs that have the same issue, which I imagine is a pretty widespread problem. If it's not helpful for me to just report the issue, like it doesn't seem important that users want to be able to fix their builds (with custom esbuild config) or more specifically, use any node library that uses Buffer, then feel free to close and mark "won't fix".

On my own project, I ended up just dropping remix. I could probably take the time to sort through what exact combo of versions of things started breaking, but honestly, remix is doing very little for me that I can't just do with a standard edge-worker and something that can build a static site (I am using vite, but also next static build works great, etc.) Our team is pretty comfortable with graphql and other react SSR frameworks, and edge-workers, so I kept feeling like we had to relearn things in a jenky way, that we could already do much better, the regular way. If I really needed SSR on this particular project, I might be more motivated to play around more with remix-esbuild-override, but it felt extremely flakey (often it would not correctly override, then just stopped working) and remix is just not needed in my current project. I had the whole thing rebuilt and working without remix, in about a day, and I will be hesitant to use or recommend remix, in the future.

My current setup is incredibly reliable, and all works much better:

• graphql server - this means easier management of queries/mutations, clear separation of data-service and UI code, and no flakey non-standard load/action stuff. "actions" are graphql mutations, and we don't have to specially form-encode hierarchical data to work with the action, the mutations just uses regular JSON, and it's all recorded in the schema for quick & easy error-checking.
• tons of other tooling around graphql, in general (in-page data-inspection plugins, editor addons, query-generators, code-generators, etc)
• It has full support for websockets, and they can be used easily, directly in graphql subscriptions
• it runs on cloudflare worker, on miniflare locally.
• I can access partially-remote service that uses remote DO + local KV, for a perfect combo of real data + fast/ephemeral dev (using cross-cf, a library I wrote)
• we have full control of all of the stack, including how it is bundled
• can work on the backend stuff separate from UI, even before it's built, with graphql-playground.
• worker pages built by vite, which runs with a proxy to the local dev backend, so it all runs like 1 site - it has better hot-reload (which never worked right for me, with remix) and builds faster.

This isn't that different from how remix works (it also builds worker-pages, and locally proxies to miniflare), but we have control of everything, and it all works in a more standard & reliable way.

If remix works for you, that's great. It's probably something I am doing wrong, as a new-to-remix person, that makes it extra flakey and prone to breaking, but I am also an experienced react/edge-worker developer, and I just followed the directions on these projects sites (remix, mantine, etc) to set things up, so in my opinion, even if it's not a great failing of remix as a framework, the docs could use some work. Maybe everything that was breaking for me works again, but as I said, I will be slow to use or recommend it, in the future, considering the time I feel like I wasted with it.

[kevinwolfcr]

@konsumer I just followed this guide and it worked.

I am using all the latest versions from remix and mantine packages as of today.

[konsumer]

@fullstackjedi I'm glad it worked for you. The experience in Apr left a very bad taste in my mouth, around remix, and it will probably be a while before I try it on an actual project, again. My setup I replaced it with is still working very well, so I will stick with that.

[shicholas]

fwiw I can reproduce this with a new project setup and the latest versions of remix/react.

[Viranchee]

I have similar error, trying to troubleshoot, might drop remix

[ErlanBelekov]

still running into this!

[grhbit]

I use version 1.14.0 of remix. I got the same error. It seems to check whether it uses Node’s built-in module.

Here is my two cents.

1. Use an aliased name in package.json to avoid built-in module checking and run npm install or pnpm install
   (You might need to update npm if the npm version is too old. I'm using pnpm anyway.)

{
  "dependencies": {
    "buffer-polyfill": "npm:buffer@^6.0.3"
  }
}

2. Create entry.client.ts and polyfill

import { RemixBrowser } from "@remix-run/react";
import { Buffer } from "buffer-polyfill";
import { hydrateRoot } from "react-dom/client";

globalThis.Buffer = Buffer as unknown as BufferConstructor;

hydrateRoot(document, <RemixBrowser />);

[lyquocnam]

hope will have solution to resolve this problem

[silence48]

globalThis.Buffer = Buffer as unknown as BufferConstructor;

This method works! using the latest remix and node 17.9.1

[nastynaz]

globalThis.Buffer = Buffer as unknown as BufferConstructor;

This method works! using the latest remix and node 17.9.1

What version of remix?

I'm still getting the issue with remix 1.15 and node 17.9.1

[dabdine]

Regarding the error above as it pertains to emotion/cloudflare/remix:

[mf:wrn] Parsed 1 valid header rule.
[mf:wrn] Service bindings are experimental. There may be breaking changes in the future.
[mf:err] ReferenceError: Buffer is not defined
    at node_modules/html-tokenize/node_modules/buffer-from/index.js (/private/Users/xxx/devel/corp-site/functions/[[path]].js:17750:59)
    at /private/Users/xxx/devel/corp-site/functions/[[path]].js:10:46
    at node_modules/html-tokenize/index.js (/private/Users/xxx/devel/corp-site/functions/[[path]].js:18569:22)
    at /private/Users/xxx/devel/corp-site/functions/[[path]].js:10:46
    at node_modules/@emotion/server/create-instance/dist/emotion-server-create-instance.cjs.dev.js (/private/Users/xxx/devel/corp-site/functions/[[path]].js:19886:49)
    at /private/Users/xxx/devel/corp-site/functions/[[path]].js:10:46
    at node_modules/@emotion/server/create-instance/dist/emotion-server-create-instance.cjs.js (/private/Users/xxx/devel/corp-site/functions/[[path]].js:19990:23)
    at /private/Users/xxx/devel/corp-site/functions/[[path]].js:10:46
    at /private/Users/xxx/devel/corp-site/functions/[[path]].js:45364:93
    at SourceTextModule.evaluate (node:internal/vm/module:226:23)

/private/Users/xxx/devel/corp-site/functions/[[path]].js:17750
    var toString5 = Object.prototype.toString, isModern = typeof Buffer.alloc == "function" && typeof Buffer.allocUnsafe == "function" && typeof Buffer.from == "function";

This worked for me on Remix 1.15.0 and react 18.2.0 using cloudflare pages:
https://github.com/aiji42/remix-esbuild-override/tree/main/examples/emotion-cloudflare

Just follow what's in the README, with a slight tweak. Instead of:

const { withEsbuildOverride } = require("remix-esbuild-override");
const GlobalsPolyfills =
  require("@esbuild-plugins/node-globals-polyfill").default;

withEsbuildOverride((option, { isServer }) => {
  if (isServer)
    option.plugins = [
      GlobalsPolyfills({
        buffer: true,
      }),
      ...option.plugins,
    ];

  return option;
});

Use this (just prepend to your remix.config.js):

const { withEsbuildOverride } = require("remix-esbuild-override");

withEsbuildOverride((option, { isServer }) => {
  if (isServer) {
    option.inject = [
      ...(option.inject ?? []),
      require.resolve("@esbuild-plugins/node-globals-polyfill/Buffer.js"),
      require.resolve("@esbuild-plugins/node-globals-polyfill/process.js"),
    ];
  }

  return option;
});

Which came from this issue:
aiji42/remix-esbuild-override#35 (comment)

[huw]

This has (arguably) been fixed upstream by Cloudflare—you should be able to opt-in to polyfilling Buffer and other Node globals if you want.

[jdnichollsc]

Hey folks, check my example to fix this issue https://gist.github.com/jdnichollsc/b73eca1f34bdd1e3a95283774fda2212

[VishakhaRastogi]

I am stilling facing this issue even after making changes in remix.config.js told in this comment
#2813 (comment)

Uncaught ReferenceError: Buffer is not defined
    at node_modules/@redis/client/dist/lib/client/commands-queue.js (commands-queue.js:299:18)
    at __require2 (chunk-PZDJHGND.js:18:50)
    at node_modules/@redis/client/dist/lib/client/index.js (index.js:17:26)
    at __require2 (chunk-PZDJHGND.js:18:50)
    at node_modules/@redis/client/dist/index.js (index.js:18:18)
    at __require2 (chunk-PZDJHGND.js:18:50)
    at node_modules/redis/dist/index.js (index.js:18:18)
    at __require2 (chunk-PZDJHGND.js:18:50)
    at redis.tsx:1:50

Using remix version 1.18.1

[markdalgleish]

The buffer polyfill is built into Remix when targeting non-Node server platforms, so similarly to #2813 (comment), you should be able to import the built-in buffer polyfill and assign it to globalThis.Buffer at the root of your app without needing to install any additional packages.

Note that in the upcoming Remix v2 you'll need to explicitly enable this polyfill via the serverNodeBuiltinsPolyfill option.

[markdalgleish]

@jtan80813 Have you manually put Buffer on global?

import { Buffer } from "buffer";

globalThis.Buffer = Buffer;

[jtan80813]

@markdalgleish. Yes I did in entry.client.ts, right?

[markdalgleish]

If you're using Cloudflare Workers, you might need this global polyfill on the server too.

[flatoy]

https://developers.cloudflare.com/workers/configuration/compatibility-dates/#nodejs-compatibility-flag

have you tried this?

[tleperou]

https://developers.cloudflare.com/workers/configuration/compatibility-dates/#nodejs-compatibility-flag

have you tried this?

Just tried; is not working for me.

But you can you it that way https://developers.cloudflare.com/workers/runtime-apis/nodejs/buffer/ – if you access the code running it though.