[SSR][NextJS] Duplicate atom key during development and during production build in nextjs

[janus-reith]

Both during local development and also when building a production build, I’m getting a duplicate atom key for each key and each page using it after the first one.

I put together a quick sandbox to demonstrate the issue:
https://codesandbox.io/s/flamboyant-sea-tqlky

The errors show up in the build log, a quick way to test is using the integrated vercel deployment button.
Looking at the default nextjs example, I can‘t spot any special settings there that would prevent whatever duplication is going on from happening: https://github.com/vercel/next.js/tree/canary/examples/with-recoil
That example however only makes use of recoil state on one page.

[alexilyaev]

Also mentioned in:
#213 (comment)

This is how it looks like in the Vercel deployment build:

[leonardomgt]

Has this been solved? I'm having the same issue.

[janus-reith]

@leonardomgt I was not able to solve this yet.

@drarmstr Tagging you as you seem to be a main contributor to this project, any chance you could have look at this? Thanks!

[drarmstr]

@janus-reith - Sorry, I'm not very familiar with Next.JS or SSR.

[alexilyaev]

@janus-reith - Sorry, I'm not very familiar with Next.JS or SSR.

Next.js has a concept of Pages. Imagine an SPA with multiple entry points, like an Admin and a Client App, both importing a module that declares a Recoil atom. So it doesn't have to be Next.js related.
Essentially it's a single Node.js process that executes JS files (build process) that eventually declare the same atom several times.

In development, when a file is changed, Next.js re-builds the relevant page entry file.
Because it's the same Node.js process, the atom has already been declared.
The same thing can happen with HMR when the file change triggers a rebuild of the whole file, or even when the atom is declared inside a component lifecycle/hook and only that is being hot-replaced.

Basically, I can't think of a solution to this problem besides providing a configuration setting to disable this check/warning.
At a minimum, developers should be able to disable it.
In Next.js or SSR that would be when running on the server.
In HMR, that would be when not in production env.

A zero config alternative might be tempting. E.g. relying on NODE_ENV=production like React does for it's optimizations.
It would probably solve the HMR issue, but not the build of Next.js, as that runs with NODE_ENV=production obviously.
Although, that would already be better than what we currently have.

[drarmstr]

Related to #247

[janus-reith]

Thanks @drarmstr for the response nonetheless, and thanks @alexilyaev for the detailed description.

Regarding the multiple entry points, I wonder if this could be worked around by some more advanced tracing logic to distinguish actual redeclaration of the key from the same code being executed again using the callsite?
I guess that depending on the way atoms are initialized there could be cases where this is not fully reliable(e.g. some dynamic declaration with variable-based key names ), also I'm not sure about browser support but still, maybe that is a feasible approach to consider.

[Him-2C]

Me to for this issue.

[juanpprieto]

Hey guys, any update on this issue? Has anybody come up with a workaround? Thanks!

[juanpprieto]

Ok guys, here's a temporary "solution" via a webpack "plugin" that kills the Duplicate Warnings by literally modifying the recoil source. sadness

from...

function registerNode(node) {
  if (nodes.has(node.key)) {
    ....code
    console.warn(message); // @oss-only
  }
  ....code
}

to:

function registerNode(node) {
  if (nodes.has(node.key)) {
    ....code
    (!process.env.__NEXT_PROCESSED_ENV && !process.env.SILENCE_RECOIL_DUPE_WARNING) && console.warn(message); // @oss-only
  }
  ....code
}

✅ Am I desperate?
✅ Is this ugly?
✅ is it a huge hack?
✅ Works? tested in next@^10.0.3 / recoil@^0.1.2 with SILENCE_RECOIL_DUPE_WARNING=true
✅ Could work outside nextjs?. Maybe.. SILENCE_RECOIL_DUPE_WARNING=true
✅ Works in netlify ?
⚠️ Is this a joke?
🚫 Fully tested (not even close)
🤷‍♂️ Its safe to use in production

// feel free to replace these
const fs = require('promise-fs')
const path = require('path')

/**
 Webpack plugin that modifies /node_modules/recoil/[...]/recoil.js
 adding process.env conditionals before the registerNode/duplicate key warning log
**/
{
  apply: (compiler) => {
    return compiler.hooks.run.tapAsync(
      "SilenceRecoilDupeWarningBeforeCompile",
      async (compilation, callback) => {
        const SILENCING_VARS = [
          "__NEXT_PROCESSED_ENV",
          "SILENCE_RECOIL_DUPE_WARNING",
        ];

        const silenceModule = async (module) => {
          // target recoil module & path (Adjust for your own project) 👈 mono repo setup
          const recoilModulePath = path.join(
            process.cwd(),
            "..",
            `/node_modules/${module}`
          );

          console.log(`Disabling ${recoilModulePath} warnings`);

          // read the source module
          const recoilSource = await fs.readFile(recoilModulePath, "utf8");

          // any of these will silence the warning
          const conditionalString = `(${SILENCING_VARS.reduce(
            (condStr, envVar) => {
              condStr = [...condStr, `!process.env.${envVar}`];
              return condStr;
            },
            []
          ).join(" && ")}) &&`;

          // regex and replace function
          const duplicateWarningRegex = /(if \(nodes\.has\(node\.key\)\) \{.*?)(console\.warn\(message\)\;)(.*?\})/gs;

          const duplicateWarningReplace = (str, $1, $2, $3) => {
            if ($1 && $3) {
              return `${
                $1 ? $1.replace(conditionalString, "") : ""
              }${conditionalString} console.warn(message);${$3}`;
            }
            return str;
          };

          // modify the recoil source file
          const modifiedRecoilSource = recoilSource.replace(
            duplicateWarningRegex,
            duplicateWarningReplace
          );

          // overwrite the recoil module - cause you can 🤷‍♂️
          await fs.writeFile(recoilModulePath, modifiedRecoilSource, "utf8");
        };

        try {
          const TARGET_RECOIL_MODULES = [
            "recoil/cjs/recoil.js",
            "recoil/es/recoil.js",
            "recoil/umd/recoil.js",
          ];

          for (let m = 0; m < TARGET_RECOIL_MODULES.length; m++) {
            const currentModule = TARGET_RECOIL_MODULES[m];
            await silenceModule(currentModule);
          }

          console.log("Disabled recoil duplicate key warning (hopefully).");
          callback();
        } catch (error) {
          console.log("SilenceRecoilDupeWarningBeforeCompile", error);
          callback();
        }
      }
    );
  };
}

Hopefully this enthusiasm sparks a proper solution... 👍

UPDATE: less invasive solution for nextjs here

[adamhenson]

I am reproducing this on my Next.js app. I only do SSR when running locally in dev mode, otherwise in production - I export static files via SSG. From what I can tell, this is just a warning and safe to ignore. Even though I'm a newbie to Recoil - it seems to be working 😨? I actually only see the output in the server logs... not the browser. Although the output is annoying, I just want to make sure this isn't a blocker in using Recoil. Can I just ignore the output? Or is this issue about actual broken functionality?

[juanpprieto]

I am reproducing this on my Next.js app. I only do SSR when running locally in dev mode, otherwise in production - I export static files via SSG. From what I can tell, this is just a warning and safe to ignore. Even though I'm a newbie to Recoil - it seems to be working 😨? I actually only see the output in the server logs... not the browser. Although the output is annoying, I just want to make sure this isn't a blocker in using Recoil. Can I just ignore the output? Or is this issue about actual broken functionality?

You can ignore the warning output (if you can bare it), functionality is not affected (We have a few sites in production working just fine) :(

[R-Bower]

I'm very eager to start using Recoil in my applications, but errors like this make me wary of adopting. Any official update on this? I'd prefer a solution that doesn't involve modifying the source code in node_modules.

[juanpprieto]

I feel you, I love recoil too much (specially on Nextjs), so I've taken a leap of faith that the staff will do something about it since SSR environments are here to stay and we have very complex production apps running beautifully with it. Fingers crossed 🙏🏼

[R-Bower]

@juanpprieto I've found a less invasive method that relies on the intercept-stdout package.

Add the following to next.config.js (outside of the exported configuration):

const intercept = require("intercept-stdout")

// safely ignore recoil warning messages in dev (triggered by HMR)
function interceptStdout(text) {
  if (text.includes("Duplicate atom key")) {
    return ""
  }
  return text
}

if (process.env.NODE_ENV === "development") {
  intercept(interceptStdout)
}

This way I don't have to modify the source code in node_modules. Do you see any downsides to this approach?

[juanpprieto]

@juanpprieto I've found a less invasive method that relies on the intercept-stdout package.

Add the following to next.config.js (outside of the exported configuration):

const intercept = require("intercept-stdout")

// safely ignore recoil warning messages in dev (triggered by HMR)
function interceptStdout(text) {
  if (text.includes("Duplicate atom key")) {
    return ""
  }
  return text
}

if (process.env.NODE_ENV === "development") {
  intercept(interceptStdout)
}

This way I don't have to modify the source code in node_modules. Do you see any downsides to this approach?

Hey @R-Bower!

On paper this seems 1000 times less invasive than my hack!

I will test at my end and let you know! — Thanks for the heads up!

[R-Bower]

The error still shows on in-browser hot reloads. Not sure if there's a way to suppress those other than the chrome console regex filter.

[juanpprieto]

The error still shows on in-browser hot reloads. Not sure if there's a way to suppress those other than the chrome console regex filter.

Yes, intercept-stdout works well for stdout suppressing in both development and production. Unfortunately, browser warnings are still there. I tried using webpack native stats suppressing and webpack-filter-warnings-plugin, but those two fail to mute recoil warning for some reason, even though it does block other warnings I wanted to mute such as Critical dependency: the request of a dependency is an expression.

This why I ended up doing the hack I did unfortunately. If anyone knows another option to try for muting browser warnings I'd love to hear and try them. Thanks!

[juanpprieto]

@R-Bower think i may have found a cleaner way to kill recoil browser warnings.

src/pages/_app.jsx — Intercept client/browser warnings. Outside the exported react component

const memoize = (fn) => {
  let cache = {};
  return (...args) => {
    let n = args[0];
    if (n in cache) {
      return cache[n];
    }
    else {
      let result = fn(n);
      cache[n] = result;
      return result;
    }
  }
}


// ignore in-browser next/js recoil warnings until its fixed.
const mutedConsole = memoize((console) => ({
  ...console,
  warn: (...args) => args[0].includes('Duplicate atom key')
    ? null
    : console.warn(...args)
}))

global.console = mutedConsole(global.console);

next.config.js — Intercept stdout warnings. Outside the exported webpack config

const intercept = require("intercept-stdout")

// safely ignore recoil stdout warning messages 
function interceptStdout(text) {
  if (text.includes('Duplicate atom key')) {
    return ''
  }
  return text
}

// Intercept in dev and prod
intercept(interceptStdout)

Let me know if this works for you!

[hsh2001]

@juanpprieto It works! Thanks :)

[spookyvert]

@R-Bower think i may have found a cleaner way to kill recoil browser warnings.

src/pages/_app.jsx — Intercept client/browser warnings. Outside the exported react component

// ignore in-browser next/js recoil warnings until its fixed.
const mutedConsole = memoize((console) => ({
  ...console,
  warn: (...args) => args[0].includes('Duplicate atom key')
    ? null
    : console.warn(...args)
}))
global.console = mutedConsole(global.console);

next.config.js — Intercept stdout warnings. Outside the exported webpack config

const intercept = require("intercept-stdout")

// safely ignore recoil stdout warning messages 
function interceptStdout(text) {
  if (text.includes('Duplicate atom key')) {
    return ''
  }
  return text
}

// Intercept in dev and prod
intercept(interceptStdout)

Let me know if this works for you!

This hangs when I start the server on next.js

[juanpprieto]

we got there in the end <3

[christian-gama]

Just faced this issue... Came here and saw this patch 0.7.6 released yesterday. Am I lucky? 😆

[aam051102]

@wfortin is correct. Recoil 0.7.6 with RECOIL_DUPLICATE_ATOM_KEY_CHECKING_ENABLED=false in .env solved the issue in our NextJS project. Thank you very much for that information.

[coderhzy]

const intercept = require('intercept-stdout');

// safely ignore recoil stdout warning messages
function interceptStdout(text) {
if (text.includes('Duplicate atom key')) {
return '';
}
return text;
}

intercept(interceptStdout);

[Mecil9]

I updated to version 0.7.6 and added the RECOIL_DUPLICATE_ATOM_KEY_CHECKING_ENABLED=false environment variable and it solved it for me. I think this ticket can be closed now.

I've been waiting a year for this answer!!!!!!

[hamtarodev]

how can i do this with Vite? on Vite it requires that the env variables should start with VITE_

[satokoki645]

how can i do this with Vite? on Vite it requires that the env variables should start with VITE_

I was thinking this same thing.

[ThomasCarstens]

@hamtarodev @satokoki645 The recoil changelog will help you, as of 6/10/22: https://yarnpkg.com/package/recoil

[tesseractjh]

@ThomasCarstens Thanks!
@hamtarodev @satokoki645 RecoilEnv can solve it with Vite.

// configs/recoil.ts
import { RecoilEnv } from 'recoil';

RecoilEnv.RECOIL_DUPLICATE_ATOM_KEY_CHECKING_ENABLED = false;

// App.tsx
import './configs/recoil';

function App() {
  return (
    <RecoilRoot>
        { ... }
    </RecoilRoot>
  );
}

[qqww08]

Thanks to this, I was able to remove the console log from nextJS. This is my code

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
  env: {
// @see https://github.com/facebookexperimental/Recoil/issues/2135#issuecomment-1362197710
    RECOIL_DUPLICATE_ATOM_KEY_CHECKING_ENABLED: "false",
  }
};

module.exports = nextConfig;

[LOGANLEEE]

@ThomasCarstens Thanks! @hamtarodev @satokoki645 RecoilEnv can solve it with Vite.

// configs/recoil.ts
import { RecoilEnv } from 'recoil';

RecoilEnv.RECOIL_DUPLICATE_ATOM_KEY_CHECKING_ENABLED = false;

// App.tsx
import './configs/recoil';

function App() {
 return (
   <RecoilRoot>
       { ... }
   </RecoilRoot>
 );
}

solved problem.

in my case

• console error on browser but not IDE if I set on env
• no console error on browser but on IDE if I set on next.config.js

lovely thanks.

[MejanH]

Just because of this issue I had to move to Jotai. disabling the Duplicate key checker is not a fix. now, recoil won't check for duplicate keys.

[SGDS666]

I tried all the above methods but they didn't solve my problem. Every time I refresh the page, I report an error, so I rudely used the following method to solve the problem

console.error = (messages, ...rest) => {
  if (messages?.message?.includes("Duplicate atom key")) {
    return
  }
  console.warn(messages, ...rest)
}

[mickdewald]

Just because of this issue I had to move to Jotai. disabling the Duplicate key checker is not a fix. now, recoil won't check for duplicate keys.

It's said, I know.
I will check out Zustand, as I already have for my mobile app. It works like a charm