Dependency of a .cjs build may not be loaded correctly #35112
Labels
bug
Issue was opened via the bug report template.
Comments
|
If I remove Edit: it doesn't seem to work in all scenarios either though... |
eric-burel
added a commit
to VulcanJS/vulcan-npm
that referenced
this issue
Mar 7, 2022
|
I've found a workaround that keeps two things stood out to me, the first being the swcMinify flag
this will make nextjs reject Have to set swcMinify to false or leave it out entirely to avoid fs and dynamic require related errors Nextjs needs a point of reference to handle consumption of the tsup-bundled package in the absence of a dts sourcemap especially if it isn't a browser-oriented package that can be handled with fix: create a root declare the package as a module as follows in the root of your nextjs repo (or whatever client-rendering repo it happens to be)
/// <reference types="@takedajobs/indexer/dist/index" />
declare module "@takedajobs/indexer";Inside of my indexer package, I'm augmenting the NodeJS namespace in the root as well, using a global.d.ts file to support the required environmental variables namespace NodeJS {
interface ProcessEnv {
ALGOLIA_SEARCH_KEY: string;
ALGOLIA_WRITE_KEY: string;
ALGOLIA_INDEX: string;
ALGOLIA_APP_ID: string;
ALGOLIA_INDICES?: string;
SANITY_DATASET: string;
SANITY_PROJECT_ID: string;
SANITY_API_TOKEN: string;
SANITY_API_VERSION: string;
SANITY_DATASET_TAG?: string;
SANITY_REVALIDATE_SECRET?: string;
SANITY_PREVIEW_SECRET?: string;
}
}Reinforced with a this way the env variables can be extracted from the .env directly or streamed using js-yaml and a couple well placed buffers then passed into if any of the required variables are missing it throws an error to the user of the package stating which env variable they're missing in their config import path from "node:path";
import * as dotenv from "dotenv";
dotenv.config({ path: path.join(process.cwd(), ".env") });
import { expand } from "dotenv-expand";
import { YamlStreamer } from "../utils/yaml-streamer";
import type { DotenvExpandOutput } from "dotenv-expand";
import type { Indexer } from "../types/namespace";
export const defaultPath = () => "takeda.config.yaml" as const;
export const SecretObj = Array.of<NonNullable<DotenvExpandOutput["parsed"]>>();
export const configPath = (path?: `${string}`): string | "takeda.config.yaml" =>
path ? path : defaultPath();
export const streamYamlFile = (path?: string) => YamlStreamer(
configPath(path)
) as Indexer.Yaml.RootConfig;
export const YamlSecrets = (streamYamlFile: Indexer.Yaml.RootConfig) => {
const expanded = expand({
parsed: {
["ALGOLIA_INDICES"]:
streamYamlFile.algoliaConfig?.alternativeIndices ?? "",
["ALGOLIA_APP_ID"]: streamYamlFile.algoliaConfig?.appId ?? "",
["ALGOLIA_SEARCH_KEY"]: streamYamlFile.algoliaConfig?.searchApiKey ?? "",
["ALGOLIA_WRITE_KEY"]: streamYamlFile.algoliaConfig?.writeApiKey ?? "",
["ALGOLIA_INDEX"]: streamYamlFile.algoliaConfig?.primaryIndex ?? "",
["SANITY_PROJECT_ID"]: streamYamlFile.sanityConfig?.projectId ?? "",
["SANITY_DATASET"]: streamYamlFile.sanityConfig?.dataset ?? "",
["SANITY_API_TOKEN"]: streamYamlFile.sanityConfig?.token ?? "",
["SANITY_REVALIDATE_SECRET"]:
streamYamlFile.sanityConfig?.revalidateSecret ?? "",
["SANITY_DATASET_TAG"]: streamYamlFile.sanityConfig?.tag ?? "",
["SANITY_PREVIEW_SECRET"]:
streamYamlFile.sanityConfig?.previewSecret ?? "",
["SANITY_API_VERSION"]: streamYamlFile.sanityConfig?.apiVersion ?? ""
}
});
SecretObj.push(expanded.parsed!);
return expanded.parsed!;
};
// consume augmented NodeJS.ProcessEnv interface defined in global.d.ts
type ENV = { [P in keyof NodeJS.ProcessEnv]?: NodeJS.ProcessEnv[P] };
interface ENVExtended extends ENV {}
type Config = { [P in keyof NodeJS.ProcessEnv]: NodeJS.ProcessEnv[P] };
interface ConfigExtended extends Config {}
// Loading process.env as ENV interface
const getConfig = (): ENVExtended => {
return {
ALGOLIA_APP_ID: process.env.ALGOLIA_APP_ID
? process.env.ALGOLIA_APP_ID
: undefined,
ALGOLIA_INDEX: process.env.ALGOLIA_INDEX
? process.env.ALGOLIA_INDEX
: undefined,
ALGOLIA_INDICES: process.env.ALGOLIA_INDICES
? process.env.ALGOLIA_INDICES
: undefined,
ALGOLIA_SEARCH_KEY: process.env.ALGOLIA_SEARCH_KEY
? process.env.ALGOLIA_SEARCH_KEY
: undefined,
ALGOLIA_WRITE_KEY: process.env.ALGOLIA_WRITE_KEY
? process.env.ALGOLIA_WRITE_KEY
: undefined,
SANITY_API_TOKEN: process.env.SANITY_API_TOKEN
? process.env.SANITY_API_TOKEN
: undefined,
SANITY_API_VERSION: process.env.SANITY_API_VERSION
? process.env.SANITY_API_VERSION
: undefined,
SANITY_DATASET: process.env.SANITY_DATASET
? process.env.SANITY_DATASET
: undefined,
SANITY_DATASET_TAG: process.env.SANITY_DATASET_TAG
? process.env.SANITY_DATASET_TAG
: undefined,
SANITY_PREVIEW_SECRET: process.env.SANITY_PREVIEW_SECRET
? process.env.SANITY_PREVIEW_SECRET
: undefined,
SANITY_PROJECT_ID: process.env.SANITY_PROJECT_ID
? process.env.SANITY_PROJECT_ID
: undefined,
SANITY_REVALIDATE_SECRET: process.env.SANITY_REVALIDATE_SECRET
? process.env.SANITY_REVALIDATE_SECRET
: undefined
};
};
const getSanitzedConfig = (config: ENVExtended): ConfigExtended => {
for (const [key, value] of Object.entries(config)) {
if (value === undefined) {
throw new Error(`Missing key ${key} in .env`)
}
}
return config as Config;
};
const config = getConfig();
export const {
ALGOLIA_APP_ID,
ALGOLIA_INDEX,
ALGOLIA_SEARCH_KEY,
ALGOLIA_WRITE_KEY,
ALGOLIA_INDICES,
SANITY_API_TOKEN,
SANITY_API_VERSION,
SANITY_DATASET,
SANITY_DATASET_TAG,
SANITY_PREVIEW_SECRET,
SANITY_PROJECT_ID,
SANITY_REVALIDATE_SECRET
} = getSanitzedConfig(config);
// yamlstream fallback method -- resolving how to make the two work in tandem
// export const {
// ALGOLIA_APP_ID = process.env.ALGOLIA_APP_ID,
// ALGOLIA_INDEX = process.env.ALGOLIA_INDEX,
// ALGOLIA_SEARCH_KEY = process.env.ALGOLIA_SEARCH_KEY,
// ALGOLIA_WRITE_KEY = process.env.ALGOLIA_WRITE_KEY,
// ALGOLIA_INDICES = process.env.ALGOLIA_INDICES,
// SANITY_API_TOKEN = process.env.SANITY_API_TOKEN,
// SANITY_API_VERSION = process.env.SANITY_API_VERSION,
// SANITY_DATASET = process.env.SANITY_DATASET,
// SANITY_DATASET_TAG = process.env.SANITY_DATASET_TAG,
// SANITY_PREVIEW_SECRET = process.env.SANITY_PREVIEW_SECRET,
// SANITY_PROJECT_ID = process.env.SANITY_PROJECT_ID,
// SANITY_REVALIDATE_SECRET = process.env.SANITY_REVALIDATE_SECRET
// } = YamlSecrets(streamYamlFile);Point being, you can make next play nicely and only use the cjs export from tsup + that declaration file triple-slash directive Here's my tsup.config.ts import { defineConfig } from "tsup";
const isProd = process.env.NODE_ENV === "production";
export default defineConfig({
clean: true,
dts: true,
entry: ["src/index.ts"],
format: ["cjs"],
minify: isProd,
tsconfig: "tsconfig.json",
splitting: false,
sourcemap: true
})tsconfig.json {
"extends": "@takedajobs/tsconfig/node16.json",
"include": ["**/*.ts", "tsup.config.ts", "global.d.ts", "takeda.config.yaml", "."],
"exclude": ["node_modules"]
}
the tsconfig it is extending {
"$schema": "https://json.schemastore.org/tsconfig",
"compilerOptions": {
"composite": false,
"declaration": true,
"declarationMap": true,
"inlineSources": false,
"isolatedModules": true,
"moduleResolution": "Node",
"noFallthroughCasesInSwitch": true,
"noStrictGenericChecks": false,
"noUnusedLocals": false,
"noUnusedParameters": false,
"pretty": true,
"preserveWatchOutput": true,
"noImplicitThis": true,
"strict": true,
"alwaysStrict": true,
"lib": ["ESNext", "DOM", "DOM.Iterable"],
"module": "CommonJS",
"target": "ES2021",
"outDir": "dist",
"esModuleInterop": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true
}
}package.json {
"name": "@takedajobs/indexer",
"version": "1.0.0",
"main": "./dist/index.js",
"module": "./dist/index.mjs",
"types": "./dist/index.d.ts",
"sideEffects": false,
"files": [
"dist/**"
],
"scripts": {
"prebuild": "yarn regenerate",
"build": "tsup",
"predev": "yarn regenerate",
"dev": "tsup --watch",
"lint": "TIMING=1 eslint ./src/**/*.ts* --fix",
"clean": "rm -rf .turbo && rm -rf node_modules && rm -rf dist",
"regenerate": "rm -rf dist"
},
"devDependencies": {
"@algolia/client-common": "^4.14.2",
"@takedajobs/tsconfig":"*",
"@types/js-yaml": "^4.0.5",
"@types/node": "^18.8.2",
"dotenv": "^16.0.3",
"dotenv-cli": "^6.0.0",
"dotenv-expand": "^9.0.0",
"eslint": "^8.24.0",
"eslint-config-custom": "*",
"ts-node": "^10.9.1",
"tslib": "^2.4.0",
"tsup": "^6.2.3",
"typescript": "^4.9.1-beta"
},
"publishConfig": {
"access": "public"
},
"dependencies": {
"@sanity/client": "^3.4.1",
"algoliasearch": "^4.14.2",
"chalk": "^5.1.0",
"js-yaml": "^4.1.0"
}
}
Lastly, the logs of a local build I just ran to show that the augmented .env values are output in nextjs with no failures when using this approach dopaminedriven@LAPTOP-2IH011V4:~/takeda/HCMS-TakedaJobs$ yarn turbo run build
yarn run v1.22.19
$ /home/dopaminedriven/takeda/HCMS-TakedaJobs/node_modules/.bin/turbo run build
• Packages in scope: @takedajobs/example, @takedajobs/indexer, @takedajobs/studio, @takedajobs/tsconfig, @takedajobs/ui, @takedajobs/utils, @takedajobs/vercel, eslint-config-custom
• Running build in 8 packages
INFO • Remote caching enabled
@takedajobs/indexer:build: cache hit, replaying output f02ab328073e34a1
@takedajobs/indexer:build: warning package.json: No license field
@takedajobs/indexer:build: $ yarn regenerate
@takedajobs/indexer:build: warning package.json: No license field
@takedajobs/indexer:build: $ rm -rf dist
@takedajobs/indexer:build: $ tsup
@takedajobs/indexer:build: CLI Building entry: src/index.ts
@takedajobs/indexer:build: CLI Using tsconfig: tsconfig.json
@takedajobs/indexer:build: CLI tsup v6.2.3
@takedajobs/indexer:build: CLI Using tsup config: /home/dopaminedriven/takeda/HCMS-TakedaJobs/packages/indexer/tsup.config.ts
@takedajobs/indexer:build: CLI Target: node14
@takedajobs/indexer:build: CLI Cleaning output folder
@takedajobs/indexer:build: CJS Build start
@takedajobs/indexer:build: CJS dist/index.js 12.25 KB
@takedajobs/indexer:build: CJS dist/index.js.map 34.08 KB
@takedajobs/indexer:build: CJS ⚡️ Build success in 33ms
@takedajobs/indexer:build: DTS Build start
@takedajobs/indexer:build: DTS ⚡️ Build success in 2618ms
@takedajobs/indexer:build: DTS dist/index.d.ts 8.20 KB
@takedajobs/studio:build: cache hit, replaying output 84ecf07ece8ad14f
@takedajobs/studio:build: $ sanity build dist -y
@takedajobs/studio:build: - Clearing output folder
@takedajobs/studio:build: ✔ Clearing output folder (6ms)
@takedajobs/studio:build: - Building Sanity
@takedajobs/studio:build: ✔ Building Sanity (29529ms)
@takedajobs/studio:build: - Building index document
@takedajobs/studio:build: ✔ Building index document (63ms)
@takedajobs/studio:build: - Minifying JavaScript bundles
@takedajobs/studio:build: ✔ Minifying JavaScript bundles (47120ms)
@takedajobs/vercel:build: cache miss, executing 533452ca23cd6578
@takedajobs/example:build: cache miss, executing fe1f1713b3acbae1
@takedajobs/vercel:build: $ next build
@takedajobs/example:build: $ next build
@takedajobs/vercel:build: info - Loaded env from /home/dopaminedriven/takeda/HCMS-TakedaJobs/apps/vercel/.env.local
@takedajobs/vercel:build: info - Loaded env from /home/dopaminedriven/takeda/HCMS-TakedaJobs/apps/vercel/.env
@takedajobs/example:build: info - Loaded env from /home/dopaminedriven/takeda/HCMS-TakedaJobs/apps/example/.env.local
@takedajobs/example:build: info - Loaded env from /home/dopaminedriven/takeda/HCMS-TakedaJobs/apps/example/.env
@takedajobs/example:build: warn - You have enabled experimental feature (fontLoaders) in next.config.js.
@takedajobs/example:build: warn - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use at your own risk.
@takedajobs/example:build:
@takedajobs/vercel:build: info - Linting and checking validity of types...
@takedajobs/example:build: info - Linting and checking validity of types...
@takedajobs/example:build: info - Creating an optimized production build...
@takedajobs/example:build: info - Using tsconfig file: ./tsconfig.json
@takedajobs/example:build: info - Compiled successfully
@takedajobs/example:build: info - Collecting page data...
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: info - Generating static pages (0/3)
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48d
@takedajobs/example:build: info - Generating static pages (3/3)
@takedajobs/example:build: info - Finalizing page optimization...
@takedajobs/example:build:
@takedajobs/example:build: Route (pages) Size First Load JS
@takedajobs/example:build: ┌ ○ / 2.48 kB 116 kB
@takedajobs/example:build: ├ └ css/9624720adf640a1b.css 1.22 kB
@takedajobs/example:build: ├ /_app 0 B 113 kB
@takedajobs/example:build: └ ○ /404 194 B 113 kB
@takedajobs/example:build: + First Load JS shared by all 117 kB
@takedajobs/example:build: ├ chunks/framework-7dc8a65f4a0cda33.js 45.2 kB
@takedajobs/example:build: ├ chunks/main-9800b5260862a44b.js 31.1 kB
@takedajobs/example:build: ├ chunks/pages/_app-7f3d0cbda28fef3c.js 36.1 kB
@takedajobs/example:build: ├ chunks/webpack-5752944655d749a0.js 840 B
@takedajobs/example:build: └ css/e766f52bde8757b7.css 3.32 kB
@takedajobs/example:build:
@takedajobs/example:build: ○ (Static) automatically rendered as static HTML (uses no initial props)
@takedajobs/example:build: The string being logged over and over again @takedajobs/example:build: 60f0a979f021a303eb3aa4bdddb84a3e764e785ebf2de79255f3cb0aea9fa679c442ae00c4afded7e805bedf210f3e036184d98859b5b46deb3e8ecc1a5c83dc698c91c94404b4ce2a2f64546e4734564821f3aa97997220b48dcorresponds to this code in _app.tsx of the nextjs example app import "../styles/index.css";
import type { AppProps } from "next/app";
import cn from "clsx";
import { inter } from "../fonts";
import { SANITY_PREVIEW_SECRET, SANITY_API_TOKEN } from "@takedajobs/indexer";
console.log(SANITY_API_TOKEN);
function MyApp({ pageProps, Component }: AppProps) {
return (
<div
className={cn(
inter.className,
"min-w-screen mx-auto min-h-screen w-fit antialiased"
)}>
<Component {...pageProps} />
</div>
);
}
export default MyApp;hopefully this helps, can't believe I just spent 2-3 hours tweaking this to work lol |
|
I confirm this issue is happening for any project using cjs only. Here is another reproductible issue : |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Verify canary release
Provide environment information
$ /code/stateof/stateofjs-next/node_modules/.bin/next info
/bin/sh: 1: pnpm: not found
Done in 1.15s.
What browser are you using? (if relevant)
No response
How are you deploying your application? (if relevant)
No response
Describe the Bug
I am building a library of components.
exportsfield like this:So, when importing "my-package-tsup/server", Next will correctly load the "dist/server/index.cjs" file. But when this CJS file require it's subdependency, using a
requirecall, it will try to pick the .js ESM export instead of sticking to the .cjs CJS export.Here is the package.json of the dependency:
To rephrase this:
If a CJS top level package requires one if it's dependency, the
exportsfield of the dependency does not seem to be respected. The ESM export will be loaded, instead of the CJS export.Expected Behavior
CommonJS dependencies should also use the CommonJS export of their own subdependencies.
To Reproduce
It's not the easiest to reproduce:
It will rely on Yalc, so to test you'll need to set it up in a Next app.
Then import
@vulcanjs/graphql/serverin your Next app and you'll end up with the dreadedhttps://nextjs.org/docs/messages/import-esm-externalserrorClone, install, and open the "with-tsup" page http://localhost:3000/with-tsup
I seem to be able to have this message:
Module not found: Package path . is not exported from package /code/npm-the-right-way/demo-next-app/node_modules/my-package-tsup-dependency (see exports field in /code/npm-the-right-way/demo-next-app/node_modules/my-package-tsup-dependency/package.json).It's different from the error I have in the other package but could be related, as if "exports" were not understood correctly.
This issue might related: #34956
The text was updated successfully, but these errors were encountered: