TypeError: Cannot read properties of undefined (reading 'wasm')

[ayushsaklani]

System Info

"@xenova/transformers": "^2.15.0",
"onnxruntime-node": "^1.14.0",
MACOS
ARCOS(Chromium)
❯ npm --version
10.2.4
❯ node --version
v20.11.0
Next.js

Environment/Platform

• Website/web-app
• Browser extension
• Server-side (e.g., Node.js, Deno, Bun)
• Desktop app (e.g., Electron)
• Other (e.g., VSCode extension)

Description

I am trying to use AutoTokenizer in my Nextjs app.
But getting below error. I have tried suggestion given in other issues but not sure what I am doing wrong.

⨯ node_modules/@xenova/transformers/src/env.js (60:9) @ wasm
 ⨯ TypeError: Cannot read properties of undefined (reading 'wasm')
    at eval (webpack-internal:///(rsc)/./node_modules/@xenova/transformers/src/env.js:52:10)
    at (rsc)/./node_modules/@xenova/transformers/src/env.js (/Users/shunya/Project/mastishk/.next/server/vendor-chunks/@xenova.js:40:1)
    at __webpack_require__ (/Users/shunya/Project/mastishk/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./node_modules/@xenova/transformers/src/utils/hub.js:10:65)
    at (rsc)/./node_modules/@xenova/transformers/src/utils/hub.js (/Users/shunya/Project/mastishk/.next/server/vendor-chunks/@xenova.js:140:1)
    at __webpack_require__ (/Users/shunya/Project/mastishk/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./node_modules/@xenova/transformers/src/tokenizers.js:49:71)
    at (rsc)/./node_modules/@xenova/transformers/src/tokenizers.js (/Users/shunya/Project/mastishk/.next/server/vendor-chunks/@xenova.js:80:1)
    at __webpack_require__ (/Users/shunya/Project/mastishk/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./node_modules/@xenova/transformers/src/pipelines.js:29:72)
    at (rsc)/./node_modules/@xenova/transformers/src/pipelines.js (/Users/shunya/Project/mastishk/.next/server/vendor-chunks/@xenova.js:60:1)
    at __webpack_require__ (/Users/shunya/Project/mastishk/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./node_modules/@xenova/transformers/src/transformers.js:459:71)
    at (rsc)/./node_modules/@xenova/transformers/src/transformers.js (/Users/shunya/Project/mastishk/.next/server/vendor-chunks/@xenova.js:90:1)
    at __webpack_require__ (/Users/shunya/Project/mastishk/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./app/api/chat/route.ts:10:78)
    at (rsc)/./app/api/chat/route.ts (/Users/shunya/Project/mastishk/.next/server/app/api/chat/route.js:163:1)
    at __webpack_require__ (/Users/shunya/Project/mastishk/.next/server/webpack-runtime.js:33:42)
    at eval (webpack-internal:///(rsc)/./node_modules/next/dist/build/webpack/loaders/next-app-loader.js?name=app%2Fapi%2Fchat%2Froute&page=%2Fapi%2Fchat%2Froute&appPaths=&pagePath=private-next-app-dir%2Fapi%2Fchat%2Froute.ts&appDir=%2FUsers%2Fshunya%2FProject%2Fmastishk%2Fapp&pageExtensions=tsx&pageExtensions=ts&pageExtensions=jsx&pageExtensions=js&rootDir=%2FUsers%2Fshunya%2FProject%2Fmastishk&isDev=true&tsconfigPath=tsconfig.json&basePath=&assetPrefix=&nextConfigOutput=&preferredRegion=&middlewareConfig=e30%3D!:17:110)
    at (rsc)/./node_modules/next/dist/build/webpack/loaders/next-app-loader.js?name=app%2Fapi%2Fchat%2Froute&page=%2Fapi%2Fchat%2Froute&appPaths=&pagePath=private-next-app-dir%2Fapi%2Fchat%2Froute.ts&appDir=%2FUsers%2Fshunya%2FProject%2Fmastishk%2Fapp&pageExtensions=tsx&pageExtensions=ts&pageExtensions=jsx&pageExtensions=js&rootDir=%2FUsers%2Fshunya%2FProject%2Fmastishk&isDev=true&tsconfigPath=tsconfig.json&basePath=&assetPrefix=&nextConfigOutput=&preferredRegion=&middlewareConfig=e30%3D! (/Users/shunya/Project/mastishk/.next/server/app/api/chat/route.js:152:1)
    at __webpack_require__ (/Users/shunya/Project/mastishk/.next/server/webpack-runtime.js:33:42)
    at __webpack_exec__ (/Users/shunya/Project/mastishk/.next/server/app/api/chat/route.js:173:39)
    at /Users/shunya/Project/mastishk/.next/server/app/api/chat/route.js:174:616
    at __webpack_require__.X (/Users/shunya/Project/mastishk/.next/server/webpack-runtime.js:163:21)
    at /Users/shunya/Project/mastishk/.next/server/app/api/chat/route.js:174:47
    at Object.<anonymous> (/Users/shunya/Project/mastishk/.next/server/app/api/chat/route.js:177:3)
    at Module._compile (node:internal/modules/cjs/loader:1376:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1435:10)
    at Module.load (node:internal/modules/cjs/loader:1207:32)
    at Module._load (node:internal/modules/cjs/loader:1023:12)
    at Module.require (node:internal/modules/cjs/loader:1235:19)
    at mod.require (/Users/shunya/Project/mastishk/node_modules/next/dist/server/require-hook.js:65:28)
    at require (node:internal/modules/helpers:176:18)
    at requirePage (/Users/shunya/Project/mastishk/node_modules/next/dist/server/require.js:109:84)
    at /Users/shunya/Project/mastishk/node_modules/next/dist/server/load-components.js:74:84
    at async loadComponentsImpl (/Users/shunya/Project/mastishk/node_modules/next/dist/server/load-components.js:74:26)
    at async DevServer.findPageComponentsImpl (/Users/shunya/Project/mastishk/node_modules/next/dist/server/next-server.js:675:36) {
  page: '/api/chat'
}
null

Below is my next.config.mjs

/** @type {import('next').NextConfig} */
const nextConfig = {
    experimental: {         
        serverComponentsExternalPackages: ['sharp', 'onnxruntime-node'],     
    },
    webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
        // config.resolve.fallback = { fs: false,
        //   "zlib": "browserify-zlib",
        //   "http":"stream-http",
        //   "https":false
        // };

        config.externals = [...config.externals, "hnswlib-node"];
        config.module.rules.push({
          test: /\.(pdf)$/,
          type: "asset/resource",
        });
        
        config.resolve.alias = {
          ...config.resolve.alias,
          "canvas":false,
          "sharp$": false,
          "onnxruntime-node$": false,
      };
        return config;
      },
};

export default nextConfig;

I am calling transformers on server side

import { AutoTokenizer } from '@xenova/transformers';
const tokenizer = await AutoTokenizer.from_pretrained("berkeley-nest/Starling-LM-7B-alpha");

Reproduction

1. create next.js app
2. install transformer js
3. call AutoTokenizer in any api route

[cjnoname]

We encounter the same issue when attempting to compile via esbuild.

[xenova]

Hi there - just to clarify, you are intentionally ignoring onnxruntime from both client-side and server-side, with the intention of only using the AutoTokenizer class? I suppose this is a valid use-case, and should be easy to resolve. I just want to make sure that you understand that you won't be able to use transformers.js for inference.

[ayushsaklani]

Yes. I only wanted to use tokenizer.apply_chat_template function. to pass to the langchain model object.

[xenova]

I see! That's a pretty reasonable feature request; I'll create a PR for this.

[xenova]

@ayushsaklani Can you test out #621? If it works for your use-case, I'll merge and make a release for it. Thanks!