Nextjs import / webpack

[lucasoptml]

It (0.1.5) seems to fail with webpack 5 / next js 15.3.4.

I really have no idea why after diving into it a bit. Tested it on a vanilla install.

3x
[webpack.cache.PackFileCacheStrategy] Serializing big strings (114kiB) impacts deserialization performance (consider using Buffer instead and decode when needed)
Failed to compile.

./node_modules/@sparkjsdev/spark/dist/spark.module.js
Module not found: Invalid generator object. Asset Modules Plugin has been initialized using a generator object that does not match the API schema.

• generator has an unknown property 'filename'. These properties are valid:
  object { binary?, dataUrl? }
  -> Generator options for asset/inline modules.

[61cygni]

The first is a warning and shouldn't be an issue.

The second is the problem "invalid generator object". Can you post your webpack.config.js here? If you have a generator.filename property, you may want to try removing it.

[lucasoptml]

I'm using the vanilla create-next-app configuration, no extra rules.

[francusli]

I'm also seeing this error, did you find a way around it?

[lucasoptml]

I tried hammering it into shape by trying webpack build instead of vite, and was partially successful but no dice yet.

Here is the config and it seems to build:

(I had to rename a few .js file imports to no extension)
npm install -g webpack webpack-cli ts-loader glslify-loader raw-loader typescript

via webpack build --mode production

---

webpack.config.js

import path from 'path';
import { fileURLToPath } from 'url';

// Define __filename and __dirname for ES6 modules
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);

export default {
entry: './src/index.ts', // Your TypeScript entry point
module: {
rules: [
{
test: /.ts$/, // Match .ts files
use: 'ts-loader', // Use ts-loader to process them
exclude: /node_modules/, // Ignore node_modules
},
{
test: /.(glsl|vs|fs)$/,
use: ['raw-loader',
'glslify-loader']
}
],
},
resolve: {
extensions: ['.ts', '.js'], // Resolve .ts and .js files
},
output: {
filename: 'spark.bundle.js', // Output file name
path: path.resolve(__dirname, 'dist'), // Output directory
},
mode: 'production', // Build in production mode
};

---

The bundle js file is actually a lot smaller than the vite (half) though I'm not sure if this translates into any real savings in reality.

It generates a long hash.wasm file alongside spark.bundle.js

Ended up with this when it tried loading the wasm. :(
Error: WebAssembly.instantiate(): expected magic word 00 61 73 6d, found 3c 21 44 4f @+0

[lucasoptml]

Oh its trying to load the wasm file but it gets a 404, so maybe it's just the next js configuration now.

[lucasoptml]

asset/inline for wasm fixed that error, but I get other missing export issues now -- I don't know enough about webpack to resolve atm. Also, maybe there is just a vite fix possible? 🤔

[61cygni]

It should work fine with vite. That's what I use for everything (see for example: https://github.com/61cygni/lolofi)

Unfortunately I don't know Next or Webpack. But we'll see if we can find someone who does :)

[lucasoptml]

I actually tracked it down to wasm-pack rust, it builds a new URL( dataurl, filename) which seems to not be supported. So there's probably a wasm-pack configuration there that will fix it.

I tried the vite-plugin-wasm for webpack asyncwebassembly also but it would only apply to import file.wasm.

Workaround: I'm currently using next/script Script tag to manually import the js module file, but I can't use it long term because it won't do tree shaking of course. Next I'm looking at manually editing the js file to find the fix for now, I'll post the diff once I figure it out.

It seemed like the vite generated spark.module.js file was 200k smaller once I added vite-plugin-wasm (or something?), not sure why or if that broke things.

[61cygni]

Right. Yeah I've been digging into this as well. I guess this is a well know problem with wasm-pack. As you say, it generates glue code that's incompatible with next / webpack.

const blah = new URL('blah.wasm', import.meta.url);

Perhaps a short term fix is to have Spark generate the .wasm file in dist/ and then copy to public and just use the url:

const wasmUrl = '/blah.wasm';

Can you use turbopack? That should work.

[lucasoptml]

I'm all for that if it works.

I just tried manually fixing the wasm import by converting to arraybuffer using buffer.from(str, 'base64') on the wasm data and it seems to instantiate now, but I don't see anything in three appearing. =(

[lucasoptml]

Wait I think it works! I just couldn't place it under a group in three (using primitive in react three fiber)

Attaching the modified version for 0.1.5 , but it could probably be simpler/faster.
I'm wondering if the whole base64 to binary conversion could be skipped somehow and use straight binary.

spark.module.js.txt

[dmarcos]

@lucasoptml awesome. would you mind opening a PR with the fix?

[61cygni]

Excellent you got it working. I presume fixing this would require a change in wasm-pack. I'm going to see if spitting out a .wasm file in dist and then importanting that directly works. If that's the case, it'll be a band aid until either webpack can support this. Or wasm-pack is fixed.

[lucasoptml]

I just have the workaround by editing the spark.module.js file to change out the URL reference right now. If someone can solve the wasm-pack generated loader issue a fix could be a PR.