CloudFlare workers support

[Tahul]

Hello @octref 😄

Thanks for this amazing project!

We are trying to use it as a syntax highlighter on the Docus project at NuxtLabs, which will most likely be ported as @nuxt/content v2 at some point.

We are using latest updates from Nuxt (Nuxt 3 beta and Nuxt Bridge) in that project, as this is made to be deployed on a CloudFlare worker environment.

While using Shiki, we encountered an issue in which it fallbacks to Node loader as it does not detect CloudFlare environment.

Here is the line that seem to be in cause:
https://github.com/shikijs/shiki/blob/main/packages/shiki/src/loader.ts#L49

Is CloudFlare Workers support planned for Shiki?

If so, maybe we could help on a PR supporting that use case!

cc: @farnabaz

[farnabaz]

I manage to patch both shiki and onigasm and create rollup plugin for Nitro. But still there is an issue with WebAssembly memory.

Everything works fine with miniflare in simulated environment. But raise memory error in CF environment 🤒
You can test it here: https://github.com/farnabaz/nuxt-shiki/tree/feat/cf-worker-detection

• Clone the branch
• Run yarn build
• Run yarn start
• Go to http://localhost:8787/api/test

Here is some info about patches:

• I used WebSocketPair class to detect whether code is running in CF environment or not
• Cloudflare does not support utf-16l text encoding

[WalrusSoup]

@farnabaz any progress since you started?

I recompiled the WASM with -s TEXTDECODER=0 \ but basically ran into a brick wall after that.

[octref]

Hi @Tahul and @farnabaz, I take a quick look of CF workers, but I'm not super familiar with them.
Can you explain in more details how does your app with CF workers and shiki work? Also, how do you detect if a script is running in CF worker environment?

[Menci]

I think we could add a method for the user to load WebAssembly module manually. Then CF Worker users could upload WASM file and load the module by new WebAssembly.Instance(UPLOADED_WASM_MODULE, {})

[antfu]

Made a fork rewrite https://github.com/antfu/shikiji that enables the CF support. Also opened proposal #510 to have them back to Shiki. Feedback appreciated ;)

[adamdehaven]

@antfu now that things are merged back to Shiki, is there any extra steps (e.g. creating a CF worker) needed to utilize Shiki highlighting via @nuxtjs/mdc when deploying to Cloudflare pages?

Everything works fine for me locally, but Shiki doesn't init for me when the page is deployed.

[adamdehaven]

Here's the open issue and linked reproduction: nuxt-modules/mdc#159