Bug: custom shiki grammars don't work with workers

[jonatanklosko]

Prerequisites

• I have searched for duplicate or closed issues
• I have read the contributing guidelines

Describe the issue

Currently a custom grammar can be registered with registerCustomLanguage (added in #303). However, when using <WorkerPoolContextProvider>, the custom grammar is not respected. Interestingly, in the network inspector, I can see the grammar file actually being fetched.

Reproduction

Modify apps/docs/app/diff-examples/ArbitraryFiles/ArbitraryFiles.tsx (the last example on the landing page):

ArbitraryFiles.tsx

'use client';

import { MultiFileDiff } from '@pierre/diffs/react';
import type { PreloadMultiFileDiffResult } from '@pierre/diffs/ssr';
import { useState } from 'react';

import { FeatureHeader } from '../FeatureHeader';
import { registerCustomLanguage } from '@pierre/diffs';

registerCustomLanguage("mylang",
  () => {
    console.log("Loading mylang")
  
    return Promise.resolve({
      default: [
        {
          "name": "mylang",
          "scopeName": "source.mylang",
          "patterns": [
            {
              "name": "keyword",
              "match": "\\b(pizza)\\b"
            }
          ]
        },
      ],
    });
  },
  ["mylang"]
);

interface ArbitraryFilesProps {
  prerenderedDiff: PreloadMultiFileDiffResult<undefined>;
}

export function ArbitraryFiles({ prerenderedDiff }: ArbitraryFilesProps) {
  const [oldFile, setOldFile] = useState({ ...prerenderedDiff.oldFile, name: "example.mylang" });
  const [newFile, setNewFile] = useState({ ...prerenderedDiff.newFile, name: "example.mylang" });

  return (
    <div className="space-y-5">
      <FeatureHeader
        id="arbitrary"
        title="Diff arbitrary files"
        description={
          <>
            In addition to rendering standard Git diffs and patches, you can
            pass any two files in <code>@pierre/diffs</code> and get a diff
            between them. This is especially useful when comparing across
            generative snapshots where linear history isn't always available.
            Edit the css below to see the diff.
          </>
        }
      />

      <div className="grid grid-cols-1 gap-4 md:grid-cols-2">
        <div className="relative">
          <FileLabel>before.css</FileLabel>
          <FileTextarea
            value={oldFile.contents}
            onChange={(e) => {
              setOldFile({ ...oldFile, contents: e.target.value });
            }}
          />
        </div>
        <div className="relative">
          <FileLabel>after.css</FileLabel>
          <FileTextarea
            value={newFile.contents}
            onChange={(e) => {
              setNewFile({ ...newFile, contents: e.target.value });
            }}
          />
        </div>
      </div>
      <MultiFileDiff
        // No SSR
        // {...prerenderedDiff}
        oldFile={oldFile}
        newFile={newFile}
        className="diff-container"
        // It works without workers.
        // disableWorkerPool
      />
    </div>
  );
}

interface FileLabelProps {
  children: React.ReactNode;
}

// Local components to avoid class name duplication
function FileLabel({ children }: FileLabelProps) {
  return (
    <label className="text-muted-foreground bg-muted absolute top-[1px] left-[1px] block rounded-lg px-3 py-2 text-xs font-medium uppercase select-none">
      {children}
    </label>
  );
}

interface FileTextareaProps {
  value: string;
  onChange: (e: React.ChangeEvent<HTMLTextAreaElement>) => void;
  className?: string;
}

function FileTextarea({ value, onChange, className = '' }: FileTextareaProps) {
  return (
    <textarea
      value={value}
      onChange={onChange}
      className={`bg-muted h-40 w-full resize-none rounded-lg border px-4 pt-10 font-mono text-sm ${className}`}
      spellCheck={false}
    />
  );
}

pizza is not highlighted, however adding disableWorkerPool makes it highlighted correctly.

What browser(s) are you seeing the problem on?

No response

What version of @pierre/diffs are you using?

1.1.10

[amadeus]

Ahh I see the issue. Custom languages are sent through properly, but the issue is the extension matching isn't synced. If you specify a lang prop on your files or diffs, it'll work properly. I'll look at getting this rectified.

[amadeus]

Fixed in main, will look to probably do a release later today or tomorrow. Thanks again for the report!

[jonatanklosko]

Fantastic, thank you!!