Allow loading and converting files to mdx (for use by mdx loader)

[rgbkrk]

Hey all!

I'd love to be able to convert .ipynb files (Jupyter Notebook) to .mdx files within my docs directory. I was hoping I could do something as simple as including the ipynb files and setting up a webpack loader for them.

Include .ipynb files in docs ✅

                docs: {
                    sidebarPath: require.resolve("./sidebars.js"),
                    // TODO: Change this link to point to the repo's inner docs contents
                    editUrl:
                        "https://github.com/rgbkrk/chatlab/tree/main/website",
                    include: ["**/*.md", "**/*.mdx", "**/*.ipynb"],
                },

Create a webpack loader and allow mdx-loader to do the rest 🫤

    plugins: [
        function ipynbLoader(context, options) {
            return {
                name: "ipynb-docusaurus-plugin",
                configureWebpack(config, isServer, utils) {
                    const { getJSLoader } = utils;

                    return {
                        module: {
                            rules: [
                                {
                                    test: /\.ipynb$/,
                                    use: [
                                        getJSLoader({ isServer }),
                                        {
                                            loader: "@docusaurus/mdx-loader",
                                            // Had to pass this in so the mdx loader would kick off
                                            // Ideally, I would use the configured mdx-loader as is
                                            options: {
                                                markdownConfig: {},
                                            },
                                        },
                                        "./src/loaders/ipynb-to-mdx-loader",
                                    ],
                                },
                            ],
                        },
                    };
                },
            };
        },
    ],

This at least gets me up to this runtime error:

Cannot read properties of undefined (reading 'unversionedId')
TypeError: Cannot read properties of undefined (reading 'unversionedId')
    at DocItem (webpack-internal:///./node_modules/@docusaurus/theme-classic/lib/theme/DocItem/index.js:15:89)
    at renderWithHooks (webpack-internal:///./node_modules/react-dom/cjs/react-dom.development.js:14985:18)
    at mountIndeterminateComponent (webpack-internal:///./node_modules/react-dom/cjs/react-dom.development.js:17811:13)
    at beginWork (webpack-internal:///./node_modules/react-dom/cjs/react-dom.development.js:19049:16)
    at HTMLUnknownElement.callCallback (webpack-internal:///./node_modules/react-dom/cjs/react-dom.development.js:3945:14)
    at Object.invokeGuardedCallbackDev (webpack-internal:///./node_modules/react-dom/cjs/react-dom.development.js:3994:16)
    at invokeGuardedCallback (webpack-internal:///./node_modules/react-dom/cjs/react-dom.development.js:4056:31)
    at beginWork$1 (webpack-internal:///./node_modules/react-dom/cjs/react-dom.development.js:23959:7)
    at performUnitOfWork (webpack-internal:///./node_modules/react-dom/cjs/react-dom.development.js:22771:12)
    at workLoopSync (webpack-internal:///./node_modules/react-dom/cjs/react-dom.development.js:22702:5)

I guess the question is... How can I get to simply loading a file that gets converted to a raw mdx string and allows the currently configured mdx setup to render it?

[slorber]

The docs include is not really a way to support another extension. Doing what you want probably requires to fork/patch our docs plugin atm, I'm not sure it's possible/easy to do in userland.

You could try adding real MDX files to the docs folder, that would import the ipynb files? The limitation is that your ipynb headings would not appear in the table of contents

[rgbkrk]

Pardon me for responding back so late on this. I didn't realize I wouldn't see notifications.

I'm happy to fork and submit a PR for the docs plugin to accept notebooks, at least to get a discussion going. I've been modifying docusaurus to "fake" notebook inputs and outputs. See https://pre.chatlab.dev/docs/intro for an example. I think there's a lot of power here in having notebooks that can be used as a reproducible computational artifact that also exists within docs.