🐛 Inline expression parsing

[rowanc1]

Fixes a bug where the inline expressions were not getting picked up properly in the mystmd side when they were previously executed in JupyterLab-MyST.

[rowanc1]

The meta --> data happens in a transform but the inline expressions happen before that transform is run. I think that storing things in meta is a relic of how we used to parse based on strings, and I think we may be able to drop that part. @fwkoch can you confirm?

[fwkoch]

Yeah, I think this is fine to do here. meta -> data transform keeps anything that already exists in data, so putting things in data earlier is ok.

We can't totally drop the meta support yet, I don't think, because it's still used in myst-parser for keeping track of content on blockBreaks (which will be converted to blocks later) https://github.com/executablebooks/mystmd/blob/main/packages/myst-parser/src/tokensToMyst.ts#L468

But... we could probably do a JSON.parse there instead, e.g.

  myst_block_break: {
    type: 'blockBreak',
    noCloseToken: true,
    isLeaf: true,
    getAttrs(t) {
      return {
        data: JSON.parse(t.content) || undefined,
      };
    },

then we could maybe get rid of the meta transform entirely...?

There's still some other vestiges of meta we would need to clean up, e.g. https://github.com/executablebooks/mystmd/blob/main/packages/myst-to-md/src/misc.ts#L14 (but that one probably doesn't work anyway as is, since meta is deleted after transforming to data...)

[rowanc1]

Merging this in, @fwkoch interested in your thoughts on if we should remove the .meta = JSON.stringify() logic and tests now that we are no longer parsing by reading in text.