Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

transform.ToMath, WASI/Wasm modules feedback thread #12736

Closed
bep opened this issue Aug 9, 2024 · 8 comments
Closed

transform.ToMath, WASI/Wasm modules feedback thread #12736

bep opened this issue Aug 9, 2024 · 8 comments

Comments

@bep
Copy link
Member

bep commented Aug 9, 2024

Also see #12737.

transform.ToMath uses Katex to render. Katex is a JS library.

  • The WASM files are produced by Javy (a Shopify library). QuickJS is dynamically linked into the Katex WASM.
  • The WASI host is Wazero

Wazero's compiler is configured for:

  • AMD64, ARM64
  • Linux, Darwin (MacOS), Windows, FreeBSD.

If not in the list above, Wazero will fall back to an interpreter, which works on all platforms, but is much slower.

The goal of this "thread" is to collect feedback, as in: "it works on Solaris, but is very slow ..."

It is technically possible for Hugo to enable the compiler for other OSes, but we're currently limited by the test setup.

Also, the hours invested in setting this up just for KaTeX doesn't make much sense. This was a proof of concept. For further ideas of where this takes us, see #2737.

@bep bep added the Proposal label Aug 9, 2024
@bep bep added this to the v0.133.0 milestone Aug 9, 2024
@bep bep self-assigned this Aug 9, 2024
bep added a commit to bep/hugo that referenced this issue Aug 9, 2024
While very useful on its own (and combined with the passthrough render hooks), this is also serves as a proof of concept of using WASI ( WebAssembly System Interface) modules in Hugo.

This will be marked _experimental_ in the documentation. Not because it will be removed or change in a dramatic way, but we need to think a little more how to best set up/configure similar services, define where these WASM files gets stored, maybe we can allow user provided WASM files plugins via Hugo Modules mounts etc.

See these issues for more context:

* gohugoio#12736
* gohugoio#12737

See gohugoio#11927
bep added a commit to bep/hugo that referenced this issue Aug 9, 2024
While very useful on its own (and combined with the passthrough render hooks), this also serves as a proof of concept of using WASI (WebAssembly System Interface) modules in Hugo.

This will be marked _experimental_ in the documentation. Not because it will be removed or changed in a dramatic way, but we need to think a little more how to best set up/configure similar services, define where these WASM files gets stored, maybe we can allow user provided WASM files plugins via Hugo Modules mounts etc.

See these issues for more context:

* gohugoio#12736
* gohugoio#12737

See gohugoio#11927
bep added a commit to bep/hugo that referenced this issue Aug 9, 2024
While very useful on its own (and combined with the passthrough render hooks), this also serves as a proof of concept of using WASI (WebAssembly System Interface) modules in Hugo.

This will be marked _experimental_ in the documentation. Not because it will be removed or changed in a dramatic way, but we need to think a little more how to best set up/configure similar services, define where these WASM files gets stored, maybe we can allow user provided WASM files plugins via Hugo Modules mounts etc.

See these issues for more context:

* gohugoio#12736
* gohugoio#12737

See gohugoio#11927
bep added a commit to bep/hugo that referenced this issue Aug 9, 2024
While very useful on its own (and combined with the passthrough render hooks), this also serves as a proof of concept of using WASI (WebAssembly System Interface) modules in Hugo.

This will be marked _experimental_ in the documentation. Not because it will be removed or changed in a dramatic way, but we need to think a little more how to best set up/configure similar services, define where these WASM files gets stored, maybe we can allow user provided WASM files plugins via Hugo Modules mounts etc.

See these issues for more context:

* gohugoio#12736
* gohugoio#12737

See gohugoio#11927
bep added a commit to bep/hugo that referenced this issue Aug 9, 2024
While very useful on its own (and combined with the passthrough render hooks), this also serves as a proof of concept of using WASI (WebAssembly System Interface) modules in Hugo.

This will be marked _experimental_ in the documentation. Not because it will be removed or changed in a dramatic way, but we need to think a little more how to best set up/configure similar services, define where these WASM files gets stored, maybe we can allow user provided WASM files plugins via Hugo Modules mounts etc.

See these issues for more context:

* gohugoio#12736
* gohugoio#12737

See gohugoio#11927
bep added a commit to bep/hugo that referenced this issue Aug 9, 2024
While very useful on its own (and combined with the passthrough render hooks), this also serves as a proof of concept of using WASI (WebAssembly System Interface) modules in Hugo.

This will be marked _experimental_ in the documentation. Not because it will be removed or changed in a dramatic way, but we need to think a little more how to best set up/configure similar services, define where these WASM files gets stored, maybe we can allow user provided WASM files plugins via Hugo Modules mounts etc.

See these issues for more context:

* gohugoio#12736
* gohugoio#12737

See gohugoio#11927
bep added a commit to bep/hugo that referenced this issue Aug 9, 2024
While very useful on its own (and combined with the passthrough render hooks), this also serves as a proof of concept of using WASI (WebAssembly System Interface) modules in Hugo.

This will be marked _experimental_ in the documentation. Not because it will be removed or changed in a dramatic way, but we need to think a little more how to best set up/configure similar services, define where these WASM files gets stored, maybe we can allow user provided WASM files plugins via Hugo Modules mounts etc.

See these issues for more context:

* gohugoio#12736
* gohugoio#12737

See gohugoio#11927
bep added a commit to bep/hugo that referenced this issue Aug 9, 2024
While very useful on its own (and combined with the passthrough render hooks), this also serves as a proof of concept of using WASI (WebAssembly System Interface) modules in Hugo.

This will be marked _experimental_ in the documentation. Not because it will be removed or changed in a dramatic way, but we need to think a little more how to best set up/configure similar services, define where these WASM files gets stored, maybe we can allow user provided WASM files plugins via Hugo Modules mounts etc.

See these issues for more context:

* gohugoio#12736
* gohugoio#12737

See gohugoio#11927
bep added a commit that referenced this issue Aug 9, 2024
While very useful on its own (and combined with the passthrough render hooks), this also serves as a proof of concept of using WASI (WebAssembly System Interface) modules in Hugo.

This will be marked _experimental_ in the documentation. Not because it will be removed or changed in a dramatic way, but we need to think a little more how to best set up/configure similar services, define where these WASM files gets stored, maybe we can allow user provided WASM files plugins via Hugo Modules mounts etc.

See these issues for more context:

* #12736
* #12737

See #11927
@bep bep pinned this issue Aug 9, 2024
@bep bep changed the title convert.ToMath, WASI/WASM modules feedback thread convert.ToMath, WASI/Wasm modules feedback thread Aug 10, 2024
@gohugoio gohugoio deleted a comment from darix Aug 13, 2024
@gohugoio gohugoio deleted a comment from darix Aug 13, 2024
@ldeso
Copy link

ldeso commented Aug 16, 2024

Hi, thanks for adding this feature, it was quite straightforward to use it to switch to server-side math rendering in my theme.

Edit: the issue reported below was my mistake, I was calling the shortcode with {{% math %}} instead of {{< math >}}, causing Hugo to interpret the output of transform.ToMath as Markdown instead of HTML.

I made a math shortcode to let users call transform.ToMath from their markdown files:

layouts/shortcodes/math.html
{{ if not .IsNamedParams | and .Params }}
    {{ errorf "The %q shortcode requires named parameters. See https://gohugo.io/functions/transform/tomath/ for a list of options." .Name }}
{{ else }}
    {{ $opts := .Params | default nil }}
    {{- transform.ToMath .Inner $opts -}}
{{ end }}

I noticed that if I don't chomp whitespace before the call to transform.ToMath, Hugo will render a code block instead of a math expression. Furthermore, if I don't chomp spaces after transform.ToMath and I use the shortcode in the middle of a sentence, Hugo will insert an unwanted newline after an inline math expression.

Perhaps transform.ToMath could chomp spaces by default? otherwise, in my opinion it would be nice to document this behavior on the pages for transform.ToMath and Passthrough render hooks.

@taxilian
Copy link

taxilian commented Aug 27, 2024

I'm trying to use transform.ToMath with render-passthrough.html as documented in https://gohugo.io/render-hooks/passthrough/#example

I'm using the hugo-book theme and it's very possible it's somehow related to that, but I'm hitting errors that look like "executing "book-section-children" at <.Content>: error calling Content:" – if I comment out the place that happens it just happens somewhere else, any place where .Content is being called. As far as I can tell I set things up correctly – 

    [markup.goldmark.extensions]
      [markup.goldmark.extensions.passthrough]
        enable = true
        [markup.goldmark.extensions.passthrough.delimiters]
          block = [['\[', '\]'], ['$$', '$$']]
          inline = [['\(', '\)']]

render-passthrough.html:

{{ if eq .Type "block" }}
  {{ $opts := dict "displayMode" true }}
  {{ transform.ToMath .Inner $opts }}
{{ else }}
  {{ transform.ToMath .Inner }}
{{ end }}

Am I missing something that should be obvious? I'm still relatively new to hugo, but I was really hoping to get server side rendering of the katex for this site :-/

$ hugo version
hugo v0.133.1+extended darwin/arm64 BuildDate=2024-08-26T13:58:46Z VendorInfo=brew

@taxilian
Copy link

taxilian commented Aug 28, 2024

I was able to get it working by using this instead:

  {{ $opts := dict "displayMode" true }}
  {{ $math := transform.ToMath .Inner $opts }}
  {{ $math.Value | safeHTML }}
{{ else }}
  {{ $math := transform.ToMath .Inner }}
  {{ $math.Value | safeHTML }}
{{ end }}```

Is something not working right or are the docs wrong? Also even with displayMode true it renders a bit differently; it's missing a container. This fixed it:

```{{ if eq .Type "block" }}{{ $opts := dict "displayMode" true 
}}{{
  $math := transform.ToMath .Inner $opts }}<span class="katex-display">{{
  $math.Value | safeHTML
}}</span>{{ else }}{{
  $math := transform.ToMath .Inner }}{{
  $math.Value | safeHTML
}}{{ end }}```

@jmooring
Copy link
Member

jmooring commented Aug 28, 2024

@taxilian Please post the markdown you used for testing. I suspect you have double-escaped something, but we shouldn't be throwing an error. For example, with this markdown (note the erroneous double escape):

$$c = \\sqrt{a^2 + b^2}$$

Hugo throws this error:

execute of template failed: template: _default/home.html:11:5: executing "main" at <.Content>: error calling Content: "/home/jmooring/code/hugo-testing/content/_index.md:1:1": execute of template failed: template: _default/_markup/render-passthrough.html:1:0: executing "_default/_markup/render-passthrough.html" at <_html_template_htmlescaper>: error calling _html_template_htmlescaper: not a function

@taxilian
Copy link

I hadn't considered that it could be related to the latex itself, given that it works when using katex on the client side, in a github gist, and with my weird changes above.

With that information I was able to track it down to this line:

$$I = \frac{ 12 \text{V} }{ 4 \text{Ω} }$$

I missed that when fixing some markdown which came from chatgpt; it works if I change it to:

$$I = \frac{12\text{V}}{4\Omega}$$

I'm not sure what kind of bug to call this – it seems like if it fails to render then there should at least be a more useful error message?

@taxilian
Copy link

taxilian commented Aug 28, 2024

It's worth noting that I still have to add the <span class="katex-display"> around the block version for the stylesheets to work correctly – otherwise the spacing isn't great.

image

vs

image

@jmooring jmooring changed the title convert.ToMath, WASI/Wasm modules feedback thread transform.ToMath, WASI/Wasm modules feedback thread Aug 29, 2024
@bep bep modified the milestones: v0.133.0, Unscheduled Aug 29, 2024
@bep
Copy link
Member Author

bep commented Sep 6, 2024

OK, I didn't get get as much relevant feedback on this as I had hoped. I was hoping that adding this WASM/Wasi/JS feature could be a good proof of concept for further work in this area ... Putting those on ice for now.

@bep bep closed this as completed Sep 6, 2024
@bep bep unpinned this issue Sep 10, 2024
Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 28, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

4 participants