static highlighting

[pietroppeter]

there's no really need to add javascript to highlight stuff at runtime. I should be able to do it when rendering html.
there was some discussion today in IRC #science channel (which unfortunately is not logged).

[ghost]

Some options:

• https://github.com/alecthomas/chroma - a tool in Go, used by Hugo for syntax highlighting
• https://nim-lang.org/docs/highlite.html - pure Nim, but only supports a few languages, example of usage - https://github.com/nim-lang/Nim/blob/devel/lib/packages/docutils/rstgen.nim#L992 (from rstgen which is used for nim doc generation and on the nim forum)
• Use highlight.js itself - make some lightweight JS CLI app (maybe custom-built), will require Node

There are many other options of course

[Vindaar]

Just adding Pygments, because aside from the Python dependency it makes for a good alternative:

https://pygments.org/

Mini example

Wouldn't use it as a default to not have it as a dependency, but it might be nice as an alternative. I use it when exporting Org files to LaTeX and HTML. I like it, because of the large number styles it ships with and in my experience it just works.

I've never used it as a library though, so I don't know how easy it is to interface with from that point of view.

[ghost]

@Vindaar the chroma tool I linked actually uses pygments definitions under the hood - "Chroma is based heavily on Pygments, and includes translators for Pygments lexers and styles."

[Vindaar]

@Vindaar the chroma tool I linked actually uses pygments definitions under the hood - "Chroma is based heavily on Pygments, and includes translators for Pygments lexers and styles."

Oh, haha. I thought it was a pure Go solution.

[ghost]

@Vindaar the chroma tool I linked actually uses pygments definitions under the hood - "Chroma is based heavily on Pygments, and includes translators for Pygments lexers and styles."

Oh, haha. I thought it was a pure Go solution.

nonono, you're right in that it's a pure Go solution, but it uses existing Pygments definitions to not reinvent the wheel

[ghost]

A rough prototype of using Nim's highlite module together with highlight.js's own syntax highlighting styles to make static highlighting - https://gist.github.com/Yardanico/144caea455c43917a409a22b56f34c03
to run correctly don't forget to do:

nbDoc.context["no_default_highlight"] = true
nbDoc.partials["style"] = """
<link rel='stylesheet' href='https://cdn.jsdelivr.net/gh/pietroppeter/nimib/assets/atom-one-light.css'>
"""

after nbInit

[ghost]

@pietroppeter when you'll implement your idea with splitting the rendering backends I'll add this highlighting myself, or you can add it yourself earlier if you want :)

[ghost]

Fixed escaping for my code, now it can highlight itself (more or less) correctly:

[pietroppeter]

hey @Yardanico, thanks for this it looks very nice and exactly in the direction I wanted it to go! Also, I think the render refactoring is orthogonal to this change, I can do it later. If you want to open a PR I would be very happy and we can take it from there!

As for code structure, you could add a highlight module in src\nimib with and exportable highlightNim function that will be used in renderHtmlCodeBody exactly how you do now. As a further example to test you can use the Nim example in highlight.hs/demo.

[pietroppeter]

fixed by #14 (additional commits in #15). In the end the only thing that got lost from removing highlight.js is the highlighting the pragmas which is not straightforward to do with the pure nim highlite approach so the tradeoff is still very much on the positive side (I also do not currenlty have any pragmas in the example code). Thanks a lot @Yardanico!

[pietroppeter]

putting here for future reference work by @Yardanico reported in #science channel (not logged) of how one could improve nim highlite to somewhat parse pragmas:

diff --git a/lib/packages/docutils/highlite.nim b/lib/packages/docutils/highlite.nim
index 8c91e0a8e..de82719c4 100644
--- a/lib/packages/docutils/highlite.nim
+++ b/lib/packages/docutils/highlite.nim
@@ -339,7 +339,24 @@ proc nimNextToken(g: var GeneralTokenizer, keywords: openArray[string] = @[]) =
             g.state = g.kind
             break
           else: inc(pos)
-    of '(', ')', '[', ']', '{', '}', '`', ':', ',', ';':
+    of '{':
+      inc(pos)
+      # pragma opening?
+      if g.buf[pos] == '.':
+        inc(pos)
+        g.kind = gtDirective
+      else:
+        g.kind = gtPunctuation
+    of '.':
+      inc(pos)
+      # possible pragma closing
+      if g.buf[pos] == '}':
+        inc(pos)
+        g.kind = gtDirective
+      else:
+        g.kind = gtOperator
+        while g.buf[pos] in OpChars: inc(pos)
+    of '(', ')', '[', ']', '}', '`', ':', ',', ';':
       inc(pos)
       g.kind = gtPunctuation
     of '\0':

doesn't handle whitespace in pragmas ( like { .) nor pragmas without . at the end
if a pragma is defined as { . test . } it won't work
because highlite is really just a hand-written lexer for simple highlighting (it doesn't use regexes)