Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Highlight functions and variables in :ncode blocks. #2746

Merged
merged 22 commits into from Feb 20, 2023

Conversation

aartaka
Copy link
Contributor

@aartaka aartaka commented Jan 19, 2023

Description

This adds a rudimentary syntax highlighting to all our :ncode blocks, which means: all code examples in manual and describe-* pages. This is a prototype, but a working one (see screenshots below).

highlight-simple
highlight-complex

Fixes #1480.

At this stage I'm interested in what all y'all have to say about this. Am I going the right way here? @aadcg, @jmercouris, @Ambrevar.

Discussion

  • I don't like that let-bindings are highlighted as functions, due to primitiveness of the code walker (resolve-linkable-symbols). We have to fix that. Anyone aware of some package/CL function that gets us to the body of the macro/special form, ignoring the arguments/bindings/special symbols? Swank should have something like that, but I can't find it. How does SLY do syntax highlighting then?
  • This change introduces a lot of compile-time computations. On my machine, this slow the compilation from 7 seconds to 20 seconds. I'm pretty sure we can optimize it, and I believe we have to. How can we speed this thing up? Maybe shift to regexps somewhere?

Checklist:

Everything in this checklist is required for each PR. Please do not approve a PR that does not have all of these items.

  • I have pulled from master before submitting this PR
  • There are no merge conflicts.
  • I've added the new dependencies as:
    • ASDF dependencies,
    • Git submodules,
      cd /path/to/nyxt/checkout
      git submodule add https://gitlab.common-lisp.net/nyxt/py-configparser _build/py-configparser
    • and Guix dependencies.
  • My code follows the style guidelines for Common Lisp code. See:
  • I have performed a self-review of my own code.
  • My code has been reviewed by at least one peer. (The peer review to approve a PR counts. The reviewer must download and test the code.)
  • Documentation:
    • All my code has docstrings and :documentations written in the aforementioned style. (It's OK to skip the docstring for really trivial parts.)
    • I have updated the existing documentation to match my changes.
    • I have commented my code in hard-to-understand areas.
    • I have updated the changelog.lisp with my changes if it's anything user-facing (new features, important bug fix, compatibility breakage).
    • I have added a migration.lisp entry for all compatibility-breaking changes.
    • (If this changes something about the features showcased on Nyxt website) I have these changes described in the new/existing article at Nyxt website or will notify one of maintainters to do so.
  • Compilation and tests:
    • My changes generate no new warnings.
    • I have added tests that prove my fix is effective or that my feature works. (If possible.)
    • New and existing unit tests pass locally with my changes.

@aartaka aartaka force-pushed the doc-syntax-highlight branch 2 times, most recently from 201d59a to 65b1ebc Compare January 19, 2023 13:20
@aartaka
Copy link
Contributor Author

aartaka commented Jan 19, 2023

Look! It highlights special forms now (progn needs more love, yes) \(^▽^@)ノ

special-form-highlight

@aartaka
Copy link
Contributor Author

aartaka commented Jan 20, 2023

Okay, now I'm fully satisfied with what's there:

  • Highlighting of:
    • Functions.
    • Variables.
    • Macros.
    • (and, separately, in accent color) CL special forms!
    • we might do something special with classes here, but let's first discuss what's there.
  • Links to relevant documentation.
  • Slow-down of complication from 7 seconds to 17 seconds (yes, I've sped it up some, but help won't hurt).

@Ambrevar
Copy link
Member

It's impressive work!

But before we get down to further discussion, the original issue #1480 mentioned https://prismjs.com/, which may already contain a CL highlighter.

There are also pure CL libraries like https://github.com/kingcons/colorize.

@aartaka
Copy link
Contributor Author

aartaka commented Jan 24, 2023

But before we get down to further discussion, the original issue #1480 mentioned https://prismjs.com/, which may already contain a CL highlighter.

Indeed, PrismJS has CL highlighting, but we agreed to not include any JS into Nyxt distribution unless strictly necessary. In our case, it's not strictly necessary at all, as compiler/language gives us enough leverage for even a more meaningful highlighting than PrismJS can ever give.

There are also pure CL libraries like https://github.com/kingcons/colorize.

Right, this one is good in being a Lisp library. But still, it's the same paradygm as PrismJS: it treats the code as a mere text, a mere thing to colorize based on pre-set rules and sets of symbols. It links ANSI CL symbols to CLHS, but no more linking/interactiveness beyond that.

This PR is different in that it inspects the image to infer the highlighting information. Importantly, it distinguishes functions, macros, and special forms from each other, and may thus highlight them differently. It also provides links to all the functions/macros used in the given code. This approach is quite demanding on compiler/distribution, but we're hard-set on making Nyxt a self-introspecting program anyway, so it should be more than relevant.

@Ambrevar
Copy link
Member

Ambrevar commented Jan 24, 2023

OK, thanks for investigating.

That said, aren't there any other library that do just that?
Actually what about Swank / Slynk? They do offer some symbol analysis.

@aadcg
Copy link
Member

aadcg commented Jan 24, 2023

Do we want to have links in code snippets?

@aartaka
Copy link
Contributor Author

aartaka commented Jan 24, 2023

Do we want to have links in code snippets?

Yes, because those allow to immediately look at the docs/sources of the function/macro/whatever used in the code.

And links are so nice and fidgety as code highlighting elements too :P

@aadcg
Copy link
Member

aadcg commented Jan 24, 2023

Do we want to have links in code snippets?

Yes, because those allow to immediately look at the docs/sources of the function/macro/whatever used in the code.

And links are so nice and fidgety as code highlighting elements too :P

I'm afraid I'm not convinced. I think visually it'd look too cluttered (syntax highlighting + link style).

@aartaka
Copy link
Contributor Author

aartaka commented Jan 24, 2023

I'm afraid I'm not convinced. I think visually it'd look too cluttered (syntax highlighting + link style).

Fair point. As a totally-not-a-fan-of-angry-fruit-salad, I concur. 348a1cf introduces a much more neutral highlight for functions and variables, making it much less cluttered (see the attached image). Should be much better now.
plain-highlight

@aartaka
Copy link
Contributor Author

aartaka commented Feb 16, 2023

So I've enabled code highlighting in describe-* pages and enabled class highlighting. Seems quite mergeable to me. Anyone willing to test?

@jmercouris
Copy link
Member

I can test if you ping me on email :-D otherwise i may forget!

@jmercouris
Copy link
Member

I did test, and it is fucking awesome! I did encounter a bug though :-O. Can't do try in repl. Any ideas?

@aartaka
Copy link
Contributor Author

aartaka commented Feb 20, 2023

Can't do try in repl. Any ideas?

Fixed on master and rebased here. Shoud be fixed now, it the error you're (not) referring to is the same as mine :)

@jmercouris
Copy link
Member

Well, I fixed the bug that I encountered :-)

@aartaka
Copy link
Contributor Author

aartaka commented Feb 20, 2023

Yep, I allowed myself a liberty to edit it some.

Need to refactor the whole macro before merging (is it OK to merge then?)

@jmercouris
Copy link
Member

It is OK :-)

@jmercouris jmercouris merged commit 4001cdf into master Feb 20, 2023
@jmercouris jmercouris deleted the doc-syntax-highlight branch February 20, 2023 22:05
@jmercouris
Copy link
Member

Feel free to refactor the macro on master, that is orthogonal to this PR, this PR is OK to merge, thus I merged :-D

@aadcg
Copy link
Member

aadcg commented Feb 21, 2023

@jmercouris, you didn't merge this according to our standards - it should have been merged with a merge commit. This is extremely important.

See the instructions
.

Also, the commits should probably have been squashed.

@jmercouris
Copy link
Member

Sorry Andre, I had totally forgotten about this. Old habits!

@aadcg
Copy link
Member

aadcg commented Feb 22, 2023

No big deal, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Add syntax highlighting for code examples
4 participants