Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Org-mode language support #2835

Open
wants to merge 58 commits into
base: master
from

Conversation

Projects
None yet
5 participants
@andreyorst
Copy link
Contributor

commented Apr 4, 2019

I think basic support of Org-mode should be included by default. This will provide syntax highlighting and indentation rules much like markdown.kak

Show resolved Hide resolved rc/filetype/org.kak Outdated
Show resolved Hide resolved rc/filetype/org.kak Outdated
Show resolved Hide resolved rc/filetype/org.kak Outdated
@Delapouite

This comment has been minimized.

Copy link
Contributor

commented Apr 4, 2019

org-mode is a behemoth of the emacs ecosystem. I'm afraid that by providing a half-assed version of the gazillions stuffs the original can do, it will open a constant flow of users complaining about "this feature is missing", "the behavior is slightly different" etc…

One of the obvious functionality missing from kakoune right now is folding support, which is omnipresent while working with org-mode.

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 4, 2019

org-mode is a behemoth of the emacs ecosystem. I'm afraid that by providing a half-assed version of the gazillions stuffs the original can do, it will open a constant flow of users complaining about "this feature is missing", "the behavior is slightly different" etc…

I think that Org-mode initial support is crucial, but I also think that if anyone wants to work with Org, he will juts use Emacs. I mean, If I would like to write some Vim documentation (with their markup) I would use Vim for that, but being available to read and do quick edits with Kakoune is still a good thing. I don't know any Vim user who uses Org-mode for real. But I know a lot of people who uses Emacs + Evil to write Org-mode.
Also org-mode plugins exist for other editors and they only provide syntax highlighting, and pretty much everyone is fine with that.

After using Emacs for Markdown, I can say that Kakoune lacks lots of markdown related features too, but I'm good with that.

One of the obvious functionality missing from Kakoune right now is folding support, which is omnipresent while working with org-mode.

so such mode would boost priority on this feature, I guess?

@Screwtapello

This comment has been minimized.

Copy link
Contributor

commented Apr 4, 2019

As a single data-point, I got interested in Org-mode when I found a neat TODO manager for Android that stores data in .org files. It would be nice to have syntax-highlighting if I edit the files in Kakoune, but I can't imagine it being worth my time to learn Emacs just to add milk to my shopping list.

@TeddyDD

This comment has been minimized.

Copy link
Contributor

commented Apr 4, 2019

Thanks for working on this, I'v got a lot of org files from time when I was using Emacs, it will be nice to have syntax highlighting for them

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 11, 2019

I've added indentation handling for lists, and by now this script covers pretty much everything that I use write Org-mode. @TeddyDD Can you test it on your files, to see if there are unhandled cases popping out?

@TeddyDD

This comment has been minimized.

Copy link
Contributor

commented Apr 11, 2019

All my files are looking okay

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 11, 2019

Thanks. However I've found some quirks with complex files, so I'll be fixing those.

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 11, 2019

I'm currently having troubles with defining a regexp for markup that will match this:

*text
... any amount of non-empty lines ...
text*

But not this:

*text
... any amount of non-empty lines ...

text*

Edit:
Seems like Org mode in emacs doesn't support this too, lol

@andreyorst andreyorst changed the title [WIP] Org-mode language support Org-mode language support Apr 14, 2019

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 14, 2019

I'm satisfied with provided highlighting and all test files look exacly the same between Emacs and Kakoune. This got pretty complex in various places, but I've followed Org Syntax Document and tried my best to provide optimal regular expressions for the syntax elements described in it.

face global section blue
face global subsection magenta
face global subsubsection red
face global bold red+b

This comment has been minimized.

Copy link
@mawww

mawww Apr 17, 2019

Owner

I do not like those 4 new faces (bold, itallic, underline and strikethrough), we already have attributes to express those (even though strikethrough is not supported pending replacing ncurses).

Regarding section/subsections/subsubsection, we already have header and title which are pretty close, I'd like to see if we can merge those.

This comment has been minimized.

Copy link
@andreyorst

andreyorst Apr 17, 2019

Author Contributor

But bold, italic, and underline are not new faces. I only added strikethrough because it was missing and it's good to have a face for it along with those even if it only can be expressed by color.

Title face is used to highlight the title, and I think that it is different from header (although I'd like to rename header to heading), because title can be only one per document, and there may be any amount of headings.

also, older conversation #2835 (comment)

This comment has been minimized.

Copy link
@mawww

mawww Apr 17, 2019

Owner

Oh, yeah, you are right, bold and italic already exist, however I think I'd rather see them go and just use default+b and default+i. What we have in the default colorscheme is the minimum set of faces a colorscheme must support, and I would like to keep that amount of colors in check.

I think having a single heading face shared by all section/subsection/subsubsection is probably enough, eventually we could use title for top level sections if we really need a different highlighting for those.

This comment has been minimized.

Copy link
@andreyorst

andreyorst Apr 18, 2019

Author Contributor

Having bold, italic, underline, and strikethrough faces is great when you don't have support for those faces in the terminal, and you can code those attributes with colors. Especially with strikethrough which isn't supported with ncurses.

In Org Mode, different heading levels represent nodes of a tree structure. Highlighting them in different colors makes semantic meaning in the Org language. And It helps organize TODOs.
Here's an example of agenda in Org mode, whith (sub){1,2}section and header highlighters using different faces:
image
Here's how in looks like when all headings use the same face:
image

Emacs screenshots:
image
Also concealing can hide bold, italics, and link markers to make it look more like text.

When folding will be implemented, I'll add it to this script in order to support this kind of thing:
image
Notice that headings in fzf.kak entry are hidden, as well as text in the four level section.

I think I can move those faces to the Org mode highlighter, and set them by default to default heading face. What do you think about it?

This comment has been minimized.

Copy link
@andreyorst

andreyorst Apr 18, 2019

Author Contributor

eventually we could use title for top level sections if we really need a different highlighting for those.

title has another semantic meaning in Org mode.
Emacs:
image

Kakoune:
image

Again, I can set all these faces only for Org within the script.

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 18, 2019

What we have in the default colorscheme is the minimum set of faces a colorscheme must support, and I would like to keep that amount of colors in check.

@mawww I've thought about it, and opened an issue that I think can solve this problem and will also allow scripts to be more unique in terms of faces: #2862

This also means that this script could be reworked to provide faces that are needed by Org mode, but not needed by another scripts. And with lazy loading model proposed this will not cost too much.

@mawww

This comment has been minimized.

Copy link
Owner

commented Apr 25, 2019

@andreyorst I think that is my preferred way for faces: have a very general and rather small set of faces provided as default (in the default color scheme) and expected to be provided by every colorscheme. Then let various scripts add their own faces if they do need them with default values based off the default color scheme (while encouraging re-use of the default ones when possible). Other colorschemes can then decide to add support for those faces as well.

4e24ba8 should help with that, as it makes it possible for scripts to base faces off other faces with slight changes (additional attributes, changing only fg or bg color...)

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 25, 2019

@andreyorst I think that is my preferred way for faces: have a very general and rather small set of faces provided as default (in the default color scheme) and expected to be provided by every colorscheme. Then let various scripts add their own faces if they do need them with default values based off the default color scheme (while encouraging re-use of the default ones when possible). Other colorschemes can then decide to add support for those faces as well.

4e24ba8 should help with that, as it makes it possible for scripts to base faces off other faces with slight changes (additional attributes, changing only fg or bg color...)

This is great change but I see the major problem here. Faces are faces, not colors. If I need to display something in red, that matches current colorscheme, I can't use face, because in another color scheme this face may not be red. What I mean is that red in tomorrow-night is #cc6666, and red in solarized-dark is #dc322f and if I want something to be red I need the color code itself. I could use Error face as a base, but Error faces (which may actually be any color) in these color schemes are different:

face global Error ${red},default+b # solarized dark
face global Error white,${red} # tomorrow night 

That's why I think that colorschemes should also provide colors and not only faces.

@mawww

This comment has been minimized.

Copy link
Owner

commented Apr 25, 2019

That would be quite a big change, currently colors are fully immutable, the named ones are just a way to access the terminal provided colors.

I can see the additional flexibility of having named colors as well as named faces, but at the same time, it seems they are very close in purpose, and having a single unified way to specify a colorscheme would be better.

Basically it seems the question boils down to whether we want a colorscheme just to provide a palette of colors, or a mapping from semantic names to faces. I would prefer to find a solution that does not requires to define both.

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 25, 2019

Currently most of color schemes define colors as shell variables. Why not define them as options instead? It will be a set of main 16 colors, and some colors for fg bg and cursors defined as options. Not really a big change, since the only thing that really changes is addition of several options. Like here: https://github.com/andreyorst/powerline.kak/blob/master/rc/themes/github.kak

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 25, 2019

@mawww I understand that this is kind of duplication of the same thing, but it allows granular control over the colors, and currently some themes and scripts fallback to terminal colors, but some users like me use different colorscheme in Kakoune and terminal, and this causes bad colors like this:

image

I use light colors cheme in terminal but dark inside kakoune, and this is the result of using plain colors.

@mawww

This comment has been minimized.

Copy link
Owner

commented Apr 25, 2019

named colors currently refer to the terminal colors, making them an option seems confusing to me... Maybe letting user tweak their terminal color palette would be a simpler and better way to solve that (and having more colorscheme using the terminal colors instead of rgb colors).

Faces are pairs of colors, I am not sure we can guarantee that every pair are going to be legible, so I am not even sure specifying colors individually is going to be that useful. It might be better to be able to express more things directly on faces, such as "Error face with brighter fg" or "Error face with dimmer fg"...

Lets get back at what we want (as I understand it):

  1. We want a way to add new faces in scripts for use cases where no default faces are good enough.
  2. We want those faces to be settable by some color scheme if their authors decide to add support for this script
  3. We want default values for those new faces not to look out of place in colorschemes that did not add explicit support for them.

We already handle 1. and 2., what we are trying to solve is 3.

Having a color palette, possibly handled through options could help, but could be tricky as we need to guarantee that whatever colors are chosen as default do look reasonably readable (fg against bg, both of those possibly implicitely defined by the colorscheme, although we should be able to assume any color scheme provided fg color is readable on its default bg).

Another option is to take a known to exist face, and add tweak it slightly, it should be harder to end-up with something unreadable, but there is not always a face semantically close to what we want, and sometimes we really want a specific color (diff highlighting for example).

None of those is really satisfying.

@Screwtapello

This comment has been minimized.

Copy link
Contributor

commented Apr 25, 2019

One issue with things like "powerline" (I haven't looked at @andreyorst's plugin specifically, but I've seen powerline plugins for Vim and bash) is that it tries to have overlapping blocks. For example, if you have a white-on-green block next to a white-on-blue block, there's a special transition character between them that needs to be drawn in green-on-blue. If a theme provides a white-on-green face and a white-on-blue face, there's no way for powerline.kak to combine them to produce the green-on-blue face it needs.

I think @andreyorst has worked around this by defining options for all the individual foreground and background colours so that he can synthesise whatever combinations he needs, but perhaps that's the kind of thing that should have dedicated syntax. For example, maybe something like SomeFace.fg to refer to the foreground colour of the SomeFace face:

set-face global InfoJoinMenu Info.bg,MenuBackground.bg
@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 25, 2019

The main problem that I see in current implementation is when I need a face which has for example purple (magenta) foreground and black background, that will stay purple/black across different color schemes, but will respect purple and black colors defined by color scheme. How can I achieve it with current approach?

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 25, 2019

One issue with things like "powerline" (I haven't looked at @andreyorst's plugin specifically, but I've seen powerline plugins for Vim and bash) is that it tries to have overlapping blocks. For example, if you have a white-on-green block next to a white-on-blue block, there's a special transition character between them that needs to be drawn in green-on-blue. If a theme provides a white-on-green face and a white-on-blue face, there's no way for powerline.kak to combine them to produce the green-on-blue face it needs.

Correct. To workaround this in my powerline plugin, I declare colors and use them to dynamically create faces.

I think @andreyorst has worked around this by defining options for all the individual foreground and background colours so that he can synthesise whatever combinations he needs, but perhaps that's the kind of thing that should have dedicated syntax. For example, maybe something like SomeFace.fg to refer to the foreground colour of the SomeFace face

This would be great but for powerline I need colors, not faces. I try to avoid separate block faces, because it will make harder to create new blocks, as they will also require a new face.

Aslo extending your syntax:

set-face global InfoJoinMenu Info.bg,MenuBackground.bg,Error.attributes

Maybe this discussion should continue in #2862?

@Screwtapello

This comment has been minimized.

Copy link
Contributor

commented Apr 25, 2019

I don't think it makes sense to say both "I want a face with purple foreground and black background" and "I want a face that looks natural with the user's current colour scheme". If the user's current colour scheme is green, orange and white, purple and black will never look right, no matter what RGB values you happen to pick.

As another example, Solarized includes a green colour, but not a bright-green. If you ask for bright-green and the user is using Solarized, what would you expect to get? If you get ordinary green, then bright-green-on-green is invisible for Solarized users. If you get the colour that replaces bright-green in the Solarized terminal palette, that's actually a pretty dim background colour, so if you wanted something contrasting against a black background, that won't work.

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented Apr 25, 2019

I don't think it makes sense to say both "I want a face with purple foreground and black background" and "I want a face that looks natural with the user's current colour scheme".

I didn't mean natural. I meant consistent. To me this is far less consistent:
image

(black text is actually green)

@andreyorst

This comment has been minimized.

Copy link
Contributor Author

commented May 24, 2019

@mawww Now, when pretty much most of the languages provide modules do I need to require them all when opening org file in order to highlight source blocks? It seems that without requiring a module highlighting doesn't happen. It affects markdown.kak too:

markdown.kak:
image

org.kak:
image

If module is required highlighting is visible:
image
image

Both scripts define those highlighters for all languages at once.
markdown.kak:

evaluate-commands %sh{
languages="
c cabal clojure coffee cpp css cucumber d diff dockerfile fish gas go
haml haskell html ini java javascript json julia kak kickstart latex
lisp lua makefile markdown moon objc perl pug python ragel ruby rust
sass scala scss sh swift toml tupfile typescript yaml sql
"
for lang in ${languages}; do
printf 'add-highlighter shared/markdown/%s region -match-capture ^(\h*)```\h*%s\\b ^(\h*)``` regions\n' "${lang}" "${lang}"
printf 'add-highlighter shared/markdown/%s/ default-region fill meta\n' "${lang}"
[ "${lang}" = kak ] && ref=kakrc || ref="${lang}"
printf 'add-highlighter shared/markdown/%s/inner region \A```[^\\n]*\K (?=```) ref %s\n' "${lang}" "${ref}"
done
}

org.kak

But I feel that requiring all modules kinda violates lazy loading model.

A discussion about this problem: #2924

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.