Theme syntax

[Lokaltog]

With the recent changes the suggested theme syntax in Lokaltog/vim-powerline#193 won't work as expected anymore. This is caused partially by the getwinvar/setwinvar workarounds (data has to be cached in a window dict), and by the possibility of having dynamic segments that are evaluated and able to change properties like colors and attributes for each statusline redraw.

An example of this dynamic behavior is the gradient cursor percentage segment from the vim example. Another example is the dynamic removal of the divider between the filename and the modified indicator if the modified indicator isn't empty.

If we decide to use JSON as the theme syntax, it may not be possible at all to have this kind of dynamic evaluation. The proposed JSON syntax will only allow passing arguments to the segment functions, but not actually change segment properties based on the current state of the window/buffer.

@ZyX-I What do you think we should do about this? Should we define themes in Python to allow this dynamic behavior, or should we stick to JSON? Do you have any suggestions to how we could use JSON and still have the possibility of dynamically changing segments?

[ZyX-I]

Changing segment color should be perfectly controlled by segment function (I am still up to that config, though now with functions instead of Segment subclasses): function should return {'text': "text", 'hl': 'Powerline…'} ('hl' is optional, colorscheme data is used as described in case they are absent, hl group name is obtained by special API only) or None (skip segment), colorscheme must additionally allow lists and dictionaries for colors. Dividers are different and less complicated case, I would have implemented this (keeping flat structure) with "draw_divider": false on both and special segment which only draws divider, it still fits perfectly into JSON structure.

Thus only colorschemes get more complicated and perhaps are to be changed to pure python. For the latter there should be a quiz, I don’t know how many users write their own colorschemes and (as personally never wrote a single one) think that having JSON for themes is needed more.

[Lokaltog]

I guess the segment functions themselves could return the highlighting information, but this doesn't solve the divider issue. In the case of the removal of the filename/modified divider one segment loses the divider based on what another segment contains. I can imagine other cases where this could happen. How would the segment functions know what other segments contain?

[ZyX-I]

I don’t get why in this case anybody needs to look at other segment contents. You just have no divider after both segments and a segment with only the divider.

[ZyX-I]

I do not think segments looking at other segments output is a very good idea, it is too easy to make theme logic hard to understand. And unlike changing colors you have not yet provided a use-case for this (for different colors even I have a use-case: my aurum:status segment needed this to make user able to distinguish better between file statuses).

[Lokaltog]

Yeah, you're probably right. It seems like a reasonable limitation not being able to look at other segment's contents. And the solution with removing the divider from the two segments and then adding an empty divider segment after those could work fine, so all my current use cases has been covered well by your suggestions. Thanks for your feedback!

[Lokaltog]

I've played around a bit with the theme syntax, and I have a draft that I'd like some feedback on. I've taken into consideration all your comments, see this gist for the default theme implemented with this syntax.

Because the segment functions are going to return the desired highlighting groups (as you can see in this example I'm using a prioritized list of fallback groups), I added a config option for setting the highlighting group for static string segments and filler segments since these segments don't have any functions that can return this information.

Since most segments will be enabled in all modes except maybe non-current windows I felt it was better to have an exclude_modes option. We could add an include_modes option that only includes the specified modes as well.

I've also grouped the segments by left/right as you suggested, it looks much cleaner than having the side specified for each segment.

[ZyX-I]

Since most segments will be enabled in all modes except maybe non-current windows I felt it was better to have an exclude_modes option. We could add an include_modes option that only includes the specified modes as well.

It is fine, but nc is not very descriptive (by the way, why do you need to put it in a list?). I would suggest just the same as for colorscheme: ["command", "normal"]. Remember, we are supposed to have modes for all renderers (I do not know tmux statusline well to determine whether it can distinguish between normal and copy mode but, AFAIR, there is a way to distinguish between at least “insert” and “normal” modes in zsh with vi bindings). Single characters does not look fine for either as only vim uses them, zsh and tmux tend to use names.

What does "highlight": ["line_current_symbol", "line_current"] mean? Apply both highlightings (i.e. take attributes absent in the first one from the second one)? Apply first highlighting found in the colorscheme? Apply first to LN (first symbol) and second to a space (which is second one)? Something else?

"type": "function": it should be default in order to be able to be skipped to reduce typing and number of non-informative lines. Or, if we allow segments computed only once (depends on segment defining API), it should be absent at all and string one will look like {"name": "string", "args": ["⭡ "], "highlight": "line_current_symbol"}.

PS: After rereading comment I now guess nc is non-current and thus is fine to be but on the list. Still not the best identifier.

[Lokaltog]

You bring up some valid points. I didn't plan on supporting modes for other applications, but it's a great idea that we should support. More descriptive mode names can be used, I just used the letters that vim's mode() function returns and "nc" as noncurrent. Maybe we could have a standard set of modes that can be translated to each application's own mode in the theme parser.

The highlighting list basically means that the plugin should first look for the first highlighting group and if it's not defined it should fallback to the next one. I think we should have this functionality so colorscheme authors don't have to support every single plugin and mode but maybe rather a set of core highlighting groups. Kind of how vim solves highlighting now, i.e. a syntax group can "inherit" a parent syntax group and vim will fallback to the first one that is defined in the colorscheme, if you understand what I mean. We could implement it differently if you have any better suggestions.

I agree that we should have a set of theme defaults so we don't have to type the same stuff for each segment. I think this can be decided later, and the function type would be one of the defaults.

How about the rest of the theme example, do you see any stuff that is missing or could be improved further?

[ZyX-I]

The highlighting list basically means that the plugin should first look for the first highlighting group and if it's not defined it should fallback to the next one. I think we should have this functionality so colorscheme authors don't have to support every single plugin and mode but maybe rather a set of core highlighting groups. Kind of how vim solves highlighting now, i.e. a syntax group can "inherit" a parent syntax group and vim will fallback to the first one that is defined in the colorscheme, if you understand what I mean. We could implement it differently if you have any better suggestions.

No better suggestions, I just failed to understand the meaning of the list.

How about the rest of the theme example, do you see any stuff that is missing or could be improved further?

Maybe it makes sense to move branch symbol to "before" key ("after" and "before" suggested earlier are prepended/appended to the returned string of the related segment if it has something to display) instead of forcing branch segment to support this on its own.

[Lokaltog]

I don't have a particular preference regarding the before/after key or the function argument way of doing it, and it does seem a bit cleaner to not return the icon along with the segments. I've updated the gist with the latest changes.

Edit: What should we do with the segments that only return a text string (e.g. the paste/readonly/modified indicators)? I don't think it makes sense to use the before/after config options for those segments.

[ZyX-I]

Before/after idea is similar: additional characters are displayed if and only if it one of the keys is present and text is not empty (or false, or not None, depending on what value segment should have in order to be removed. Currently it seems that either not empty or false are to be used, in python I prefer to write the latter, in vimscript the None or false* variants). This is universal and will work for paste/readonly/modified indicators, but only if anybody is willing to add them to the theme. Being absent they affect nothing and do not cause any problems. In any case you have to check falseness of the returned value to determine whether dividers should be drawn.

* Checking value of empty() vimL function looks like much checking object for falseness.

By the way, what is the reason in duplicating file name in the file contents?

[derekbrokeit]

Out of curiosity, is there a reason you prefer to use json over less verbose (character hungry) syntaxes like yaml? As for the difference between pure-python and json, I think that it would be best to have themes converted from this simple text file to a class Theme, which would inevitably allow more dynamic/creative behavior if you wanted to create a pure-python theme as well. Essentially, the json-version would be a restricted version of a pure-python Theme allowing both implementations if possible.

Looking at your sample JSON file, it is not immediately clear what governs the highlighting or coloring of each segment except a few that have explicit highlighting. Is it expected that the function returns the color? If so, should these functions be able to tell what format the color needs to be in (e.g. Vim-highlight, ANSI, etc.)? I expect we will need to have separate theme files for different applications (tmux, vim, ...), but from my understanding, the functions are written in python and should be as purpose-agnostic as possible.

Another thought, I think the theme file should also be capable of specifying the default separator icons ">","<", etc. For example, having a different separator for tmux and vim.

[Lokaltog]

While YAML is a nice format there are a couple of drawbacks, the main problem in my opinion is that we would introduce a dependency to the project, since Python doesn't have any YAML parsers in its core library. JSON translates well into Python and vimscript dicts, and it's not picky about indentation either. I'm not saying it's problem-free, but I agree that JSON is a good choice for the config files because of its simplicity and portability.

I'll ask you the same thing as ZyX-I asked me when I suggested a pure-Python approach: Do you have any specific examples where this would be a huge benefit?

The theme file is only a specification of which segments are going to be included, their modes, and arguments like whether the divider should be drawn around it. The segments are Python functions which I think are going to return a dict with the contents and the suggested highlighting group, like this:

{ 'contents': file_path, 'highlight': ['file_path'] }

The colorscheme would then define a file_path highlighting group for one or more modes, and the core lib would use the colorscheme info to highlight this particular segment. Colorschemes can also be global and define a coherent colorscheme both for vim, tmux, prompts, etc.

The idea that we should have a purpose-agnostic way of defining segments would certainly be nice, but I can't see how this would be possible at all. How would you make e.g. the line percentage segment in vim work transparently with e.g. zsh prompts? I think we should approach this by dividing stuff into application-specific modules, where the renderer, segments, themes, etc. is located in each module with no connection to the other applications.

Regarding the separator icons, I think the plan is to have a global config file where the separator icons can be defined for each application, but it would be global for that application and not for each segment. It's possible to have a segment-specific divider but I'm not sure I'd prioritize that functionality before the project is launched.

[derekbrokeit]

One use case for a pure-python method I can think off the top of my head is segment-segment dependencies. For example, if git_branch fails to render, do not try to render git_upstream_compare. However, now that I think about it, it makes sense usually to put dependent segments left-right (at least in English) and it could be implemented in JSON with a "depends":"git_branch" segment or something similar to have conditional rendering.

So perhaps the best and only idea I can think of is having personal segment-functions embedded directly into the theme file, which, as you said, would then require the user to learn/know python. Perhaps it's not that important, but it could have benefits for some users.

My thinking before was just to have it attempt a json.load, and if it throws an exception for bad json, you could assume some form of python function. I am not sure if json does this because yaml does not throw an exception and I have more experience load yaml files. Anyway, those are just my thoughts, since I think it would be more of a secondary attempt in case of failure of json loading since users may want to specify functions that are not in the powerline library module already.

[ZyX-I]

Personal segment functions are one of the things that are to be done. But there is no need to mess segment code and theme, user knowing python will just write a separate module. (By the way, here comes another issue: if in Vim suggestion No 2 (just provide an API for adding segments and don’t care about anything else) can be accepted, how is this going to work in zsh/tmux?)

Segment dependencies are not much useful: nothing can prevent segment module from having global state where all the necessary things are cached to introduce such behavior. This is also more flexible: in this case any first segment that discovers absence of .git directory may speed up all of the subsequent git-requiring ones. WIth your suggestion you have to rewrite "depends": key each time you want to exchange their positions. I also hope we will eventually add threading here and thus won’t guarantee segment evaluation order.

---

By the way, since when YAML parser (pyyaml) stopped throwing exceptions? Perhaps it just takes python function as a valid YAML (it is not hard to write it this way).

I would also like to see YAML here, but I never suggested this feature due to mentioned problem: adding unnecessary dependencies.

[ZyX-I]

Looking at your sample JSON file, it is not immediately clear what governs the highlighting or coloring of each segment except a few that have explicit highlighting. Is it expected that the function returns the color? If so, should these functions be able to tell what format the color needs to be in (e.g. Vim-highlight, ANSI, etc.)? I expect we will need to have separate theme files for different applications (tmux, vim, ...), but from my understanding, the functions are written in python and should be as purpose-agnostic as possible.

Current suggestion:

1. If function returned highlighting use this. Hl specification must be obtained by some API, either before it is put into a returned dictionary or when that dictionary is processed (we have not talked about this detail).
2. If user has specified highlighting in the theme use it.
3. If colorscheme has highlighting equivalent to the one provided when defining segment (defaults to segment name?) use this.
4. Raise an exception refusing to show anything, explain the issue and ways to solve it in the exception message.

It is my understanding of the summary of what was discussed only partly with the related topics, not the official position of the author.

[Lokaltog]

Themes and colorschemes have now been implemented in 98337d2, like @ZyX-I describes it in the last comment.