WIP: Syntax Highlighting

[jeff-hykin]

This is more me talking about how I'm working on adding it

Pre-alpha prototyping:

Explanation

Play around with the parser here on the tree-sitter site

• I'm using web-tree-sitter
• There's a query syntax
• The good/nice part is, the color theme can just be a list of query-to-style mappings
  • I specialize in syntax-highlighting for VS Code, and let me tell yah, this is the most clean and powerful theming approach of any editor/terminal right now
• For rendering, I made a function that accepts text plus query-to-style mappings and spits out HTML (spans and paragraph tags)
  • They can be heirarchical/nested
  • Its incredibly efficient at handling massive amounts of pasted zsh code

Discussion

• As-is generating raw HTML doesn't fit nicely into React, and (other than a wrapper) the task in general, with heiarchy, basically can't be done performantly with the virtual dom. So, a react wrapper we shall need.
• Nice-ness aside, even if handed a react component, right now the colored text is independent of whats going on in the textarea
  • A hacky way would be make the textarea-font color invisible, and then overlay the colored text. This would prevent any weird cursor/selection stuff from happening, but bold styles would make the cursor not line up and the line overflow/expansion of the textarea would pobably be hard to keep in sync with the colored text
  • Moving away from Textarea, maybe to a contenteditable div or some similar approach could work. The issue is handing cursor movement/selections even though they are going across spans/paragraphs. Proabably the better approach, but IDK what all hooks are going into the textarea

Just some food for thought. It'll probably be a month before I work on it again

[esimkowitz]

This is awesome, thank you for sharing! We have been thinking about how to redo the prompt area to make it more flexible going forward. FYI though, I've found contenteditable divs to be a nightmare to work with. More recently, I was looking at just swapping the textarea for a full-fledged text editor plugin like Monaco or Lexical. We currently use Monaco to power our codeedit plugin, but it isn't very easy to integrate with React. From what I've seen, Lexical plays better (it is built by Facebook so I'd hope so LOL), but it is a newer project and the docs are not as mature. Let me know if you'd like to discuss this further, I think the Discord would be a good forum for this.

[jeff-hykin]

I've found contenteditable divs to be a nightmare to work with.

Same

More recently, I was looking at just swapping the textarea for a full-fledged text editor plugin like Monaco or Lexical

I agree I think that is the way to go. It seems like editing colorized text is a whole project in and of itself, so the only realistic way (AFAIK) is finding someone who's already done the work.

[roke-julian-lockhart]

@jeff-hykin, Warp provides this. I don't know of any [F]OSS alternative that does.