Syntax highlighter refactor #38440

Paulb23 · 2020-05-03T18:24:09Z

First part of #31739

Started as part of #17923 however as we could not break compatibility full separation was not possible. This PR completed that work leaving the base concept intact.

Firstly, the SyntaxHighlighterclass has been converted into a resource and exposed to GDScript. All syntax highlighting code is now managed by this class not TextEdit.

The return of _get_line_syntax_highlighting has been changed to Dictionary<int, Dictionary> instead of the struct HighlighterInfo to allow use in GDScript, currently the only supported value in this Dictionary is the key 'color'.

The TextEdit changes are as follows:

New property to set the syntax highlighter, the bound methods for which have been renamed from get/set_syntax_highlighting to get/set_syntax_highlighter.
The syntax_coloring property has been removed.
All syntax highlighting colours in themes, properties and methods have been removed.
A new signal line_edited_from with the param of the changed line number.
The current highlighting code now exists in the CodeHighlighter syntax highlighter resource.

On the editor side a new EditorSyntaxHighlighter resource extending SyntaxHighlighter has been created for use in the script editor. This class and the get/set methods have been exposed for tool scripts. There are now three editor highlighters:

EditorStandardSyntaxHighlighter
EditorPlainTextSyntaxHighlighter
GDScriptSyntaxHighlighter

Lastly, It is now also possible to edit the syntax highlighter in the inspector.

Closes godotengine/godot-proposals/issues/597
Closes #33094
Part of #37540

YuriSizov · 2020-05-03T18:46:07Z

Thank you very much!

I know that this breaks compatibility in some parts, but given the nature of this change and the careful way it was implemented, how possible it is to be included in 3.x at some point before 4.x release? I'd argue, that general uses of TextEdit are intact and only those who developed some devtools/plugins are affected. And this crowd (myself included) can adjust their codebase.

Zylann · 2020-05-05T00:36:29Z

The return of _get_line_syntax_highlighting has been changed to Dictionary<int, Dictionary> instead of the struct HighlighterInfo to allow use in GDScript, currently the only supported value in this Dictionary is the key 'color'.

I got a bit surprised by this. Any reason why it should be a nested dictionary? Right now for a highlighter, forcing to create plenty of small dictionaries with string keys inside just to represent colors is calling for a lot of unnecessary allocations. At least if the goal is to allow more data (but which data?) it could use Variant, or have a wrapper object to fill in that data (like SurfaceTool does).
In the case of C# it would also cause more GC.

Paulb23 · 2020-05-05T19:15:13Z

@pycbouh Unfortunately it's unlikely to be included in a 3.x release in part due to the compatibility breakage.

@Zylann Yeah I'm not the biggest fan of it either. The nested dictionary is Dictionary<String, Variant> as you say is to allow adding more data, currently I have in mind foreground / background colour, font weight and italics. I can't see a way to avoid the Dictionary allocation other then replacing it with a class. To avoid the string key there is the potential to use StringName instead.

akien-mga · 2020-05-06T07:06:08Z

core/script_language.cpp

@@ -355,6 +355,38 @@ ScriptCodeCompletionCache::ScriptCodeCompletionCache() {
 	singleton = this;
 }

+void ScriptLanguage::get_core_type_words(List<String> *p_core_type_words) const {


This could likely be used to remove the hardcoded core type names from modules/gdscript/gdscript_editor.cpp.

That's likely work for another PR though. There might be more synergy possible between the syntax highlighter code and the auto-completion code, at least as far as identifying core types, classes and keywords goes. (Or maybe not, it has to be assessed whether it makes sense/would be convenient or would just be too cumbersome.)

akien-mga · 2020-05-06T07:08:08Z

core/script_language.cpp

+	p_core_type_words->push_back("StringName");
+	p_core_type_words->push_back("NodePath");
+	p_core_type_words->push_back("RID");
+	p_core_type_words->push_back("Callable");


From variant.h you missed "Signal" after "Callable".

There's also Object, null, bool, int, float, but I guess those are left out on purpose.

akien-mga · 2020-05-06T07:09:39Z

CC @neikeq as this might be relevant if we want to add C# syntax highlighting.

Paulb23 · 2020-05-07T19:58:03Z

Added 'Signal' into the core type list.

akien-mga · 2020-05-16T11:15:19Z

Needs a rebase after my recent style and init changes, otherwise should be good to merge.

scene/resources/syntax_highlighter.h

ThakeeNathees · 2020-06-13T04:24:41Z

does it fix syntax highlighting for methods with the same name of builtin functions?

Paulb23 · 2020-06-14T14:36:26Z

@ThakeeNathees No, the syntax highlighting part is unchanged. (#28866)

…hter - Extacted all syntax highlighting code from text edit - Removed enable syntax highlighting from text edit - Added line_edited_from signal to text_edit - Renamed get/set_syntax_highlighting to get/set_syntax_highlighter - Added EditorSyntaxHighligher

akien-mga · 2020-07-14T07:49:06Z

Thanks!

akien-mga · 2020-07-14T11:41:03Z

@Paulb23 Raises those errors in a simple test project:

ERROR: color region with start key '/*' already exists.
   at: add_color_region (scene/resources/syntax_highlighter.cpp:484)
ERROR: color region with start key '//' already exists.
   at: add_color_region (scene/resources/syntax_highlighter.cpp:484)
ERROR: color region with start key '/*' already exists.
   at: add_color_region (scene/resources/syntax_highlighter.cpp:484)
ERROR: color region with start key '//' already exists.
   at: add_color_region (scene/resources/syntax_highlighter.cpp:484)

Stuffe · 2022-01-15T09:23:15Z

Will this allow complete control over text coloring? For my game it would be extremely useful to be able to color by character index range manually.

Paulb23 added enhancement feature proposal topic:core topic:editor labels May 3, 2020

Paulb23 added this to the 4.0 milestone May 3, 2020

Paulb23 requested review from bojidar-bg and vnen as code owners May 3, 2020 18:24

akien-mga mentioned this pull request May 6, 2020

Expose SyntaxHighlighter to scripts #33094

Closed

akien-mga reviewed May 6, 2020

View reviewed changes

akien-mga requested a review from neikeq May 6, 2020 07:09

Paulb23 force-pushed the syntax_highlighter_refactor branch from e930f04 to f910327 Compare May 7, 2020 19:52

akien-mga reviewed May 16, 2020

View reviewed changes

scene/resources/syntax_highlighter.h Outdated Show resolved Hide resolved

akien-mga mentioned this pull request May 16, 2020

Highlight control flow keywords with a different color #33577

Merged

Paulb23 force-pushed the syntax_highlighter_refactor branch 2 times, most recently from 5649a0d to 45c5a02 Compare May 17, 2020 16:56

Paulb23 force-pushed the syntax_highlighter_refactor branch from 45c5a02 to d92d4b6 Compare July 10, 2020 12:20

Paulb23 added 3 commits July 11, 2020 15:19

Add core types to ScriptLanguage

be119c5

Convert syntax highlighters into a resource

2f1080b

Expose Syntax highlighter for editor plugins

156dadd

Paulb23 force-pushed the syntax_highlighter_refactor branch from d92d4b6 to 5fcafc3 Compare July 11, 2020 15:05

Paulb23 force-pushed the syntax_highlighter_refactor branch from 5fcafc3 to bc4cee4 Compare July 11, 2020 16:23

akien-mga merged commit ca5958d into godotengine:master Jul 14, 2020

Paulb23 deleted the syntax_highlighter_refactor branch July 14, 2020 12:13

Paulb23 mentioned this pull request Jul 14, 2020

Fixed visual shader editor not clearing colour regions #40375

Merged

Paulb23 mentioned this pull request Jul 29, 2020

Fix TextEdit line width cache not being updated #40851

Merged

Paulb23 mentioned this pull request Oct 11, 2020

Update SyntaxHighlighter documentation #42721

Merged

Calinou mentioned this pull request Mar 25, 2021

Allow only one text selection at a time #27517

Closed

Atlinx mentioned this pull request Jul 15, 2021

Make text edit syntax highlighting more readable. Fractural/FracturalVisualNovelEngine#20

Open

kitbdev mentioned this pull request Jul 2, 2024

add_keyword_color in textedit does not works using symbols ([ < > ] ....) #30586

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Syntax highlighter refactor #38440

Syntax highlighter refactor #38440

Paulb23 commented May 3, 2020

YuriSizov commented May 3, 2020

Zylann commented May 5, 2020 •

edited

Loading

Paulb23 commented May 5, 2020

akien-mga May 6, 2020

akien-mga May 6, 2020

akien-mga commented May 6, 2020

Paulb23 commented May 7, 2020

akien-mga commented May 16, 2020

ThakeeNathees commented Jun 13, 2020

Paulb23 commented Jun 14, 2020

akien-mga commented Jul 14, 2020

akien-mga commented Jul 14, 2020

Stuffe commented Jan 15, 2022 •

edited

Loading

Syntax highlighter refactor #38440

Syntax highlighter refactor #38440

Conversation

Paulb23 commented May 3, 2020

YuriSizov commented May 3, 2020

Zylann commented May 5, 2020 • edited Loading

Paulb23 commented May 5, 2020

akien-mga May 6, 2020

Choose a reason for hiding this comment

akien-mga May 6, 2020

Choose a reason for hiding this comment

akien-mga commented May 6, 2020

Paulb23 commented May 7, 2020

akien-mga commented May 16, 2020

ThakeeNathees commented Jun 13, 2020

Paulb23 commented Jun 14, 2020

akien-mga commented Jul 14, 2020

akien-mga commented Jul 14, 2020

Stuffe commented Jan 15, 2022 • edited Loading

Zylann commented May 5, 2020 •

edited

Loading

Stuffe commented Jan 15, 2022 •

edited

Loading