paint each line with different lexer - slow

[veksha]

hi, @Alexey-T.
in cuda_search_lite i'm using Assembly lexer right now as default lexer for all lines.
it is fast and have okay colors.

i tried to implement feature where every line will be painted with different lexer colors.
(for .py files it will use Python, for .ini it will use Ini files, etc...)

what i did: i used hidden editor control and each line that needs colors i add to it and get tokens out of it and paint these colors using attr markers.
but I've got very bad performance.

the following lines take very long time:

#def paint_line
self.colors_ed.set_prop(PROP_LEXER_FILE, lexer)
self.colors_ed.action(EDACTION_LEXER_SCAN)

(by the way: EDACTION_LEXER_SCAN is needed because without it get_token(TOKEN_LIST) returns nothing)

PROP_LEXER_FILE takes 0.18 sec
EDACTION_LEXER_SCAN takes 0.12 sec
this is for one line of text. (so 0.18+0.12 = 0.3 sec for every line)

on video i use F2 key to switch to new feature (DETECT_LEXER=True). watch time on titlebar:

cudatext_nERxRFWjts.mp4

i edited some pascal files and got faster results.
now PROP_LEXER_FILE takes 0.03 sec
EDACTION_LEXER_SCAN takes 0.03 sec
0.06 sec for every line = better:

cudatext_aTBUgrbpAM.mp4

but this patch may have some drawbacks:

diff --git a/atsynedit_ex/atsynedit_adapter_econtrol.pas b/atsynedit_ex/atsynedit_adapter_econtrol.pas
index 8dc74a1..fb07e8d 100644
--- a/atsynedit_ex/atsynedit_adapter_econtrol.pas
+++ b/atsynedit_ex/atsynedit_adapter_econtrol.pas
@@ -1196,7 +1196,7 @@ procedure TATAdapterEControl.SetLexer(AAnalizer: TecSyntAnalyzer);
 begin
   while IsParsingBusy do
   begin
-    Sleep(50);
+//    Sleep(50);
     Application.ProcessMessages;
   end;
 
@@ -1740,7 +1740,7 @@ begin
 
     //this method gives ok duration times, like 140ms
     repeat
-      Sleep(60);
+      Sleep(1);
       Application.ProcessMessages;
     until AnClient.IsFinished or Application.Terminated;
   end;

diff --git a/econtrol/ec_syntanal.pas b/econtrol/ec_syntanal.pas
index 8e8fff4..4b6e14a 100644
--- a/econtrol/ec_syntanal.pas
+++ b/econtrol/ec_syntanal.pas
@@ -1181,7 +1181,7 @@ begin
     An.PublicData.FinishedPartially := True;
 
     //constant in WaitFor() affects how fast 'Close all tabs' will run
-    if An.EventParseNeeded.WaitFor(100)<>wrSignaled then
+    if An.EventParseNeeded.WaitFor(1)<>wrSignaled then
       Continue;
 
     An.EventParseIdle.ResetEvent;
@@ -3263,7 +3263,7 @@ begin
   if not IsFinished then
   begin
     FBuffer.Valid := False;
-    Sleep(15);
+//    Sleep(15);
   end;
   FFinished := True;
   //FChangeAtLine := -1; //this causes CudaText issue #3410

so I ask you, Alexey. what can we do about this situation?
maybe you suggest another faster method to paint each line with different lexer?
or maybe we can change something in atsynedit_adapter_econtrol.pas and ec_syntanal.pas to make it faster, but without drawbacks?
or maybe even think to rewrite some pascal code from scratch to make it even faster then 0.03 (0.06) sec ?