Improve performance during refresh

[blogh]

Scrolling is very slow and a long press on up/down is not very manageable.

[blogh]

I used the line_profiler to profile screen() and the processes_rows() with -w1, -w2 and -w3.

Here is what I get for -w3 (but it's the same for -w1 and -w2).
Most of the time is spent in shorten().

Timer unit: 1e-06 s

Total time: 3.87632 s
File: /home/benoit/git/dalibo/pg_activity/pgactivity/views.py
Function: wrapper at line 117

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
  117                                               @functools.wraps(func)
  118                                               def wrapper(term: Terminal, *args: Any, **kwargs: Any) -> None:
  119        92        248.0      2.7      0.0          counter = kwargs.pop("lines_counter", None)
  120        92        128.0      1.4      0.0          width = kwargs.pop("width", None)
  121        92      21321.0    231.8      0.6          signature = inspect.signature(func)
  122        92        336.0      3.7      0.0          if "width" in signature.parameters:
  123        23         30.0      1.3      0.0              kwargs["width"] = width
  124      1334     366825.0    275.0      9.5          for line in func(term, *args, **kwargs):
  125      1265    3481531.0   2752.2     89.8              print(shorten(term, line, width) + term.clear_eol)
  126      1265       5664.0      4.5      0.1              if counter is not None and next(counter) == 1:
  127        23        233.0     10.1      0.0                  break

Total time: 4.02128 s
File: /home/benoit/git/dalibo/pg_activity/pgactivity/views.py
Function: screen at line 464

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
  464                                           @profile
  465                                           def screen(
  466                                               term: Terminal,
  467                                               ui: UI,
  468                                               *,
  469                                               host: Host,
  470                                               dbinfo: DBInfo,
  471                                               pg_version: str,
  472                                               tps: int,
  473                                               active_connections: int,
  474                                               activity_stats: ActivityStats,
  475                                               message: Optional[str],
  476                                               render_header: bool = True,
  477                                               render_footer: bool = True,
  478                                               width: Optional[int] = None,
  479                                           ) -> None:
  480                                               """Display the screen."""
  481                                           
  482                                               system_info: Optional[SystemInfo]
  483        23        150.0      6.5      0.0      if isinstance(activity_stats, tuple):
  484        23         55.0      2.4      0.0          processes, system_info = activity_stats
  485                                               else:
  486                                                   processes, system_info = activity_stats, None
  487        23       4714.0    205.0      0.1      processes.set_items(sorted_processes(processes, key=ui.sort_key, reverse=True))
  488                                           
  489        23        242.0     10.5      0.0      print(term.home, end="")
  490        23       1094.0     47.6      0.0      top_height = term.height - (1 if render_footer else 0)
  491        23        147.0      6.4      0.0      lines_counter = line_counter(top_height)
  492                                           
  493        23         39.0      1.7      0.0      if render_header:
  494        46     239162.0   5199.2      5.9          header(
  495        23         34.0      1.5      0.0              term,
  496        23         40.0      1.7      0.0              ui,
  497        23         36.0      1.6      0.0              host=host,
  498        23         38.0      1.7      0.0              dbinfo=dbinfo,
  499        23         63.0      2.7      0.0              pg_version=pg_version,
  500        23         36.0      1.6      0.0              tps=tps,
  501        23         40.0      1.7      0.0              active_connections=active_connections,
  502        23         36.0      1.6      0.0              system_info=system_info,
  503        23         39.0      1.7      0.0              lines_counter=lines_counter,
  504        23         38.0      1.7      0.0              width=width,
  505                                                   )
  506                                           
  507        23      89445.0   3888.9      2.2      query_mode(term, ui, lines_counter=lines_counter, width=width)
  508        23     233861.0  10167.9      5.8      columns_header(term, ui, lines_counter=lines_counter, width=width)
  509        46    3319001.0  72152.2     82.5      processes_rows(
  510        23         32.0      1.4      0.0          term,
  511        23         31.0      1.3      0.0          ui,
  512        23         33.0      1.4      0.0          processes,
  513        23         36.0      1.6      0.0          maxlines=lines_counter.value,  # Used by process_rows
  514        23         32.0      1.4      0.0          lines_counter=lines_counter,  # Used by @limit
  515        23         31.0      1.3      0.0          width=width,
  516                                               )
  517                                           
  518                                               # Clear remaining lines in screen until footer (or EOS)
  519        23        147.0      6.4      0.0      print(f"{term.clear_eol}\n" * lines_counter.value, end="")
  520                                           
  521        23         21.0      0.9      0.0      if render_footer:
  522        23       1269.0     55.2      0.0          with term.location(x=0, y=top_height):
  523        23         29.0      1.3      0.0              if message is not None:
  524                                                           footer_message(term, message, width)
  525        23         62.0      2.7      0.0              elif ui.interactive():
  526        22     125027.0   5683.0      3.1                  footer_interative_help(term, width)
  527                                                       else:
  528         1       6225.0   6225.0      0.2                  footer_help(term, width)

[blogh]

My idea was to do create the line once and scroll on the result after :
(Note: the testcases are different so only the repartition is usable)

Total time: 0.267382 s
File: /home/benoit/git/dalibo/pg_activity/pgactivity/views.py
Function: processes_rows at line 411

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   411                                           @profile
   412                                           def processes_rows(
   413                                               term: Terminal,
   414                                               ui: UI,
   415                                               processes: SelectableProcesses,
   416                                               lines_counter: line_counter,
   417                                               width: int,
   418                                               paused: bool = False,
   419                                           ) -> None:
   420        20         18.0      0.9      0.0      if not paused:
   421                                                   # generate the lines only the first time we pause
   422         2     228228.0 114114.0     85.4          prepare_processes_rows(
   423         1          0.0      0.0      0.0              term,
   424         1          0.0      0.0      0.0              ui,
   425         1          0.0      0.0      0.0              processes,
   426         1          1.0      1.0      0.0              width=width,
   427                                                   )
   428                                               # display the line and add the focus, selection on top as we go
   429        40      39030.0    975.8     14.6      display_processes_rows(
   430        20         12.0      0.6      0.0          term,
   431        20         18.0      0.9      0.0          ui,
   432        20         15.0      0.8      0.0          processes,
   433        20         19.0      0.9      0.0          lines_counter.value,          # for display_process_rows()
   434        20         13.0      0.7      0.0          lines_counter=lines_counter,  # for @limit
   435        20         12.0      0.6      0.0          width=width,
   436        20         16.0      0.8      0.0          do_wrapping=False
   437                                               )

I would like to improve it so that:

• when we are out of pause / interactive mode. we only generate a screen worth of line.
• when we enter interactive mode / pause we generate all the line the first time.

[dlax]

Most of the time is spent in shorten().

We could then cache this function.

diff --git a/pgactivity/views.py b/pgactivity/views.py
index 854596f..796a206 100644
--- a/pgactivity/views.py
+++ b/pgactivity/views.py
@@ -57,6 +57,7 @@ class line_counter:
         return current_value
 
 
+@functools.lru_cache(maxsize=512)
 def shorten(term: Terminal, text: str, width: Optional[int] = None) -> str:
     r"""Truncate 'text' to fit in the given 'width' (or term.width).
 

Does this improves things?

[blogh]

works like a charm :)

Total time: 1.53388 s
File: /home/benoit/git/dalibo/pg_activity/pgactivity/views.py
Function: wrapper at line 117

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
   117                                               @functools.wraps(func)
   118                                               def wrapper(term: Terminal, *args: Any, **kwargs: Any) -> None:
   119       124        379.0      3.1      0.0          counter = kwargs.pop("lines_counter", None)
   120       124        218.0      1.8      0.0          width = kwargs.pop("width", None)
   121       124      41869.0    337.7      2.7          signature = inspect.signature(func)
   122       124        578.0      4.7      0.0          if "width" in signature.parameters:
   123        31         51.0      1.6      0.0              kwargs["width"] = width
   124      1704    1051225.0    616.9     68.5          for line in func(term, *args, **kwargs):
   125      1609     429054.0    266.7     28.0              print(shorten(term, line, width) + term.clear_eol)
   126      1609      10080.0      6.3      0.7              if counter is not None and next(counter) == 1:
   127        29        425.0     14.7      0.0                  break

[blogh]

Done with functools.lru_cache() in the PR.

[blogh]

thanks