-
Notifications
You must be signed in to change notification settings - Fork 101
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Screen optimizations #160
base: master
Are you sure you want to change the base?
Screen optimizations #160
Commits on Jun 17, 2022
-
Upgrade pyperf (drop support for Python 2.x)
Since 0.8.1 pyte does not support Python 2.x anymore so it makes sense to upgrade one of its dev dependencies, pyperf.
Configuration menu - View commit details
-
Copy full SHA for 8513fe8 - Browse repository at this point
Copy the full SHA 8513fe8View commit details -
Allow change the screen geometry
Receive via environ the geometry of the screen to test with a default of 24 lines by 80 columns. Add this and the input file into Runner's metadata so it is preserved in the log file (if any)
Configuration menu - View commit details
-
Copy full SHA for cabc0a5 - Browse repository at this point
Copy the full SHA cabc0a5View commit details -
Impl benchmark tests for screen.display, .reset and .resize
Implement three more benchmark scenarios for testing screen.display, screen.reset and screen.resize. For the standard 24x80 geometry, these methods have a negligible cost however of larger geometries, they can be up to 100 times slower than stream.feed so benchmarking them is important. Changed how the metadata is stored so on each bench_func call we encode which scenario are we testing, with which screen class and geometry.
Configuration menu - View commit details
-
Copy full SHA for 940e19b - Browse repository at this point
Copy the full SHA 940e19bView commit details
Commits on Jun 18, 2022
-
Impl script to run a full benchmark
A shell script to test all the captured input files and run them under different terminal geometries (24x80, 240x800, 2400x8000, 24x8000 and 2400x80). These settings aim to stress pyte with larger and larger screens (by a 10 factor on both dimensions and on each dimension separately).
Configuration menu - View commit details
-
Copy full SHA for 0b8007a - Browse repository at this point
Copy the full SHA 0b8007aView commit details
Commits on Jul 2, 2022
-
Fix benchmark.py using ByteStream and not Stream
The input files in the tests/captured must be loaded with ByteStream and not Stream, otherwise the \r are lost and the benchmark results may not reflect real scenarios.
Configuration menu - View commit details
-
Copy full SHA for e0b0e8b - Browse repository at this point
Copy the full SHA e0b0e8bView commit details
Commits on Jul 4, 2022
-
Configuration menu - View commit details
-
Copy full SHA for eec4a2e - Browse repository at this point
Copy the full SHA eec4a2eView commit details -
display meth: iterate over data entries filling the gap between
The former `for x in range(...)` implementation iterated over the all the possibly indexes (for columns and lines) wasting cyclies because some of those indexes (and in some cases most) pointed to non-existing entries. These non-existing entries were faked and a default character was returned in place. This commit instead makes display to iterate over the existing entries. When gaps between to entries are detected, the gap is filled with the same default character without having to pay for indexing non-entries. Note: I found that in the current implementation of screen, screen.buffer may have entries (chars in a line) outside of the width of the screen. At the display method those are filtered out however I'm not sure if this is not a real bug that was uncovered because never we iterated over the data entries. If this is true, we may be wasting space as we keep in memory chars that are outside of the screen.
Configuration menu - View commit details
-
Copy full SHA for f899535 - Browse repository at this point
Copy the full SHA f899535View commit details -
Inline generator into display inner loop
Python generators (yield) and function calls are slower then normal for-loops. Improve screen.display by x1 to x1.8 times faster by inlining the code.
Configuration menu - View commit details
-
Copy full SHA for b3b7db4 - Browse repository at this point
Copy the full SHA b3b7db4View commit details -
The assert that checks the width of each char is removed from screen.display and put it into the tests. This ensures that our test suite maintains the same quality and at the same time we make screen.display ~x1.7 faster.
Configuration menu - View commit details
-
Copy full SHA for de59245 - Browse repository at this point
Copy the full SHA de59245View commit details -
Instead of computing it on each screen.display, compute the width of the char once on screen.draw and store it in the Char tuple. This makes screen.display ~x1.10 to ~x1.20 faster and it makes stream.feed only ~x1.01 slower in the worst case. This negative impact is due the change on screen.draw but measurements on my lab show inconsistent results (stream.feed didn't show a consistent performance regression and ~x1.01 slower was the worst value that I've got).
Configuration menu - View commit details
-
Copy full SHA for 020fce6 - Browse repository at this point
Copy the full SHA 020fce6View commit details -
Pre-fetch attributes on screen.draw (x1.20 to x2.0 faster)
Fetch some attributes that were frequently accessed in the for-loop of screen.draw avoiding accessing them on each iteration. Most of them remain constant within the draw() method anyways. Others, like cursor.x, cursor.y and line are updated infrequently inside the for-loop so it still faster pre-fetch them outside and update them if needed than accessing them on each iteration. Benchmark results show stream.feed is x1.20 to x2.0 faster with these optimizations. Benchmark files that have more control sequences (like htop, mc and vim) have a lower improvement as the parsing of these sequences dominates the runtime of stream.feed.
Configuration menu - View commit details
-
Copy full SHA for 8e7ee07 - Browse repository at this point
Copy the full SHA 8e7ee07View commit details -
Allow temporal cursor_x > columns (x1.05 to x1.14 faster)
Instead of checking if cursor_x > columns at the end of iteration and set cursor_x to the minimum of (cursor_x and columns), delay that decision to the begin of the next iteration or at the end of the for-loop. This removes one "if" statement at the end of the for-loop and allows us to use the local variable cursor_x all the time without having to update cursor.x. Only this happens before insert_characters() and at the end of the draw() method when the cursor.x is been visible by code outside draw() and therefore must be updated with the latest value of cursor_x. This optimization makes stream.feed between x1.05 and x1.14 faster. As in any optimization on draw(), the use cases that gets more improvements are the ones that have very few control sequences in their input (so stream.feed is dominated by screen.draw and not be stream._parse_fsm)
Configuration menu - View commit details
-
Copy full SHA for d4d2e4a - Browse repository at this point
Copy the full SHA d4d2e4aView commit details -
Make Char mutable (not a namedtuple) (API may break)
Make Char mutable (and ordinary object) so we can modify each char in place avoiding calling _replace. This commit only changed the Char class and implements some methods to emulate the namedtuple API. Theoretically it could be possible to emulate the whole namedtuple API but it is unclear if it worth. In this scenario, user code may break. Using a plain object instead of a namedtuple added a regression on memory usage of x1.20 for htop and mc benchmark files when HistoryScreen was used. The rest of the benchmarks didn't change significantly (but it is expected to be slightly more inefficient).
Configuration menu - View commit details
-
Copy full SHA for 945b19b - Browse repository at this point
Copy the full SHA 945b19bView commit details -
Configuration menu - View commit details
-
Copy full SHA for e881d25 - Browse repository at this point
Copy the full SHA e881d25View commit details -
Reuse/share char styles (x1.05 to x1.30 lighter)
Reduce the memory footprint reusing/sharing the same CharStyle object among different Char instances. A specialized _replace_data changes the data and width of the char but not its style. This reduces the footprint between x1.05 and x1.30 with respect the 0.8.1-memory.json baseline results.
Configuration menu - View commit details
-
Copy full SHA for 5f784ec - Browse repository at this point
Copy the full SHA 5f784ecView commit details -
Update Chars in-place instead of recreating them (x1.20 and x1.90 fas…
…ter; regress on mem) Instead of calling _replace() to create a new Char object, modify the existing one. For that, the Line (ex StaticDefaultDict) is in charge to fetch the char and do the modifications. If no Char is found, only then a Char is created and inserted in the Line (dict). See write_data(). In some cases we need to get a Char, read it and then update it so a copy of the Line's default char is returned and added to the line. A copy is required because now the Char are mutable. See char_at(). Changed the API of Char: _asdict renamed as as_dict and _replace as copy_and_change; removed _replace_data and added a copy method. The constructor also changed: it is required data, width and style. The former way to construct a Char can be done with from_attributes class method. This commits improved the runtime of stream.feed by x1.20 to x1.90 (faster) however a regression on the memory footprint was found (between x1.10 and x1.50). I don't have an explanation for this last point.
Configuration menu - View commit details
-
Copy full SHA for 9721698 - Browse repository at this point
Copy the full SHA 9721698View commit details -
Fix test_reverse_index (history) due old API
The test was using a legacy API of screen.buffer when the buffer was a dense matrix. Now it is sparse we cannot use len(screen.buffer) anymore or buffer[-1] either.
Configuration menu - View commit details
-
Copy full SHA for e49fb3f - Browse repository at this point
Copy the full SHA e49fb3fView commit details -
Use binary search over non-empty lines on index/reverse_index
This improvement impacts slighly negatively over small geometries (x1.01 to x1.05 slower) but improves on larger geometries and for almost all the cases of HistoryScreen (x1.10 to x1.20)
Configuration menu - View commit details
-
Copy full SHA for d94299d - Browse repository at this point
Copy the full SHA d94299dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 912028f - Browse repository at this point
Copy the full SHA 912028fView commit details
Commits on Jul 5, 2022
-
Calculate statistics about buffer's and lines' internals (no stable API)
It is handy to get some stats about the layout and chars locations in the lines/buffer and see how sparse they are. The statistics are not part of the stable API so they may change between versions.
Configuration menu - View commit details
-
Copy full SHA for 4c04935 - Browse repository at this point
Copy the full SHA 4c04935View commit details -
On screen.buffer return a read-only view (BufferView/LineView)
This layer of abstraction will allow use to changes on the real buffer without breaking the public API.
Configuration menu - View commit details
-
Copy full SHA for 84cd21f - Browse repository at this point
Copy the full SHA 84cd21fView commit details -
Configuration menu - View commit details
-
Copy full SHA for 5ae46bc - Browse repository at this point
Copy the full SHA 5ae46bcView commit details -
Do not unintentionally create empty lines
Because screen._buffer is a defaultdict, an access to a non-existent element has the side effect
Configuration menu - View commit details
-
Copy full SHA for da66a7e - Browse repository at this point
Copy the full SHA da66a7eView commit details
Commits on Jul 6, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 1fd373a - Browse repository at this point
Copy the full SHA 1fd373aView commit details -
Use a space for padding screen.display
screen.default_char is always the space character so instead of using screen.default_char for padding we use the space character directly.
Configuration menu - View commit details
-
Copy full SHA for b8250ea - Browse repository at this point
Copy the full SHA b8250eaView commit details -
Replace line's default style instead overwriting its char
Because the default char of a line is always the space character, we don't need to overwrite it with screen.default_char, just we need to change its style.
Configuration menu - View commit details
-
Copy full SHA for 8d71528 - Browse repository at this point
Copy the full SHA 8d71528View commit details -
Fix a bug on index that top line was not removed.
If the next line of the margin's top was not empty, the for-loop used that entry to override the top line working as expected. But when the next line of the top was empty, the top line was untouched so an explicit pop is required. A similar issue happen on reverse_index. Both bugs were covered due a side effect to iterating over screen.buffer: on each line lookup, if no such exist, a new line is added to the buffer. This added new line then was used to override the top on an index() call.
Configuration menu - View commit details
-
Copy full SHA for 71ab12a - Browse repository at this point
Copy the full SHA 71ab12aView commit details -
Configuration menu - View commit details
-
Copy full SHA for 01f96bc - Browse repository at this point
Copy the full SHA 01f96bcView commit details
Commits on Jul 7, 2022
-
Try to delete entries on erase instead of write spaces
If the line's default char and the cursor's attributes are the same, instead of writing spaces into the line, delete the chars. This should be equivalent from user's perspective. This applies to erase_characters, erase_in_line and erase_in_display. This optimization reduced the number of false lines (lines without any char) and the number of blank lines (lines with only spaces). With less entries in the buffer, the rest of the iterations can take advantage of the sparsity of the buffer and process much less.
Configuration menu - View commit details
-
Copy full SHA for 70d4b15 - Browse repository at this point
Copy the full SHA 70d4b15View commit details
Commits on Jul 8, 2022
-
Impl prev_page/next_page with sparse iteration
We avoid a full scan and instead we do a sparse iteration. This also avoids adding empty lines into the buffer as real entries when a non-existing entry has the same effect and it consumes less memory.
Configuration menu - View commit details
-
Copy full SHA for 4a15d3a - Browse repository at this point
Copy the full SHA 4a15d3aView commit details -
Make lines in history a LineView and make Line raise on non-key
Instead of storing Line objects in the history's top and bottom lists, store LineView of them. This makes them read-only and allows us to make Line a full dict without an overloaded __missing__. Skipping __missing__ ensures that the code of Screen does not access an x position that does not exist by mistake.
Configuration menu - View commit details
-
Copy full SHA for 6b4f088 - Browse repository at this point
Copy the full SHA 6b4f088View commit details -
Configuration menu - View commit details
-
Copy full SHA for 13cf059 - Browse repository at this point
Copy the full SHA 13cf059View commit details -
Configuration menu - View commit details
-
Copy full SHA for a4e7ed5 - Browse repository at this point
Copy the full SHA a4e7ed5View commit details -
Configuration menu - View commit details
-
Copy full SHA for 3056742 - Browse repository at this point
Copy the full SHA 3056742View commit details -
Configuration menu - View commit details
-
Copy full SHA for f96ab6b - Browse repository at this point
Copy the full SHA f96ab6bView commit details -
Configuration menu - View commit details
-
Copy full SHA for 70763b6 - Browse repository at this point
Copy the full SHA 70763b6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 51e79a6 - Browse repository at this point
Copy the full SHA 51e79a6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 47d7c62 - Browse repository at this point
Copy the full SHA 47d7c62View commit details -
Configuration menu - View commit details
-
Copy full SHA for b4258e1 - Browse repository at this point
Copy the full SHA b4258e1View commit details
Commits on Jul 9, 2022
-
Configuration menu - View commit details
-
Copy full SHA for f178712 - Browse repository at this point
Copy the full SHA f178712View commit details -
Configuration menu - View commit details
-
Copy full SHA for 05c8c2c - Browse repository at this point
Copy the full SHA 05c8c2cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 80aa50a - Browse repository at this point
Copy the full SHA 80aa50aView commit details -
Optional use a dummy set for tracking dirty lines
By default Screen tracks which lines are dirty and should be of interest for the user. This functionality may not be of interest for all the use cases and this tracking is expensibe, specially for large geometries. If track_dirty_lines is set to False, the screen.dirty attribute becomes a dummy or null set that it is always empty saving memory and time.
Configuration menu - View commit details
-
Copy full SHA for 1bee165 - Browse repository at this point
Copy the full SHA 1bee165View commit details -
Configuration menu - View commit details
-
Copy full SHA for 065b31d - Browse repository at this point
Copy the full SHA 065b31dView commit details -
Make (0, lines-1) the default margin instead of None
Instead of using None as a special case, set the margins to (0, lines-1) by default. This may break user code if the user is expecting None as a valid value or if it is setting it. If required we could make screen.margins a property and hide the internal implementation.
Configuration menu - View commit details
-
Copy full SHA for 8f11049 - Browse repository at this point
Copy the full SHA 8f11049View commit details -
Make internal _buffer a dict and not a defaultdict
Using a defaultdict can easily introduce false entries in the buffer making it less sparse. The new Buffer class supports all the dict's operations but it does not add an entry if the key is missing. To add a new entry (a new line), do a buffer.line_at(y). This is equivalent to dict.setdefault(y, new_line()) but avoids the call to new_line() if an entry y exists.
Configuration menu - View commit details
-
Copy full SHA for 01b7d56 - Browse repository at this point
Copy the full SHA 01b7d56View commit details -
Optionally disable display graphic attributes
When the graphic attributes are disabled, select_graphic_rendition always set the default style. With this, the lines' default char and the cursor's char will always match and the screen will optimize the erase* methods.
Configuration menu - View commit details
-
Copy full SHA for e3fdf41 - Browse repository at this point
Copy the full SHA e3fdf41View commit details -
Configuration menu - View commit details
-
Copy full SHA for 7839ded - Browse repository at this point
Copy the full SHA 7839dedView commit details
Commits on Jul 10, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 2ca29a5 - Browse repository at this point
Copy the full SHA 2ca29a5View commit details -
Configuration menu - View commit details
-
Copy full SHA for c589265 - Browse repository at this point
Copy the full SHA c589265View commit details -
Configuration menu - View commit details
-
Copy full SHA for 1b42c89 - Browse repository at this point
Copy the full SHA 1b42c89View commit details -
Configuration menu - View commit details
-
Copy full SHA for c65ac82 - Browse repository at this point
Copy the full SHA c65ac82View commit details -
Impl compressed_display to optionally avoid left/right spaces, top/bo…
…ttom empty lines This is an optimization over screen.display where it is possible to strip left/right spaces of the lines and/or filter top/bottom whole empty lines. It is implemented in an opportunistic fashion so it may not fully strip/filter all that it is expected. In particular, lines with left or right spaces with non-default attributes are not stripped; lines that contains only spaces at the top or bottom are not filtered (even if lstrip/rstrip is set). This implementation is meant to be used when the screen has very large geometries and screen.display wastes too much time and memory on padding large lines and/or filling with a lot of empty lines.
Configuration menu - View commit details
-
Copy full SHA for f4ea46a - Browse repository at this point
Copy the full SHA f4ea46aView commit details
Commits on Jul 12, 2022
-
Configuration menu - View commit details
-
Copy full SHA for ba980a0 - Browse repository at this point
Copy the full SHA ba980a0View commit details