-
Notifications
You must be signed in to change notification settings - Fork 271
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GC improvements 1: fix timeless tables not being RowId
-ordered
#4395
Conversation
ba1bba9
to
dc13021
Compare
6828177
to
5b7ab98
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looking great!
// Timeless is RowId-ordered too! | ||
// | ||
// * Insert timeless `point1` with a random `RowId`. | ||
// * Insert timeless `point2` using `point1`'s `RowId`, decremented by one. | ||
// * Query timelessly and make sure we get `point1` because of timeless tie-breaks. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
overviews like that a very appreciated :)
|
||
ent_path.total_size_bytes() | ||
+ cluster_key.total_size_bytes() | ||
+ col_insert_id.total_size_bytes() | ||
+ col_row_id.total_size_bytes() | ||
+ col_num_instances.total_size_bytes() | ||
+ columns.total_size_bytes() | ||
+ is_sorted.total_size_bytes() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
offtopic, but this way of counting the size of a struct (and what it points to) isn't all that accurate since padding between fields can play a big role :/. Not that it matters all that much in the big picture here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, the stats are not meant to be perfectly accurate, they just need to be "good enough" and more importantly consistent: since the GC works with percentages rather than raw numbers, as long as things are consistent it works out okay.
By far our biggest problem at the moment is the size of serialized arrow data, which is both wrong and inconsistent in many ways lol.
Quoting the docs:
/// The size of both the control & component data stored in this bucket, heap and stack
/// included, in bytes.
///
/// This is a best-effort approximation, adequate for most purposes (stats,
/// triggering GCs, …).
Introduce 2 new benchmark suites that drive the development of this PR series: 1. Logging a tons of scalars, in order, across a bunch of series, themselves scattered across a bunch of plots. 2. Logging a tons of timeless data, across a bunch of entities. ### Benchmarks Hint: it's bad. ``` .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024 1.00 1084.0±4.47ms 54.1 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048 1.00 2.1±0.02s 27.6 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256 1.00 465.8±2.50ms 125.8 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512 1.00 655.3±2.61ms 89.4 KElem/sec .../plotting_dashboard/drop_at_least=0.3/default 1.00 652.8±4.12ms 89.8 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024 1.00 2.4±0.05s 24.2 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048 1.00 2.4±0.03s 24.1 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256 1.00 2.5±0.08s 23.5 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512 1.00 2.4±0.02s 24.5 KElem/sec .../timeless_logs/drop_at_least=0.3/default 1.00 2.4±0.03s 24.4 KElem/sec ``` --- Part of the GC improvements series: - #4394 - #4395 - #4396 - #4397 - #4398 - #4399 - #4400 - #4401
3a163cd
to
3e18fe1
Compare
This turns every single column in `DataStore`/`DataTable` into a ringbuffer (`VecDeque`). This means that on the common/happy path of data being ingested in order: 1. Inserting new rows doesn't require re-sorting the bucket (that's nothing new), and 2. garbage collecting rows doesn't require re-sorting the bucket nor copying anything (that's very new). This leads to very significant performance improvements on the common path. - Fixes #1823 ### Benchmarks Compared to `main`: ``` group gc_improvements_0 gc_improvements_3 ----- ----------------- ----------------- .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024 4.50 1084.0±4.47ms 54.1 KElem/sec 1.00 241.0±1.66ms 243.1 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048 8.86 2.1±0.02s 27.6 KElem/sec 1.00 239.9±2.70ms 244.3 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256 1.88 465.8±2.50ms 125.8 KElem/sec 1.00 247.4±3.94ms 236.8 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512 2.72 655.3±2.61ms 89.4 KElem/sec 1.00 241.2±2.06ms 243.0 KElem/sec .../plotting_dashboard/drop_at_least=0.3/default 2.72 652.8±4.12ms 89.8 KElem/sec 1.00 239.6±1.98ms 244.6 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024 40.21 2.4±0.05s 24.2 KElem/sec 1.00 60.3±1.16ms 972.3 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048 40.08 2.4±0.03s 24.1 KElem/sec 1.00 60.8±1.14ms 964.3 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256 40.97 2.5±0.08s 23.5 KElem/sec 1.00 61.0±1.99ms 960.9 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512 39.45 2.4±0.02s 24.5 KElem/sec 1.00 60.6±1.45ms 966.9 KElem/sec .../timeless_logs/drop_at_least=0.3/default 41.78 2.4±0.03s 24.4 KElem/sec 1.00 57.6±0.35ms 1018.1 KElem/sec ``` Compared to previous PR: ``` group gc_improvements_1 gc_improvements_3 ----- ----------------- ----------------- .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024 4.63 1117.2±9.07ms 52.4 KElem/sec 1.00 241.0±1.66ms 243.1 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048 8.96 2.1±0.01s 27.3 KElem/sec 1.00 239.9±2.70ms 244.3 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256 1.91 471.5±4.76ms 124.3 KElem/sec 1.00 247.4±3.94ms 236.8 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512 2.76 666.7±6.64ms 87.9 KElem/sec 1.00 241.2±2.06ms 243.0 KElem/sec .../plotting_dashboard/drop_at_least=0.3/default 2.78 665.6±4.67ms 88.0 KElem/sec 1.00 239.6±1.98ms 244.6 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024 134.66 8.1±0.10s 7.2 KElem/sec 1.00 60.3±1.16ms 972.3 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048 132.44 8.0±0.09s 7.3 KElem/sec 1.00 60.8±1.14ms 964.3 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256 132.22 8.1±0.11s 7.3 KElem/sec 1.00 61.0±1.99ms 960.9 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512 133.27 8.1±0.11s 7.3 KElem/sec 1.00 60.6±1.45ms 966.9 KElem/sec .../timeless_logs/drop_at_least=0.3/default 140.04 8.1±0.07s 7.3 KElem/sec 1.00 57.6±0.35ms 1018.1 KElem/sec ``` --- Part of the GC improvements series: - #4394 - #4395 - #4396 - #4397 - #4398 - #4399 - #4400 - #4401
Indexes `EntityPathHash`es alongside `TimePoint`s in the metadata registry to avoid having to run fullscans during garbage collection. Yields some more significant wins in the common case. ### Benchmarks Compared to `main`: ``` group gc_improvements_0 gc_improvements_4 ----- ----------------- ----------------- .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024 10.32 1084.0±4.47ms 54.1 KElem/sec 1.00 105.0±0.91ms 558.1 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048 19.80 2.1±0.02s 27.6 KElem/sec 1.00 107.3±0.83ms 546.2 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256 4.38 465.8±2.50ms 125.8 KElem/sec 1.00 106.3±0.74ms 551.3 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512 6.16 655.3±2.61ms 89.4 KElem/sec 1.00 106.4±0.94ms 550.6 KElem/sec .../plotting_dashboard/drop_at_least=0.3/default 6.34 652.8±4.12ms 89.8 KElem/sec 1.00 102.9±0.75ms 569.4 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024 37.12 2.4±0.05s 24.2 KElem/sec 1.00 65.3±0.81ms 897.6 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048 37.54 2.4±0.03s 24.1 KElem/sec 1.00 64.9±1.07ms 903.2 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256 38.81 2.5±0.08s 23.5 KElem/sec 1.00 64.4±0.99ms 910.2 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512 37.00 2.4±0.02s 24.5 KElem/sec 1.00 64.6±1.08ms 906.9 KElem/sec .../timeless_logs/drop_at_least=0.3/default 36.82 2.4±0.03s 24.4 KElem/sec 1.00 65.3±1.29ms 897.3 KElem/sec ``` Compared to previous PR: ``` group gc_improvements_3 gc_improvements_4 ----- ----------------- ----------------- .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024 2.30 241.0±1.66ms 243.1 KElem/sec 1.00 105.0±0.91ms 558.1 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048 2.24 239.9±2.70ms 244.3 KElem/sec 1.00 107.3±0.83ms 546.2 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256 2.33 247.4±3.94ms 236.8 KElem/sec 1.00 106.3±0.74ms 551.3 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512 2.27 241.2±2.06ms 243.0 KElem/sec 1.00 106.4±0.94ms 550.6 KElem/sec .../plotting_dashboard/drop_at_least=0.3/default 2.33 239.6±1.98ms 244.6 KElem/sec 1.00 102.9±0.75ms 569.4 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024 1.00 60.3±1.16ms 972.3 KElem/sec 1.08 65.3±0.81ms 897.6 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048 1.00 60.8±1.14ms 964.3 KElem/sec 1.07 64.9±1.07ms 903.2 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256 1.00 61.0±1.99ms 960.9 KElem/sec 1.06 64.4±0.99ms 910.2 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512 1.00 60.6±1.45ms 966.9 KElem/sec 1.07 64.6±1.08ms 906.9 KElem/sec .../timeless_logs/drop_at_least=0.3/default 1.00 57.6±0.35ms 1018.1 KElem/sec 1.13 65.3±1.29ms 897.3 KElem/sec ``` --- Part of the GC improvements series: - #4394 - #4395 - #4396 - #4397 - #4398 - #4399 - #4400 - #4401
Optimize the creation of `StoreDiff`s and `StoreEvent`s, which turns out to be a major cost in time series use cases, when it is common to generate several millions of those on any single GC run. Once again some pretty significant wins. ### Benchmarks Compared to `main`: ``` group gc_improvements_0 gc_improvements_5 ----- ----------------- ----------------- .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024 13.00 1084.0±4.47ms 54.1 KElem/sec 1.00 83.4±1.16ms 702.9 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048 25.37 2.1±0.02s 27.6 KElem/sec 1.00 83.7±0.61ms 700.0 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256 5.55 465.8±2.50ms 125.8 KElem/sec 1.00 84.0±0.50ms 697.8 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512 7.94 655.3±2.61ms 89.4 KElem/sec 1.00 82.5±1.33ms 710.0 KElem/sec .../plotting_dashboard/drop_at_least=0.3/default 8.02 652.8±4.12ms 89.8 KElem/sec 1.00 81.4±0.94ms 720.0 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024 35.87 2.4±0.05s 24.2 KElem/sec 1.00 67.5±2.21ms 867.5 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048 35.91 2.4±0.03s 24.1 KElem/sec 1.00 67.8±1.86ms 863.9 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256 37.02 2.5±0.08s 23.5 KElem/sec 1.00 67.5±1.43ms 868.2 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512 35.47 2.4±0.02s 24.5 KElem/sec 1.00 67.4±1.40ms 869.4 KElem/sec .../timeless_logs/drop_at_least=0.3/default 36.00 2.4±0.03s 24.4 KElem/sec 1.00 66.8±0.85ms 877.3 KElem/sec ``` Compared to previous PR: ``` group gc_improvements_4 gc_improvements_5 ----- ----------------- ----------------- .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024 1.26 105.0±0.91ms 558.1 KElem/sec 1.00 83.4±1.16ms 702.9 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048 1.28 107.3±0.83ms 546.2 KElem/sec 1.00 83.7±0.61ms 700.0 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256 1.27 106.3±0.74ms 551.3 KElem/sec 1.00 84.0±0.50ms 697.8 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512 1.29 106.4±0.94ms 550.6 KElem/sec 1.00 82.5±1.33ms 710.0 KElem/sec .../plotting_dashboard/drop_at_least=0.3/default 1.26 102.9±0.75ms 569.4 KElem/sec 1.00 81.4±0.94ms 720.0 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024 1.00 65.3±0.81ms 897.6 KElem/sec 1.03 67.5±2.21ms 867.5 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048 1.00 64.9±1.07ms 903.2 KElem/sec 1.05 67.8±1.86ms 863.9 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256 1.00 64.4±0.99ms 910.2 KElem/sec 1.05 67.5±1.43ms 868.2 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512 1.00 64.6±1.08ms 906.9 KElem/sec 1.04 67.4±1.40ms 869.4 KElem/sec .../timeless_logs/drop_at_least=0.3/default 1.00 65.3±1.29ms 897.3 KElem/sec 1.02 66.8±0.85ms 877.3 KElem/sec ``` --- Part of the GC improvements series: - #4394 - #4395 - #4396 - #4397 - #4398 - #4399 - #4400 - #4401
Makes the GC capable of dropping entire buckets in one go when the conditions are met (and they are pretty simple to meet in the common case of in-order data). Unfortunately, I couldn't make the batched GC match -- let alone improve -- the performance of the standard GC. I even have a branch with a parallel batched GC, and it's still slower: the overhead of the batching datastructures just kills me everytime. For that reason, batching is disabled by default. I still want to commit the code so as to prevent it from rotting though, so we can come back to it at a later time. This introduces a slight performance deterioration on the non-batched path, that's fine. ### Benchmarks Compared to `main`: ``` group gc_improvements_0 gc_improvements_6 ----- ----------------- ----------------- .../plotting_dashboard/drop_at_least=0.3/default 7.62 652.8±4.12ms 89.8 KElem/sec 1.00 85.7±1.14ms 683.8 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256 5.34 465.8±2.50ms 125.8 KElem/sec 1.00 87.2±0.55ms 671.9 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512 7.12 655.3±2.61ms 89.4 KElem/sec 1.00 92.0±1.85ms 636.8 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024 12.45 1084.0±4.47ms 54.1 KElem/sec 1.00 87.1±0.40ms 672.7 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048 23.63 2.1±0.02s 27.6 KElem/sec 1.00 89.9±1.13ms 652.0 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256/gc_batching=true 1.00 98.6±0.82ms 594.2 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512/gc_batching=true 1.00 94.6±1.26ms 619.3 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024/gc_batching=true 1.00 93.2±1.29ms 628.9 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048/gc_batching=true 1.00 94.3±1.43ms 621.1 KElem/sec .../timeless_logs/drop_at_least=0.3/default 33.30 2.4±0.03s 24.4 KElem/sec 1.00 72.2±2.46ms 811.4 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256 35.16 2.5±0.08s 23.5 KElem/sec 1.00 71.1±2.31ms 824.5 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512 35.08 2.4±0.02s 24.5 KElem/sec 1.00 68.1±1.20ms 859.9 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024 36.86 2.4±0.05s 24.2 KElem/sec 1.00 65.7±0.87ms 891.4 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048 35.99 2.4±0.03s 24.1 KElem/sec 1.00 67.7±1.33ms 865.9 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256/gc_batching=true 1.00 68.7±1.40ms 853.1 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512/gc_batching=true 1.00 67.3±0.32ms 870.8 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024/gc_batching=true 1.00 67.7±1.21ms 865.2 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048/gc_batching=true 1.00 67.6±1.31ms 866.6 KElem/sec ``` Compared to previous PR: ``` group gc_improvements_5 gc_improvements_6 ----- ----------------- ----------------- .../plotting_dashboard/drop_at_least=0.3/default 1.00 81.4±0.94ms 720.0 KElem/sec 1.05 85.7±1.14ms 683.8 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256 1.00 84.0±0.50ms 697.8 KElem/sec 1.04 87.2±0.55ms 671.9 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512 1.00 82.5±1.33ms 710.0 KElem/sec 1.11 92.0±1.85ms 636.8 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024 1.00 83.4±1.16ms 702.9 KElem/sec 1.04 87.1±0.40ms 672.7 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048 1.00 83.7±0.61ms 700.0 KElem/sec 1.07 89.9±1.13ms 652.0 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=256/gc_batching=true 1.00 98.6±0.82ms 594.2 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=512/gc_batching=true 1.00 94.6±1.26ms 619.3 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=1024/gc_batching=true 1.00 93.2±1.29ms 628.9 KElem/sec .../plotting_dashboard/drop_at_least=0.3/bucketsz=2048/gc_batching=true 1.00 94.3±1.43ms 621.1 KElem/sec .../timeless_logs/drop_at_least=0.3/default 1.00 66.8±0.85ms 877.3 KElem/sec 1.08 72.2±2.46ms 811.4 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256 1.00 67.5±1.43ms 868.2 KElem/sec 1.05 71.1±2.31ms 824.5 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512 1.00 67.4±1.40ms 869.4 KElem/sec 1.01 68.1±1.20ms 859.9 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024 1.03 67.5±2.21ms 867.5 KElem/sec 1.00 65.7±0.87ms 891.4 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048 1.00 67.8±1.86ms 863.9 KElem/sec 1.00 67.7±1.33ms 865.9 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=1024/gc_batching=true 1.00 67.7±1.21ms 865.2 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=2048/gc_batching=true 1.00 67.6±1.31ms 866.6 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=256/gc_batching=true 1.00 68.7±1.40ms 853.1 KElem/sec .../timeless_logs/drop_at_least=0.3/bucketsz=512/gc_batching=true 1.00 67.3±0.32ms 870.8 KElem/sec ``` --- Part of the GC improvements series: - #4394 - #4395 - #4396 - #4397 - #4398 - #4399 - #4400 - #4401
Adds a configurable time bound to the GC, in addition to the pre-existing space bound. ```rust /// How long the garbage collection in allowed to run for. /// /// Trades off latency for throughput: /// - A smaller `time_budget` will clear less data in a shorter amount of time, allowing for a /// more responsive UI at the cost of more GC overhead and more frequent runs. /// - A larger `time_budget` will clear more data in a longer amount of time, increasing the /// chance of UI freeze frames but decreasing GC overhead and running less often. /// /// The default is an unbounded time budget (i.e. throughput only). pub time_budget: Duration, ``` No time budget: https://github.com/rerun-io/rerun/assets/2910679/8ca63aa3-5ad4-4575-9486-21d805026c1e 3.5ms budget: https://github.com/rerun-io/rerun/assets/2910679/e1bd1a41-6353-4a0e-90e5-8c05b76e92ea --- Part of the GC improvements series: - #4394 - #4395 - #4396 - #4397 - #4398 - #4399 - #4400 - #4401
* Uses: rerun-io/egui_tiles#67 * Uses: lampsitter/egui_commonmark#51 * Split off from #6171 * Closes #5280 ### Test * [x] image loading * [x] gltf ### Checklist * [x] I have read and agree to [Contributor Guide](https://github.com/rerun-io/rerun/blob/main/CONTRIBUTING.md) and the [Code of Conduct](https://github.com/rerun-io/rerun/blob/main/CODE_OF_CONDUCT.md) * [x] I've included a screenshot or gif (if applicable) * [x] I have tested the web demo (if applicable): * Using examples from latest `main` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6448?manifest_url=https://app.rerun.io/version/main/examples_manifest.json) * Using full set of examples from `nightly` build: [rerun.io/viewer](https://rerun.io/viewer/pr/6448?manifest_url=https://app.rerun.io/version/nightly/examples_manifest.json) * [x] The PR title and labels are set such as to maximize their usefulness for the next release's CHANGELOG * [x] If applicable, add a new check to the [release checklist](https://github.com/rerun-io/rerun/blob/main/tests/python/release_checklist)! - [PR Build Summary](https://build.rerun.io/pr/6448) - [Recent benchmark results](https://build.rerun.io/graphs/crates.html) - [Wasm size tracking](https://build.rerun.io/graphs/sizes.html) To run all checks from `main`, comment on the PR with `@rerun-bot full-check`. # egui changelog so far Full diff at https://github.com/emilk/egui/compare/0.27.0..HEAD #### ecolor * Fix `hex_color!` macro by re-exporting `color_hex` crate from `ecolor` [#4372](emilk/egui#4372) (thanks [@dataphract](https://github.com/dataphract)!) * Remove `extra_asserts` and `extra_debug_asserts` feature flags [#4478](emilk/egui#4478) #### eframe * Add `register_native_texture` in `eframe::Frame` [#4246](emilk/egui#4246) (thanks [@Chaojimengnan](https://github.com/Chaojimengnan)!) * Early-out from context switching the `glow` backend [#4284](emilk/egui#4284) * Fix `ViewportCommand::InnerSize` not resizing viewport on Wayland (#4211) [#4211](emilk/egui#4211) (thanks [@rustbasic](https://github.com/rustbasic)!) * Only avoid glow context switching on Windows [#4296](emilk/egui#4296) * Improve IME support with new `Event::Ime` [#4358](emilk/egui#4358) (thanks [@rustbasic](https://github.com/rustbasic)!) * Allow users to create viewports larger than monitor on Windows & macOS [#4337](emilk/egui#4337) (thanks [@lopo12123](https://github.com/lopo12123)!) * Use `objc2` and its framework crates [#4395](emilk/egui#4395) (thanks [@madsmtm](https://github.com/madsmtm)!) * Update to Rust 1.76 [#4411](emilk/egui#4411) * Egui-winit: emit physical key presses when a non-Latin layout is active [#4461](emilk/egui#4461) (thanks [@TicClick](https://github.com/TicClick)!) * IME for chinese [#4436](emilk/egui#4436) (thanks [@rustbasic](https://github.com/rustbasic)!) * Fix : In Windows, the 'egui_demo_app' screen does not appear. [#4410](emilk/egui#4410) (thanks [@rustbasic](https://github.com/rustbasic)!) * Fix: Window position creeps between executions on scaled monitors [#4443](emilk/egui#4443) (thanks [@avery-radmacher](https://github.com/avery-radmacher)!) * Update `image` crate to 0.25 [#4160](emilk/egui#4160) * Ignore synthetic key presses [#4514](emilk/egui#4514) (thanks [@hut](https://github.com/hut)!) * Fix: still track mouse when dragging outside web canvas [#4522](emilk/egui#4522) * Add `NativeOptions::persistence_path` [#4423](emilk/egui#4423) (thanks [@lucasmerlin](https://github.com/lucasmerlin)!) * Use ResizeObserver instead of `resize` event [#4536](emilk/egui#4536) (thanks [@jprochazk](https://github.com/jprochazk)!) * Fix: Don't `.forget()` RAF closure [#4551](emilk/egui#4551) (thanks [@jprochazk](https://github.com/jprochazk)!) #### egui_extras * Update `image` crate to 0.25 [#4160](emilk/egui#4160) #### egui_plot * `Plot::Items:allow_hover` give possibility to masked the interaction on hovered item [#2558](emilk/egui#2558) (thanks [@haricot](https://github.com/haricot)!) * Expose `ClosestElem` and `PlotConfig` [#4380](emilk/egui#4380) (thanks [@Narcha](https://github.com/Narcha)!) * Disable interaction for `ScrollArea` and `Plot` when UI is disabled [#4457](emilk/egui#4457) (thanks [@varphone](https://github.com/varphone)!) * Make sure plot size is positive [#4429](emilk/egui#4429) (thanks [@rustbasic](https://github.com/rustbasic)!) * Introduce lifetime to `egui_plot::Plot` to replace `'static` fields [#4435](emilk/egui#4435) (thanks [@Fabus1184](https://github.com/Fabus1184)!) * Hide all other series when alt-clicking in the legend [#4549](emilk/egui#4549) (thanks [@abey79](https://github.com/abey79)!) * Plot now respects the `interact_radius` set in the UI's style [#4520](emilk/egui#4520) (thanks [@YgorSouza](https://github.com/YgorSouza)!) #### egui_glow * Enable egui_glow's winit feature on wasm (#4420) [#4421](emilk/egui#4421) (thanks [@simon-frankau](https://github.com/simon-frankau)!) #### egui-wgpu * Update to wgpu 0.20 [#4433](emilk/egui#4433) (thanks [@KeKsBoTer](https://github.com/KeKsBoTer)!) * Revert update to wgpu 0.20 => downgrade to wgpu 0.19.1 [#4559](emilk/egui#4559) #### egui-winit * Update `webbrowser` to `v1.0.0` [#4394](emilk/egui#4394) (thanks [@torokati44](https://github.com/torokati44)!) * Emit physical key presses when a non-Latin layout is active [#4461](emilk/egui#4461) (thanks [@TicClick](https://github.com/TicClick)!) * IME for chinese [#4436](emilk/egui#4436) (thanks [@rustbasic](https://github.com/rustbasic)!) * Fix: Window position creeps between executions on scaled monitors [#4443](emilk/egui#4443) (thanks [@avery-radmacher](https://github.com/avery-radmacher)!) * Ignore synthetic key presses [#4514](emilk/egui#4514) (thanks [@hut](https://github.com/hut)!) #### egui * Improve the UI for changing the egui theme [#4257](emilk/egui#4257) * Change the resize cursor when you reach the resize limit [#4275](emilk/egui#4275) * Make `TextEdit` an atomic widget [#4276](emilk/egui#4276) * Overload operators for `Rect + Margin`, `Rect - Margin` etc [#4277](emilk/egui#4277) * Implement blinking text cursor in `TextEdit` [#4279](emilk/egui#4279) * Rename `fn scroll2` to `fn scroll` [#4282](emilk/egui#4282) * Change `Frame::multiply_with_opacity` to multiply in gamma space [#4283](emilk/egui#4283) * Support order on windows [#4301](emilk/egui#4301) (thanks [@alexparlett](https://github.com/alexparlett)!) * Fix wrong replacement function in deprecation notice of `drag_released*` [#4314](emilk/egui#4314) (thanks [@sornas](https://github.com/sornas)!) * Consider layer transform when positioning text agent [#4319](emilk/egui#4319) (thanks [@juancampa](https://github.com/juancampa)!) * Fix incorrect line breaks [#4377](emilk/egui#4377) (thanks [@juancampa](https://github.com/juancampa)!) * Fix `hex_color!` macro by re-exporting `color_hex` crate from `ecolor` [#4372](emilk/egui#4372) (thanks [@dataphract](https://github.com/dataphract)!) * Change `Ui::allocate_painter` to inherit properties from `Ui` [#4343](emilk/egui#4343) (thanks [@varphone](https://github.com/varphone)!) * Use parent `Ui`s style for popups [#4325](emilk/egui#4325) (thanks [@alexparlett](https://github.com/alexparlett)!) * Fix : take `rounding` into account when using `Slider::trailing_fill` [#4308](emilk/egui#4308) (thanks [@rustbasic](https://github.com/rustbasic)!) * Add a way to specify Undoer settings and construct Undoers more easily [#4357](emilk/egui#4357) (thanks [@valadaptive](https://github.com/valadaptive)!) * Add xtask crate [#4293](emilk/egui#4293) (thanks [@YgorSouza](https://github.com/YgorSouza)!) * Add `ViewportCommand::RequestCut`, `RequestCopy` and `RequestPaste` to trigger Clipboard actions [#4035](emilk/egui#4035) (thanks [@bu5hm4nn](https://github.com/bu5hm4nn)!) * Fix `Panel` incorrect size [#4351](emilk/egui#4351) (thanks [@zhatuokun](https://github.com/zhatuokun)!) * Improve IME support with new `Event::Ime` [#4358](emilk/egui#4358) (thanks [@rustbasic](https://github.com/rustbasic)!) * Allow users to create viewports larger than monitor on Windows & macOS [#4337](emilk/egui#4337) (thanks [@lopo12123](https://github.com/lopo12123)!) * Added ability to define colors at UV coordinates along a path [#4353](emilk/egui#4353) (thanks [@murl-digital](https://github.com/murl-digital)!) * Eframe: update ViewportBuilder.with_icon() documentation [#4408](emilk/egui#4408) (thanks [@roccoblues](https://github.com/roccoblues)!) * Update to Rust 1.76 [#4411](emilk/egui#4411) * Add a `Display` impl for `Vec2`, `Pos2`, and `Rect` [#4428](emilk/egui#4428) (thanks [@tgross35](https://github.com/tgross35)!) * Remove `extra_asserts` and `extra_debug_asserts` feature flags [#4478](emilk/egui#4478) * Egui-winit: emit physical key presses when a non-Latin layout is active [#4461](emilk/egui#4461) (thanks [@TicClick](https://github.com/TicClick)!) * Disable interaction for `ScrollArea` and `Plot` when UI is disabled [#4457](emilk/egui#4457) (thanks [@varphone](https://github.com/varphone)!) * Update ahash 0.8.6 -> 0.8.11 [#4507](emilk/egui#4507) (thanks [@hellodword](https://github.com/hellodword)!) * `include_image!` now accepts expressions [#4521](emilk/egui#4521) (thanks [@YgorSouza](https://github.com/YgorSouza)!) * Remove `Event::Scroll` and handle it in egui [#4524](emilk/egui#4524) * Remove scroll latency for smooth trackpads [#4526](emilk/egui#4526) * Smooth out zooming with discreet scroll wheel [#4530](emilk/egui#4530) * Add `Options::line_scroll_speed` and `scroll_zoom_speed` [#4532](emilk/egui#4532) * Don't panic when replacement glyph is not found [#4542](emilk/egui#4542) (thanks [@RyanBluth](https://github.com/RyanBluth)!) * Make `TextEdit::return_key` optional [#4543](emilk/egui#4543) (thanks [@doonv](https://github.com/doonv)!) * Add `TextEdit::hint_text_font` [#4517](emilk/egui#4517) (thanks [@zaaarf](https://github.com/zaaarf)!) * Add `Options::reduce_texture_memory` to free up RAM [#4431](emilk/egui#4431) (thanks [@varphone](https://github.com/varphone)!) * Fix `Ui::scroll_with_delta` only scrolling if the `ScrollArea` is focused [#4303](emilk/egui#4303) (thanks [@lucasmerlin](https://github.com/lucasmerlin)!) * Add support for text truncation to `egui::Style` [#4556](emilk/egui#4556) (thanks [@abey79](https://github.com/abey79)!) * Hide toolip when opening `ComboBox` drop-down [#4546](emilk/egui#4546) (thanks [@abey79](https://github.com/abey79)!) * Better spacing and sizes for (menu) buttons [#4558](emilk/egui#4558) #### epaint * Add `RectShape::blur_width` to implement shadows [#4267](emilk/egui#4267) * Overload operators for `Rect + Margin`, `Rect - Margin` etc [#4277](emilk/egui#4277) * Fix incorrect line breaks [#4377](emilk/egui#4377) (thanks [@juancampa](https://github.com/juancampa)!) * Fix `hex_color!` macro by re-exporting `color_hex` crate from `ecolor` [#4372](emilk/egui#4372) (thanks [@dataphract](https://github.com/dataphract)!) * Add `emath::OrderedFloat` (moved from `epaint::util::OrderedFloat`) [#4389](emilk/egui#4389) * Added ability to define colors at UV coordinates along a path [#4353](emilk/egui#4353) (thanks [@murl-digital](https://github.com/murl-digital)!) * Add a `Display` impl for `Vec2`, `Pos2`, and `Rect` [#4428](emilk/egui#4428) (thanks [@tgross35](https://github.com/tgross35)!) * Remove `extra_asserts` and `extra_debug_asserts` feature flags [#4478](emilk/egui#4478) * Make `epaint::mutex::RwLock` allow `?Sized` types [#4485](emilk/egui#4485) (thanks [@crumblingstatue](https://github.com/crumblingstatue)!) * Update ahash 0.8.6 -> 0.8.11 [#4507](emilk/egui#4507) (thanks [@hellodword](https://github.com/hellodword)!) * Don't panic when replacement glyph is not found [#4542](emilk/egui#4542) (thanks [@RyanBluth](https://github.com/RyanBluth)!) --------- Co-authored-by: Antoine Beyeler <antoine@rerun.io> Co-authored-by: Andreas Reich <r_andreas2@web.de>
Fixes a long-standing bug: timeless tables not being sorted by
RowId
, which means they effectively always return incorrect results for out-of-order data (yes, that is a thing even in a timeless context).This worsens GC performance for timeless tables, but:
RowId
-based #1807Benchmarks
Hint: it's even worse!
Part of the GC improvements series:
RowId
-ordered #4395VecDeque
extensions & benchmarks #4396EntityPathHash
es in metadata registry #4398Store{Diff,Event}
optimizations #4399time_budget
GC setting #4401Checklist