Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance of the index selector #493

Merged
merged 3 commits into from
Mar 15, 2024
Merged

Conversation

V0ldek
Copy link
Member

@V0ldek V0ldek commented Mar 15, 2024

Short description

Improved the access patterns for array index selectors.

First, member and array transitions are now in separate smallvec arrays, so we can always look only at the relevant array transitions.

Second, properties like "has array transition" or "can accept via array transition" are now cached during minimize as StateAttributes.

Overall on chirop@lille this gives from 7% to 12% thpt increase on most of the main benchmark.
See full bench report at the bottom.

Issue

Resolves: #138
Rel: #151

Checklist

All of these should be ticked off before you submit the PR.

  • I ran just verify locally and it succeeded.
  • Issue was given go ahead and is linked above OR I have included justification for a minor change.
  • Unit tests for my changes are included OR no functionality was changed.

Benchmark report

There is a weird regression on ./data/twitter/twitter.json_user_mentions_indices/rsonpath_mmap_count which should be investigated as part of #151. However, the change looks reasonably beneficial on all other benches.

./data/nativejson/canada.json_canada::second_coord_component/rsonpath
                        time:   [5.0144 ms 5.0161 ms 5.0178 ms]
                        thrpt:  [448.61 MB/s 448.77 MB/s 448.92 MB/s]
                 change:
                        time:   [-5.7562% -5.7026% -5.6470%] (p = 0.00 < 0.05)
                        thrpt:  [+5.9850% +6.0474% +6.1078%]
                        Performance has improved.
./data/nativejson/canada.json_canada::second_coord_component/rsonpath_count
                        time:   [2.2977 ms 2.2989 ms 2.3004 ms]
                        thrpt:  [978.57 MB/s 979.18 MB/s 979.69 MB/s]
                 change:
                        time:   [-6.9102% -6.7433% -6.4948%] (p = 0.00 < 0.05)
                        thrpt:  [+6.9460% +7.2308% +7.4232%]
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
./data/nativejson/canada.json_canada::second_coord_component/rsonpath_mmap
                        time:   [5.1043 ms 5.1071 ms 5.1144 ms]
                        thrpt:  [440.14 MB/s 440.77 MB/s 441.01 MB/s]
                 change:
                        time:   [-7.7144% -7.6213% -7.5149%] (p = 0.00 < 0.05)
                        thrpt:  [+8.1256% +8.2500% +8.3593%]
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
./data/nativejson/canada.json_canada::second_coord_component/rsonpath_mmap_count
                        time:   [2.3352 ms 2.3417 ms 2.3467 ms]
                        thrpt:  [959.22 MB/s 961.29 MB/s 963.96 MB/s]
                 change:
                        time:   [-11.937% -11.472% -11.059%] (p = 0.00 < 0.05)
                        thrpt:  [+12.434% +12.958% +13.555%]
                        Performance has improved.

./data/nativejson/canada.json_canada::coord_476_1446_1/rsonpath
                        time:   [1.5509 ms 1.5513 ms 1.5518 ms]
                        thrpt:  [1.4506 GB/s 1.4510 GB/s 1.4515 GB/s]
                 change:
                        time:   [-11.986% -11.870% -11.763%] (p = 0.00 < 0.05)
                        thrpt:  [+13.332% +13.469% +13.618%]
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
./data/nativejson/canada.json_canada::coord_476_1446_1/rsonpath_count
                        time:   [1.2077 ms 1.2155 ms 1.2213 ms]
                        thrpt:  [1.8431 GB/s 1.8519 GB/s 1.8639 GB/s]
                 change:
                        time:   [-12.584% -12.304% -11.970%] (p = 0.00 < 0.05)
                        thrpt:  [+13.598% +14.031% +14.396%]
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
./data/nativejson/canada.json_canada::coord_476_1446_1/rsonpath_mmap
                        time:   [1.6201 ms 1.6213 ms 1.6226 ms]
                        thrpt:  [1.3874 GB/s 1.3884 GB/s 1.3894 GB/s]
                 change:
                        time:   [-11.730% -11.653% -11.571%] (p = 0.00 < 0.05)
                        thrpt:  [+13.085% +13.190% +13.289%]
                        Performance has improved.
./data/nativejson/canada.json_canada::coord_476_1446_1/rsonpath_mmap_count
                        time:   [1.1908 ms 1.1921 ms 1.1931 ms]
                        thrpt:  [1.8868 GB/s 1.8883 GB/s 1.8903 GB/s]
                 change:
                        time:   [-15.611% -15.479% -15.355%] (p = 0.00 < 0.05)
                        thrpt:  [+18.140% +18.314% +18.498%]
                        Performance has improved.
Found 2 outliers among 10 measurements (20.00%)
  1 (10.00%) low mild
  1 (10.00%) high mild

./data/nativejson/citm.json_citm::seatCategoryId/rsonpath
                        time:   [375.17 µs 375.32 µs 375.43 µs]
                        thrpt:  [4.6007 GB/s 4.6019 GB/s 4.6037 GB/s]
                 change:
                        time:   [-3.8983% -3.1712% -2.4736%] (p = 0.00 < 0.05)
                        thrpt:  [+2.5363% +3.2751% +4.0564%]
                        Performance has improved.
./data/nativejson/citm.json_citm::seatCategoryId/rsonpath_count
                        time:   [233.44 µs 235.26 µs 237.55 µs]
                        thrpt:  [7.2708 GB/s 7.3418 GB/s 7.3989 GB/s]
                 change:
                        time:   [-16.799% -15.243% -13.734%] (p = 0.00 < 0.05)
                        thrpt:  [+15.920% +17.984% +20.191%]
                        Performance has improved.
Found 2 outliers among 10 measurements (20.00%)
  2 (20.00%) high mild
./data/nativejson/citm.json_citm::seatCategoryId/rsonpath_mmap
                        time:   [355.23 µs 356.24 µs 357.78 µs]
                        thrpt:  [4.8275 GB/s 4.8485 GB/s 4.8622 GB/s]
                 change:
                        time:   [+0.7732% +1.3615% +1.9851%] (p = 0.00 < 0.05)
                        thrpt:  [-1.9464% -1.3432% -0.7673%]
                        Change within noise threshold.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
./data/nativejson/citm.json_citm::seatCategoryId/rsonpath_mmap_count
                        time:   [218.55 µs 218.87 µs 219.34 µs]
                        thrpt:  [7.8747 GB/s 7.8916 GB/s 7.9030 GB/s]
                 change:
                        time:   [+1.5263% +1.7435% +1.9861%] (p = 0.00 < 0.05)
                        thrpt:  [-1.9475% -1.7137% -1.5033%]
                        Performance has regressed.

./data/ast/ast.json_ast::nested_inner/rsonpath
                        time:   [28.864 ms 28.876 ms 28.888 ms]
                        thrpt:  [885.95 MB/s 886.32 MB/s 886.68 MB/s]
                 change:
                        time:   [-2.9534% -2.8978% -2.8412%] (p = 0.00 < 0.05)
                        thrpt:  [+2.9242% +2.9843% +3.0432%]
                        Performance has improved.
Found 3 outliers among 10 measurements (30.00%)
  1 (10.00%) low mild
  2 (20.00%) high mild
./data/ast/ast.json_ast::nested_inner/rsonpath_count
                        time:   [20.616 ms 20.626 ms 20.637 ms]
                        thrpt:  [1.2402 GB/s 1.2408 GB/s 1.2414 GB/s]
                 change:
                        time:   [-3.9317% -3.7078% -3.5175%] (p = 0.00 < 0.05)
                        thrpt:  [+3.6458% +3.8505% +4.0926%]
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
./data/ast/ast.json_ast::nested_inner/rsonpath_mmap
                        time:   [29.178 ms 29.187 ms 29.196 ms]
                        thrpt:  [876.61 MB/s 876.87 MB/s 877.13 MB/s]
                 change:
                        time:   [-4.0546% -4.0203% -3.9910%] (p = 0.00 < 0.05)
                        thrpt:  [+4.1569% +4.1887% +4.2260%]
                        Performance has improved.
./data/ast/ast.json_ast::nested_inner/rsonpath_mmap_count
                        time:   [20.391 ms 20.416 ms 20.435 ms]
                        thrpt:  [1.2524 GB/s 1.2536 GB/s 1.2551 GB/s]
                 change:
                        time:   [-8.6349% -8.1167% -7.7519%] (p = 0.00 < 0.05)
                        thrpt:  [+8.4034% +8.8337% +9.4510%]
                        Performance has improved.
Found 2 outliers among 10 measurements (20.00%)
  1 (10.00%) low mild
  1 (10.00%) high mild

./data/ast/ast.json_ast::deepest/rsonpath
                        time:   [17.790 ms 17.795 ms 17.806 ms]
                        thrpt:  [1.4373 GB/s 1.4382 GB/s 1.4387 GB/s]
                 change:
                        time:   [+0.0534% +0.1436% +0.2409%] (p = 0.01 < 0.05)
                        thrpt:  [-0.2403% -0.1434% -0.0533%]
                        Change within noise threshold.
./data/ast/ast.json_ast::deepest/rsonpath_count
                        time:   [13.634 ms 13.652 ms 13.676 ms]
                        thrpt:  [1.8714 GB/s 1.8747 GB/s 1.8771 GB/s]
                 change:
                        time:   [+1.0579% +1.2534% +1.4533%] (p = 0.00 < 0.05)
                        thrpt:  [-1.4325% -1.2379% -1.0468%]
                        Performance has regressed.
./data/ast/ast.json_ast::deepest/rsonpath_mmap
                        time:   [17.880 ms 17.891 ms 17.902 ms]
                        thrpt:  [1.4296 GB/s 1.4305 GB/s 1.4314 GB/s]
                 change:
                        time:   [-1.6591% -1.6058% -1.5529%] (p = 0.00 < 0.05)
                        thrpt:  [+1.5774% +1.6320% +1.6870%]
                        Performance has improved.
./data/ast/ast.json_ast::deepest/rsonpath_mmap_count
                        time:   [13.273 ms 13.323 ms 13.411 ms]
                        thrpt:  [1.9083 GB/s 1.9210 GB/s 1.9282 GB/s]
                 change:
                        time:   [-2.1778% -1.7634% -1.3030%] (p = 0.00 < 0.05)
                        thrpt:  [+1.3202% +1.7951% +2.2263%]
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

./data/pison/bestbuy_short_record.json_bestbuy::products_category/rsonpath
                        time:   [34.178 ms 34.222 ms 34.271 ms]
                        thrpt:  [3.0122 GB/s 3.0166 GB/s 3.0204 GB/s]
                 change:
                        time:   [+1.4234% +1.5005% +1.5818%] (p = 0.00 < 0.05)
                        thrpt:  [-1.5572% -1.4784% -1.4035%]
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
./data/pison/bestbuy_short_record.json_bestbuy::products_category/rsonpath_count
                        time:   [26.736 ms 27.117 ms 28.026 ms]
                        thrpt:  [3.6834 GB/s 3.8069 GB/s 3.8612 GB/s]
                 change:
                        time:   [-4.8178% -1.5078% +1.1216%] (p = 0.36 > 0.05)
                        thrpt:  [-1.1091% +1.5309% +5.0616%]
                        No change in performance detected.
./data/pison/bestbuy_short_record.json_bestbuy::products_category/rsonpath_mmap
                        time:   [34.782 ms 34.791 ms 34.810 ms]
                        thrpt:  [2.9656 GB/s 2.9672 GB/s 2.9679 GB/s]
                 change:
                        time:   [+1.3360% +1.4133% +1.5023%] (p = 0.00 < 0.05)
                        thrpt:  [-1.4801% -1.3936% -1.3183%]
                        Performance has regressed.
./data/pison/bestbuy_short_record.json_bestbuy::products_category/rsonpath_mmap_count
                        time:   [28.898 ms 29.015 ms 29.126 ms]
                        thrpt:  [3.5443 GB/s 3.5579 GB/s 3.5722 GB/s]
                 change:
                        time:   [+1.4799% +1.8876% +2.3335%] (p = 0.00 < 0.05)
                        thrpt:  [-2.2803% -1.8526% -1.4583%]
                        Performance has regressed.

./data/pison/bestbuy_short_record.json_bestbuy::products_video_only/rsonpath_direct_count
                        time:   [109.04 ms 109.07 ms 109.09 ms]
                        thrpt:  [946.33 MB/s 946.50 MB/s 946.71 MB/s]
                 change:
                        time:   [-8.5134% -8.4821% -8.4559%] (p = 0.00 < 0.05)
                        thrpt:  [+9.2370% +9.2683% +9.3056%]
                        Performance has improved.
./data/pison/bestbuy_short_record.json_bestbuy::products_video_only/rsonpath_descendant_count
                        time:   [9.5149 ms 9.5809 ms 9.6211 ms]
                        thrpt:  [10.730 GB/s 10.775 GB/s 10.850 GB/s]
                 change:
                        time:   [-2.3945% -1.9227% -1.4300%] (p = 0.00 < 0.05)
                        thrpt:  [+1.4508% +1.9604% +2.4533%]
                        Performance has improved.
./data/pison/bestbuy_short_record.json_bestbuy::products_video_only/rsonpath_direct_nodes
                        time:   [135.55 ms 135.62 ms 135.67 ms]
                        thrpt:  [760.92 MB/s 761.17 MB/s 761.57 MB/s]
                 change:
                        time:   [-4.2566% -4.1786% -4.1099%] (p = 0.00 < 0.05)
                        thrpt:  [+4.2860% +4.3608% +4.4458%]
                        Performance has improved.
./data/pison/bestbuy_short_record.json_bestbuy::products_video_only/rsonpath_descendant_nodes
                        time:   [12.185 ms 12.193 ms 12.206 ms]
                        thrpt:  [8.4577 GB/s 8.4666 GB/s 8.4723 GB/s]
                 change:
                        time:   [-6.1811% -5.7272% -5.2631%] (p = 0.00 < 0.05)
                        thrpt:  [+5.5555% +6.0751% +6.5884%]
                        Performance has improved.

./data/pison/bestbuy_short_record.json_bestbuy::all_nodes/rsonpath
                        time:   [691.68 ms 699.48 ms 709.60 ms]
                        thrpt:  [145.48 MB/s 147.58 MB/s 149.25 MB/s]
                 change:
                        time:   [-2.5324% -0.9290% +0.9107%] (p = 0.33 > 0.05)
                        thrpt:  [-0.9024% +0.9377% +2.5982%]
                        No change in performance detected.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
./data/pison/bestbuy_short_record.json_bestbuy::all_nodes/rsonpath_count
                        time:   [87.599 ms 87.647 ms 87.709 ms]
                        thrpt:  [1.1770 GB/s 1.1778 GB/s 1.1785 GB/s]
                 change:
                        time:   [-2.8238% -2.7557% -2.6838%] (p = 0.00 < 0.05)
                        thrpt:  [+2.7578% +2.8338% +2.9059%]
                        Performance has improved.
./data/pison/bestbuy_short_record.json_bestbuy::all_nodes/rsonpath_mmap
                        time:   [705.00 ms 708.70 ms 712.45 ms]
                        thrpt:  [144.90 MB/s 145.66 MB/s 146.43 MB/s]
                 change:
                        time:   [-0.6276% +0.3405% +1.2947%] (p = 0.52 > 0.05)
                        thrpt:  [-1.2782% -0.3394% +0.6316%]
                        No change in performance detected.
./data/pison/bestbuy_short_record.json_bestbuy::all_nodes/rsonpath_mmap_count
                        time:   [94.240 ms 94.306 ms 94.371 ms]
                        thrpt:  [1.0939 GB/s 1.0946 GB/s 1.0954 GB/s]
                 change:
                        time:   [-3.4564% -3.3477% -3.2443%] (p = 0.00 < 0.05)
                        thrpt:  [+3.3531% +3.4637% +3.5801%]
                        Performance has improved.

./data/pison/google_map_short_record.json_google_map::routes/rsonpath
                        time:   [60.286 ms 60.296 ms 60.312 ms]
                        thrpt:  [1.7724 GB/s 1.7729 GB/s 1.7732 GB/s]
                 change:
                        time:   [-2.4644% -2.4199% -2.3758%] (p = 0.00 < 0.05)
                        thrpt:  [+2.4336% +2.4799% +2.5267%]
                        Performance has improved.
./data/pison/google_map_short_record.json_google_map::routes/rsonpath_count
                        time:   [48.933 ms 48.960 ms 48.981 ms]
                        thrpt:  [2.1824 GB/s 2.1833 GB/s 2.1846 GB/s]
                 change:
                        time:   [-3.7676% -3.1628% -2.6783%] (p = 0.00 < 0.05)
                        thrpt:  [+2.7520% +3.2661% +3.9151%]
                        Performance has improved.
Found 2 outliers among 10 measurements (20.00%)
  2 (20.00%) high mild
./data/pison/google_map_short_record.json_google_map::routes/rsonpath_mmap
                        time:   [60.411 ms 60.429 ms 60.441 ms]
                        thrpt:  [1.7686 GB/s 1.7690 GB/s 1.7695 GB/s]
                 change:
                        time:   [-4.9174% -4.8854% -4.8553%] (p = 0.00 < 0.05)
                        thrpt:  [+5.1031% +5.1364% +5.1717%]
                        Performance has improved.
./data/pison/google_map_short_record.json_google_map::routes/rsonpath_mmap_count
                        time:   [48.279 ms 48.293 ms 48.321 ms]
                        thrpt:  [2.2122 GB/s 2.2135 GB/s 2.2141 GB/s]
                 change:
                        time:   [-5.1179% -5.0657% -5.0165%] (p = 0.00 < 0.05)
                        thrpt:  [+5.2814% +5.3360% +5.3940%]
                        Performance has improved.

./data/pison/google_map_short_record.json_google_map::travel_modes/rsonpath_direct_count
                        time:   [16.981 ms 17.003 ms 17.024 ms]
                        thrpt:  [6.2790 GB/s 6.2869 GB/s 6.2953 GB/s]
                 change:
                        time:   [-1.6973% -1.5067% -1.3452%] (p = 0.00 < 0.05)
                        thrpt:  [+1.3636% +1.5297% +1.7266%]
                        Performance has improved.
./data/pison/google_map_short_record.json_google_map::travel_modes/rsonpath_descendant_count
                        time:   [9.9101 ms 9.9382 ms 9.9941 ms]
                        thrpt:  [10.696 GB/s 10.756 GB/s 10.787 GB/s]
                 change:
                        time:   [-10.888% -10.255% -9.7726%] (p = 0.00 < 0.05)
                        thrpt:  [+10.831% +11.427% +12.218%]
                        Performance has improved.
./data/pison/google_map_short_record.json_google_map::travel_modes/rsonpath_direct_nodes
                        time:   [18.157 ms 18.177 ms 18.206 ms]
                        thrpt:  [5.8716 GB/s 5.8810 GB/s 5.8874 GB/s]
                 change:
                        time:   [-1.4163% -1.3285% -1.2425%] (p = 0.00 < 0.05)
                        thrpt:  [+1.2581% +1.3464% +1.4367%]
                        Performance has improved.
./data/pison/google_map_short_record.json_google_map::travel_modes/rsonpath_descendant_nodes
                        time:   [12.299 ms 12.301 ms 12.303 ms]
                        thrpt:  [8.6883 GB/s 8.6903 GB/s 8.6917 GB/s]
                 change:
                        time:   [-1.6944% -1.3742% -1.1106%] (p = 0.00 < 0.05)
                        thrpt:  [+1.1231% +1.3934% +1.7236%]
                        Performance has improved.

./data/ast/ast.json_inner_array/rsonpath
                        time:   [57.144 ms 57.256 ms 57.330 ms]
                        thrpt:  [446.42 MB/s 447.00 MB/s 447.87 MB/s]
                 change:
                        time:   [-5.4647% -5.2995% -5.1376%] (p = 0.00 < 0.05)
                        thrpt:  [+5.4159% +5.5960% +5.7806%]
                        Performance has improved.
./data/ast/ast.json_inner_array/rsonpath_count
                        time:   [17.636 ms 17.646 ms 17.655 ms]
                        thrpt:  [1.4496 GB/s 1.4504 GB/s 1.4512 GB/s]
                 change:
                        time:   [-8.8154% -8.7135% -8.6176%] (p = 0.00 < 0.05)
                        thrpt:  [+9.4302% +9.5452% +9.6676%]
                        Performance has improved.
./data/ast/ast.json_inner_array/rsonpath_mmap
                        time:   [58.707 ms 59.534 ms 60.543 ms]
                        thrpt:  [422.73 MB/s 429.89 MB/s 435.95 MB/s]
                 change:
                        time:   [-4.1012% -2.8027% -1.4657%] (p = 0.00 < 0.05)
                        thrpt:  [+1.4875% +2.8836% +4.2765%]
                        Performance has improved.
Found 2 outliers among 10 measurements (20.00%)
  2 (20.00%) high severe
./data/ast/ast.json_inner_array/rsonpath_mmap_count
                        time:   [17.074 ms 17.084 ms 17.090 ms]
                        thrpt:  [1.4975 GB/s 1.4981 GB/s 1.4990 GB/s]
                 change:
                        time:   [-12.367% -12.304% -12.224%] (p = 0.00 < 0.05)
                        thrpt:  [+13.927% +14.030% +14.113%]
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild

./data/twitter/twitter.json_user_mentions_indices/rsonpath
                        time:   [161.30 µs 161.58 µs 162.12 µs]
                        thrpt:  [4.7330 GB/s 4.7486 GB/s 4.7571 GB/s]
                 change:
                        time:   [-9.8309% -9.1361% -8.5100%] (p = 0.00 < 0.05)
                        thrpt:  [+9.3015% +10.055% +10.903%]
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  9 (9.00%) high mild
  3 (3.00%) high severe
./data/twitter/twitter.json_user_mentions_indices/rsonpath_count
                        time:   [134.68 µs 135.88 µs 137.15 µs]
                        thrpt:  [5.5947 GB/s 5.6467 GB/s 5.6971 GB/s]
                 change:
                        time:   [-6.7577% -5.8580% -4.8230%] (p = 0.00 < 0.05)
                        thrpt:  [+5.0674% +6.2225% +7.2475%]
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
./data/twitter/twitter.json_user_mentions_indices/rsonpath_mmap
                        time:   [160.25 µs 160.64 µs 161.12 µs]
                        thrpt:  [4.7622 GB/s 4.7766 GB/s 4.7881 GB/s]
                 change:
                        time:   [-3.9817% -2.9742% -1.5994%] (p = 0.00 < 0.05)
                        thrpt:  [+1.6254% +3.0653% +4.1468%]
                        Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
  2 (2.00%) low mild
  3 (3.00%) high mild
  10 (10.00%) high severe
./data/twitter/twitter.json_user_mentions_indices/rsonpath_mmap_count
                        time:   [173.79 µs 174.72 µs 175.59 µs]
                        thrpt:  [4.3699 GB/s 4.3916 GB/s 4.4151 GB/s]
                 change:
                        time:   [+25.393% +27.325% +28.926%] (p = 0.00 < 0.05)
                        thrpt:  [-22.436% -21.461% -20.251%]
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) low severe
  6 (6.00%) low mild

./data/pison/walmart_short_record.json_walmart::items_name/rsonpath_direct_count
                        time:   [18.427 ms 18.535 ms 18.604 ms]
                        thrpt:  [5.1218 GB/s 5.1409 GB/s 5.1709 GB/s]
                 change:
                        time:   [+8.1211% +8.5307% +8.9897%] (p = 0.00 < 0.05)
                        thrpt:  [-8.2482% -7.8602% -7.5111%]
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
./data/pison/walmart_short_record.json_walmart::items_name/rsonpath_descendant_count
                        time:   [9.9934 ms 10.042 ms 10.110 ms]
                        thrpt:  [9.4242 GB/s 9.4882 GB/s 9.5347 GB/s]
                 change:
                        time:   [-3.6318% -3.1427% -2.6831%] (p = 0.00 < 0.05)
                        thrpt:  [+2.7571% +3.2447% +3.7687%]
                        Performance has improved.
Found 3 outliers among 10 measurements (30.00%)
  2 (20.00%) low severe
  1 (10.00%) high mild
./data/pison/walmart_short_record.json_walmart::items_name/rsonpath_direct_nodes
                        time:   [21.521 ms 21.563 ms 21.620 ms]
                        thrpt:  [4.4071 GB/s 4.4189 GB/s 4.4274 GB/s]
                 change:
                        time:   [-1.3653% -1.0551% -0.7977%] (p = 0.00 < 0.05)
                        thrpt:  [+0.8041% +1.0664% +1.3841%]
                        Change within noise threshold.
Found 2 outliers among 10 measurements (20.00%)
  1 (10.00%) low severe
  1 (10.00%) low mild
./data/pison/walmart_short_record.json_walmart::items_name/rsonpath_descendant_nodes
                        time:   [12.366 ms 12.398 ms 12.464 ms]
                        thrpt:  [7.6447 GB/s 7.6851 GB/s 7.7055 GB/s]
                 change:
                        time:   [-0.2497% +0.0079% +0.2481%] (p = 0.96 > 0.05)
                        thrpt:  [-0.2475% -0.0079% +0.2503%]
                        No change in performance detected.

./data/twitter/twitter.json_twitter::metadata/rsonpath_direct_count
                        time:   [91.214 µs 91.317 µs 91.493 µs]
                        thrpt:  [8.3864 GB/s 8.4025 GB/s 8.4120 GB/s]
                 change:
                        time:   [+0.5776% +0.6478% +0.7364%] (p = 0.00 < 0.05)
                        thrpt:  [-0.7310% -0.6437% -0.5743%]
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe
./data/twitter/twitter.json_twitter::metadata/rsonpath_descendant_count
                        time:   [52.897 µs 52.920 µs 52.962 µs]
                        thrpt:  [14.488 GB/s 14.499 GB/s 14.506 GB/s]
                 change:
                        time:   [+0.1294% +0.2235% +0.2983%] (p = 0.00 < 0.05)
                        thrpt:  [-0.2974% -0.2230% -0.1292%]
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe
./data/twitter/twitter.json_twitter::metadata/rsonpath_direct_nodes
                        time:   [102.01 µs 102.05 µs 102.09 µs]
                        thrpt:  [7.5161 GB/s 7.5192 GB/s 7.5215 GB/s]
                 change:
                        time:   [+0.0891% +0.1242% +0.1677%] (p = 0.00 < 0.05)
                        thrpt:  [-0.1674% -0.1241% -0.0890%]
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  7 (7.00%) high mild
  2 (2.00%) high severe
./data/twitter/twitter.json_twitter::metadata/rsonpath_descendant_nodes
                        time:   [74.357 µs 74.431 µs 74.521 µs]
                        thrpt:  [10.296 GB/s 10.309 GB/s 10.319 GB/s]
                 change:
                        time:   [-0.3190% -0.1850% -0.0455%] (p = 0.01 < 0.05)
                        thrpt:  [+0.0455% +0.1853% +0.3200%]
                        Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe

./data/twitter/twitter.json_all_first_index/rsonpath
                        time:   [724.08 µs 724.25 µs 724.50 µs]
                        thrpt:  [1.0591 GB/s 1.0594 GB/s 1.0597 GB/s]
                 change:
                        time:   [-7.3278% -7.2660% -7.2140%] (p = 0.00 < 0.05)
                        thrpt:  [+7.7749% +7.8353% +7.9073%]
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  4 (4.00%) high mild
  2 (2.00%) high severe
./data/twitter/twitter.json_all_first_index/rsonpath_count
                        time:   [507.35 µs 507.44 µs 507.54 µs]
                        thrpt:  [1.5118 GB/s 1.5121 GB/s 1.5124 GB/s]
                 change:
                        time:   [-8.2887% -7.0478% -6.3786%] (p = 0.00 < 0.05)
                        thrpt:  [+6.8132% +7.5822% +9.0378%]
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high severe
./data/twitter/twitter.json_all_first_index/rsonpath_mmap
                        time:   [678.50 µs 678.89 µs 679.33 µs]
                        thrpt:  [1.1295 GB/s 1.1302 GB/s 1.1309 GB/s]
                 change:
                        time:   [-8.3701% -8.3077% -8.2431%] (p = 0.00 < 0.05)
                        thrpt:  [+8.9837% +9.0604% +9.1347%]
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe
./data/twitter/twitter.json_all_first_index/rsonpath_mmap_count
                        time:   [494.52 µs 494.62 µs 494.74 µs]
                        thrpt:  [1.5509 GB/s 1.5513 GB/s 1.5516 GB/s]
                 change:
                        time:   [-7.0028% -6.9334% -6.8719%] (p = 0.00 < 0.05)
                        thrpt:  [+7.3790% +7.4499% +7.5302%]
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

     Running benches/main_micro.rs (target/release/deps/main_micro-1331e359774e7db8)
./data/small/az_tenants.json_az_tenant::shallow_ids/rsonpath
                        time:   [43.159 µs 43.167 µs 43.175 µs]
                        thrpt:  [874.18 MB/s 874.34 MB/s 874.51 MB/s]
                 change:
                        time:   [+0.2612% +0.3179% +0.3653%] (p = 0.00 < 0.05)
                        thrpt:  [-0.3639% -0.3169% -0.2605%]
                        Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) low mild
  1 (1.00%) high mild
./data/small/az_tenants.json_az_tenant::shallow_ids/rsonpath_count
                        time:   [31.997 µs 32.001 µs 32.006 µs]
                        thrpt:  [1.1792 GB/s 1.1794 GB/s 1.1796 GB/s]
                 change:
                        time:   [-5.4801% -5.4514% -5.4244%] (p = 0.00 < 0.05)
                        thrpt:  [+5.7355% +5.7657% +5.7978%]
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe
./data/small/az_tenants.json_az_tenant::shallow_ids/rsonpath_mmap
                        time:   [42.456 µs 42.531 µs 42.615 µs]
                        thrpt:  [885.67 MB/s 887.43 MB/s 888.99 MB/s]
                 change:
                        time:   [-7.1473% -6.9907% -6.7334%] (p = 0.00 < 0.05)
                        thrpt:  [+7.2196% +7.5161% +7.6975%]
                        Performance has improved.
Found 15 outliers among 100 measurements (15.00%)
  8 (8.00%) low severe
  1 (1.00%) low mild
  2 (2.00%) high mild
  4 (4.00%) high severe
./data/small/az_tenants.json_az_tenant::shallow_ids/rsonpath_mmap_count
                        time:   [30.896 µs 30.903 µs 30.911 µs]
                        thrpt:  [1.2210 GB/s 1.2213 GB/s 1.2216 GB/s]
                 change:
                        time:   [-5.1662% -5.0628% -4.9991%] (p = 0.00 < 0.05)
                        thrpt:  [+5.2621% +5.3328% +5.4476%]
                        Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
  2 (2.00%) high mild
  4 (4.00%) high severe

./data/small/az_tenants.json_az_tenants::recursive_ids/rsonpath
                        time:   [18.098 µs 18.105 µs 18.115 µs]
                        thrpt:  [2.0836 GB/s 2.0847 GB/s 2.0855 GB/s]
                 change:
                        time:   [-6.0036% -5.0050% -4.2824%] (p = 0.00 < 0.05)
                        thrpt:  [+4.4740% +5.2687% +6.3871%]
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe
./data/small/az_tenants.json_az_tenants::recursive_ids/rsonpath_count
                        time:   [11.713 µs 11.717 µs 11.723 µs]
                        thrpt:  [3.2195 GB/s 3.2212 GB/s 3.2224 GB/s]
                 change:
                        time:   [+2.8433% +2.9230% +2.9848%] (p = 0.00 < 0.05)
                        thrpt:  [-2.8983% -2.8400% -2.7647%]
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe
./data/small/az_tenants.json_az_tenants::recursive_ids/rsonpath_mmap
                        time:   [17.176 µs 17.179 µs 17.183 µs]
                        thrpt:  [2.1965 GB/s 2.1970 GB/s 2.1975 GB/s]
                 change:
                        time:   [+1.6192% +1.6764% +1.7241%] (p = 0.00 < 0.05)
                        thrpt:  [-1.6949% -1.6488% -1.5934%]
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe
./data/small/az_tenants.json_az_tenants::recursive_ids/rsonpath_mmap_count
                        time:   [10.359 µs 10.364 µs 10.370 µs]
                        thrpt:  [3.6395 GB/s 3.6418 GB/s 3.6433 GB/s]
                 change:
                        time:   [-4.8141% -4.4703% -4.0937%] (p = 0.00 < 0.05)
                        thrpt:  [+4.2684% +4.6795% +5.0576%]
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  2 (2.00%) high mild
  2 (2.00%) high severe

./data/small/az_tenants.json_az_tenants::tenant_17/rsonpath
                        time:   [7.5337 µs 7.5354 µs 7.5374 µs]
                        thrpt:  [5.0074 GB/s 5.0088 GB/s 5.0099 GB/s]
                 change:
                        time:   [-0.5213% -0.4641% -0.4105%] (p = 0.00 < 0.05)
                        thrpt:  [+0.4122% +0.4662% +0.5241%]
                        Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low severe
  1 (1.00%) low mild
  3 (3.00%) high mild
  4 (4.00%) high severe
./data/small/az_tenants.json_az_tenants::tenant_17/rsonpath_count
                        time:   [6.4154 µs 6.4181 µs 6.4208 µs]
                        thrpt:  [5.8783 GB/s 5.8807 GB/s 5.8832 GB/s]
                 change:
                        time:   [-0.9056% -0.8671% -0.8265%] (p = 0.00 < 0.05)
                        thrpt:  [+0.8334% +0.8747% +0.9139%]
                        Change within noise threshold.
./data/small/az_tenants.json_az_tenants::tenant_17/rsonpath_mmap
                        time:   [7.3077 µs 7.3086 µs 7.3097 µs]
                        thrpt:  [5.1634 GB/s 5.1642 GB/s 5.1648 GB/s]
                 change:
                        time:   [-0.8907% -0.8532% -0.8201%] (p = 0.00 < 0.05)
                        thrpt:  [+0.8269% +0.8606% +0.8987%]
                        Change within noise threshold.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe
./data/small/az_tenants.json_az_tenants::tenant_17/rsonpath_mmap_count
                        time:   [6.2868 µs 6.2874 µs 6.2882 µs]
                        thrpt:  [6.0022 GB/s 6.0029 GB/s 6.0035 GB/s]
                 change:
                        time:   [-1.2607% -1.2344% -1.2102%] (p = 0.00 < 0.05)
                        thrpt:  [+1.2250% +1.2498% +1.2768%]
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low mild
  1 (1.00%) high mild
  5 (5.00%) high severe

./data/small/az_tenants.json_az_tenants::tenant_last/rsonpath
                        time:   [12.444 µs 12.447 µs 12.449 µs]
                        thrpt:  [3.0318 GB/s 3.0324 GB/s 3.0330 GB/s]
                 change:
                        time:   [-2.8514% -2.8136% -2.7752%] (p = 0.00 < 0.05)
                        thrpt:  [+2.8544% +2.8951% +2.9350%]
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe
./data/small/az_tenants.json_az_tenants::tenant_last/rsonpath_count
                        time:   [10.906 µs 10.913 µs 10.922 µs]
                        thrpt:  [3.4556 GB/s 3.4584 GB/s 3.4607 GB/s]
                 change:
                        time:   [-0.3602% -0.3057% -0.2481%] (p = 0.00 < 0.05)
                        thrpt:  [+0.2487% +0.3067% +0.3615%]
                        Change within noise threshold.
Found 15 outliers among 100 measurements (15.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  11 (11.00%) high severe
./data/small/az_tenants.json_az_tenants::tenant_last/rsonpath_mmap
                        time:   [11.975 µs 11.977 µs 11.979 µs]
                        thrpt:  [3.1508 GB/s 3.1513 GB/s 3.1518 GB/s]
                 change:
                        time:   [-5.9785% -5.9469% -5.9143%] (p = 0.00 < 0.05)
                        thrpt:  [+6.2860% +6.3229% +6.3587%]
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low severe
  1 (1.00%) low mild
  2 (2.00%) high mild
  1 (1.00%) high severe
./data/small/az_tenants.json_az_tenants::tenant_last/rsonpath_mmap_count
                        time:   [10.783 µs 10.785 µs 10.786 µs]
                        thrpt:  [3.4991 GB/s 3.4997 GB/s 3.5002 GB/s]
                 change:
                        time:   [-2.1246% -1.9450% -1.7757%] (p = 0.00 < 0.05)
                        thrpt:  [+1.8078% +1.9836% +2.1707%]
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe

@V0ldek V0ldek enabled auto-merge (squash) March 15, 2024 22:19
@V0ldek V0ldek disabled auto-merge March 15, 2024 22:19
@V0ldek V0ldek merged commit 8910617 into main Mar 15, 2024
46 checks passed
@V0ldek V0ldek deleted the v0ldek/index-selector branch March 15, 2024 22:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve index selector performance by caching state attributes
1 participant