Avoid decoding doc subkey during compaction #1178
Comments
yugabyte-ci
pushed a commit
that referenced
this issue
Apr 12, 2019
Summary: During compaction we could avoid decoding doc key, since we need the hybrid time and a number of leading components that match the previous key. Test Plan: 1) Jenkins 2) Did some performance measurements. Launched RF1 local cluster with existing workload: `java -jar target/yb-sample-apps.jar --workload CassandraBatchTimeseries --nodes 127.0.0.1:9042 --num_threads_read 1 --num_threads_write 128 --num_unique_keys -1` Ran this workload for quite a long time (40mins), Measurements for master branch: ``` Read: 1193.28 ops/sec (0.84 ms/op), 378043 total ops | Write: 77892.08 ops/sec (830.30 ms/op), 23092500 total ops | Uptime: 300841 ms | Read: 1170.09 ops/sec (0.85 ms/op), 765027 total ops | Write: 79792.78 ops/sec (798.74 ms/op), 50498500 total ops | Uptime: 600937 ms | Read: 866.53 ops/sec (1.15 ms/op), 1119817 total ops | Write: 65994.92 ops/sec (997.80 ms/op), 74687500 total ops | Uptime: 901522 ms | Read: 1131.47 ops/sec (0.88 ms/op), 1484840 total ops | Write: 78890.29 ops/sec (819.77 ms/op), 98903000 total ops | Uptime: 1201549 ms | Read: 1539.09 ops/sec (0.65 ms/op), 1846343 total ops | Write: 93893.89 ops/sec (688.40 ms/op), 122960000 total ops | Uptime: 1501690 ms | Read: 1149.48 ops/sec (0.87 ms/op), 2239940 total ops | Write: 71066.71 ops/sec (914.96 ms/op), 149928000 total ops | Uptime: 1802010 ms | Read: 1472.51 ops/sec (0.68 ms/op), 2620166 total ops | Write: 99193.19 ops/sec (658.40 ms/op), 175090000 total ops | Uptime: 2102031 ms | Read: 1135.11 ops/sec (0.88 ms/op), 2994563 total ops | Write: 79494.09 ops/sec (801.23 ms/op), 200440000 total ops | Uptime: 2402051 ms | ``` Before Change: Write Ops Steps (ops per 5 minutes): `23.1M, 27.4M, 24.2M, 24.2M, 24.1M, 27M, 25.2M, 25.3M` Measurements with this change: ``` Read: 920.91 ops/sec (1.08 ms/op), 289708 total ops | Write: 89590.59 ops/sec (717.88 ms/op), 27673000 total ops | Uptime: 300089 ms | Read: 720.93 ops/sec (1.39 ms/op), 579106 total ops | Write: 103291.92 ops/sec (621.91 ms/op), 59884000 total ops | Uptime: 600116 ms | Read: 1120.31 ops/sec (0.89 ms/op), 877834 total ops | Write: 115891.00 ops/sec (561.94 ms/op), 93062000 total ops | Uptime: 900761 ms | Read: 1002.70 ops/sec (1.00 ms/op), 1168595 total ops | Write: 110190.38 ops/sec (590.94 ms/op), 125572000 total ops | Uptime: 1200786 ms | Read: 915.92 ops/sec (1.09 ms/op), 1464335 total ops | Write: 94191.80 ops/sec (681.43 ms/op), 157753000 total ops | Uptime: 1500810 ms | Read: 890.92 ops/sec (1.12 ms/op), 1761815 total ops | Write: 83092.58 ops/sec (769.05 ms/op), 189002000 total ops | Uptime: 1800834 ms | Read: 1052.34 ops/sec (0.95 ms/op), 2059466 total ops | Write: 108293.37 ops/sec (583.80 ms/op), 220064500 total ops | Uptime: 2101200 ms | Read: 979.53 ops/sec (1.02 ms/op), 2347703 total ops | Write: 101492.80 ops/sec (613.49 ms/op), 252623000 total ops | Uptime: 2401320 ms | ``` After Change: Write Ops Steps (ops per 5 minutes): `27.7M, 32.2M, 33.2M, 32.5M, 32.2M, 31.2M, 31.1M, 32.6M` So it is stable & better for all 40 minutes. Reviewers: timur, bogdan, mikhail Reviewed By: mikhail Subscribers: karthik, kannan, bogdan, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D6452
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
It is not necessary to fully decode doc subkey during compaction, since all calculations could be done using raw bytes.
The text was updated successfully, but these errors were encountered: