Skip to content

sql: turn on column family heuristics #7623

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jul 8, 2016
Merged

sql: turn on column family heuristics #7623

merged 1 commit into from
Jul 8, 2016

Conversation

danhhz
Copy link
Contributor

@danhhz danhhz commented Jul 5, 2016 •
edited by tamird
Loading 

name                                    old time/op    new time/op    delta
Bank2_Cockroach-32                        12.2ms ± 6%    14.3ms ±29%     ~             (p=0.690 n=5+5)
Bank4_Cockroach-32                        8.78ms ±25%    9.02ms ±26%     ~             (p=1.000 n=5+5)
Bank8_Cockroach-32                        4.74ms ±25%    4.40ms ±25%     ~             (p=0.548 n=5+5)
Bank16_Cockroach-32                       3.00ms ±47%    3.33ms ±20%     ~             (p=0.310 n=5+5)
Bank32_Cockroach-32                       2.26ms ±20%    1.98ms ±18%     ~             (p=0.310 n=5+5)
Bank64_Cockroach-32                       1.40ms ±23%    1.66ms ±40%     ~             (p=0.222 n=5+5)
Select1_Cockroach-32                      61.7µs ± 1%    64.1µs ± 1%   +3.91%          (p=0.008 n=5+5)
Select2_Cockroach-32                       801µs ± 1%     674µs ± 1%  -15.88%          (p=0.008 n=5+5)
Select3_Cockroach-32                      1.07ms ± 1%    0.94ms ± 1%  -12.33%          (p=0.008 n=5+5)
Insert1_Cockroach-32                       513µs ± 1%     521µs ± 3%     ~             (p=0.056 n=5+5)
Insert10_Cockroach-32                      671µs ± 2%     675µs ± 1%     ~             (p=0.690 n=5+5)
Insert100_Cockroach-32                    1.82ms ± 1%    1.81ms ± 2%     ~             (p=0.222 n=5+5)
Insert1000_Cockroach-32                   12.1ms ± 5%    12.0ms ± 4%     ~             (p=0.421 n=5+5)
Update1_Cockroach-32                       715µs ± 2%     720µs ± 0%     ~             (p=0.095 n=5+5)
Update10_Cockroach-32                     1.14ms ± 2%    1.13ms ± 2%     ~             (p=0.421 n=5+5)
Update100_Cockroach-32                    4.53ms ± 1%    4.33ms ± 2%   -4.46%          (p=0.008 n=5+5)
Update1000_Cockroach-32                   34.5ms ± 1%    33.3ms ± 2%   -3.66%          (p=0.008 n=5+5)
Upsert1_Cockroach-32                      1.75ms ± 1%    1.68ms ± 1%   -4.01%          (p=0.008 n=5+5)
Upsert10_Cockroach-32                     2.20ms ± 2%    1.93ms ± 3%  -12.24%          (p=0.008 n=5+5)
Upsert100_Cockroach-32                    7.24ms ± 3%    4.73ms ± 3%  -34.72%          (p=0.008 n=5+5)
Upsert1000_Cockroach-32                   55.5ms ± 3%    30.4ms ± 3%  -45.30%          (p=0.008 n=5+5)
Delete1_Cockroach-32                       733µs ± 1%     693µs ± 2%   -5.45%          (p=0.008 n=5+5)
Delete10_Cockroach-32                     1.25ms ± 2%    0.85ms ± 2%  -31.91%          (p=0.008 n=5+5)
Delete100_Cockroach-32                    6.33ms ± 0%    2.35ms ± 2%  -62.82%          (p=0.008 n=5+5)
Delete1000_Cockroach-32                   63.4ms ± 3%    17.9ms ± 2%  -71.80%          (p=0.008 n=5+5)
Scan1_Cockroach-32                         182µs ± 2%     185µs ± 1%     ~             (p=0.095 n=5+5)
Scan10_Cockroach-32                        210µs ± 2%     214µs ± 2%     ~             (p=0.151 n=5+5)
Scan100_Cockroach-32                       479µs ± 1%     488µs ± 1%   +1.68%          (p=0.008 n=5+5)
Scan1000_Cockroach-32                     2.68ms ± 2%    2.74ms ± 2%     ~             (p=0.095 n=5+5)
Scan10000_Cockroach-32                    23.9ms ± 1%    24.3ms ± 4%     ~             (p=0.151 n=5+5)
Scan1000Limit1_Cockroach-32                192µs ± 1%     195µs ± 2%     ~             (p=0.095 n=5+5)
Scan1000Limit10_Cockroach-32               223µs ± 2%     227µs ± 2%     ~             (p=0.095 n=5+5)
Scan1000Limit100_Cockroach-32              477µs ± 1%     484µs ± 3%     ~             (p=0.095 n=5+5)
Scan10000FilterLimit1_Cockroach-32         379µs ± 1%     384µs ± 2%     ~             (p=0.056 n=5+5)
Scan10000FilterLimit10_Cockroach-32        427µs ± 1%     429µs ± 4%     ~             (p=1.000 n=5+5)
Scan10000FilterLimit50_Cockroach-32       2.49ms ± 2%    2.47ms ± 1%     ~             (p=0.690 n=5+5)
Sort100000Limit10_Cockroach-32             483ms ± 1%     220ms ± 5%  -54.45%          (p=0.008 n=5+5)
Sort100000Limit10Distinct_Cockroach-32     498ms ± 2%     239ms ± 2%  -52.09%          (p=0.008 n=5+5)
TrackChoices1_Cockroach-32                 582µs ± 2%     574µs ± 2%     ~             (p=0.222 n=5+5)
TrackChoices10_Cockroach-32                117µs ± 3%     110µs ± 5%   -5.64%          (p=0.016 n=5+5)
TrackChoices100_Cockroach-32              66.1µs ± 2%    61.5µs ± 4%   -6.95%          (p=0.008 n=5+5)
TrackChoices1000_Cockroach-32             62.7µs ± 4%    54.2µs ± 3%  -13.62%          (p=0.008 n=5+5)
InsertDistinct1_Cockroach-32              1.56ms ± 2%    1.59ms ± 2%     ~             (p=0.151 n=5+5)
InsertDistinct10_Cockroach-32             1.09ms ± 2%    1.19ms ± 7%   +9.64%          (p=0.032 n=5+5)
InsertDistinct100_Cockroach-32            1.34ms ± 2%    1.80ms ±15%  +34.41%          (p=0.008 n=5+5)
WideTable1_Cockroach-32                   2.19ms ± 1%    2.31ms ± 2%   +5.44%          (p=0.008 n=5+5)
WideTable10_Cockroach-32                  3.42ms ± 1%    3.68ms ± 2%   +7.43%          (p=0.008 n=5+5)
WideTable100_Cockroach-32                 15.2ms ± 2%    16.0ms ± 1%   +4.99%          (p=0.008 n=5+5)
WideTable1000_Cockroach-32                 132ms ± 2%     136ms ± 2%   +2.83%          (p=0.016 n=5+5)
PgbenchQuery_Cockroach-32                 3.21ms ± 2%    3.08ms ± 2%   -4.07%          (p=0.016 n=5+5)
ParallelPgbenchQuery_Cockroach-32        32.7ms ±114%    25.7ms ±47%     ~             (p=0.421 n=5+5)

name                                    old alloc/op   new alloc/op   delta
Bank2_Cockroach-32                        1.08MB ±24%    1.16MB ± 7%     ~             (p=0.548 n=5+5)
Bank4_Cockroach-32                         862kB ±34%     828kB ±27%     ~             (p=0.690 n=5+5)
Bank8_Cockroach-32                         558kB ±12%     580kB ±17%     ~             (p=0.548 n=5+5)
Bank16_Cockroach-32                        429kB ± 4%     444kB ±14%     ~             (p=0.690 n=5+5)
Bank32_Cockroach-32                        345kB ±12%     329kB ±10%     ~             (p=0.548 n=5+5)
Bank64_Cockroach-32                        256kB ± 4%     263kB ± 1%     ~             (p=0.056 n=5+5)
Select1_Cockroach-32                      3.73kB ± 0%    3.73kB ± 0%     ~             (p=0.063 n=5+5)
Select2_Cockroach-32                      91.8kB ± 0%    71.4kB ± 0%  -22.25%          (p=0.008 n=5+5)
Select3_Cockroach-32                       136kB ± 0%     116kB ± 0%  -15.00%          (p=0.008 n=5+5)
Insert1_Cockroach-32                      29.8kB ± 0%    29.8kB ± 0%     ~             (p=0.421 n=5+5)
Insert10_Cockroach-32                     74.9kB ± 0%    74.9kB ± 0%     ~             (p=0.333 n=5+5)
Insert100_Cockroach-32                     488kB ± 0%     488kB ± 0%     ~             (p=0.841 n=5+5)
Insert1000_Cockroach-32                   4.18MB ± 0%    4.18MB ± 0%     ~             (p=0.841 n=5+5)
Update1_Cockroach-32                      51.7kB ± 0%    51.6kB ± 0%   -0.14%          (p=0.008 n=5+5)
Update10_Cockroach-32                      122kB ± 0%     120kB ± 0%   -1.57%          (p=0.016 n=4+5)
Update100_Cockroach-32                     795kB ± 0%     774kB ± 0%   -2.68%          (p=0.016 n=4+5)
Update1000_Cockroach-32                   6.59MB ± 0%    6.39MB ± 0%   -3.01%          (p=0.029 n=4+4)
Upsert1_Cockroach-32                       104kB ± 0%      98kB ± 0%   -6.01%          (p=0.008 n=5+5)
Upsert10_Cockroach-32                      172kB ± 0%     129kB ± 0%  -25.06%          (p=0.008 n=5+5)
Upsert100_Cockroach-32                     942kB ± 0%     554kB ± 0%  -41.22%          (p=0.008 n=5+5)
Upsert1000_Cockroach-32                   7.87MB ± 1%    4.48MB ± 0%  -43.10%          (p=0.008 n=5+5)
Delete1_Cockroach-32                      45.3kB ± 0%    44.0kB ± 0%   -2.87%          (p=0.008 n=5+5)
Delete10_Cockroach-32                     70.3kB ± 0%    54.1kB ± 0%  -23.06%          (p=0.008 n=5+5)
Delete100_Cockroach-32                     307kB ± 0%     155kB ± 0%  -49.35%          (p=0.008 n=5+5)
Delete1000_Cockroach-32                   3.19MB ± 0%    1.07MB ± 0%  -66.46%          (p=0.016 n=4+5)
Scan1_Cockroach-32                        16.0kB ± 0%    16.0kB ± 0%     ~             (p=0.365 n=5+5)
Scan10_Cockroach-32                       19.5kB ± 0%    19.5kB ± 0%     ~             (p=0.968 n=5+5)
Scan100_Cockroach-32                      50.5kB ± 0%    50.5kB ± 0%     ~             (p=0.730 n=5+5)
Scan1000_Cockroach-32                      318kB ± 0%     318kB ± 0%     ~             (p=1.000 n=5+5)
Scan10000_Cockroach-32                    5.54MB ± 0%    5.54MB ± 0%     ~             (p=0.841 n=5+5)
Scan1000Limit1_Cockroach-32               16.6kB ± 0%    16.6kB ± 0%     ~             (p=0.952 n=5+5)
Scan1000Limit10_Cockroach-32              19.9kB ± 0%    19.9kB ± 0%     ~             (p=0.667 n=5+5)
Scan1000Limit100_Cockroach-32             50.8kB ± 0%    50.8kB ± 0%     ~             (p=0.992 n=5+5)
Scan10000FilterLimit1_Cockroach-32        48.6kB ± 0%    48.6kB ± 0%     ~             (p=0.421 n=5+5)
Scan10000FilterLimit10_Cockroach-32       54.8kB ± 0%    54.8kB ± 0%     ~             (p=1.000 n=5+5)
Scan10000FilterLimit50_Cockroach-32        462kB ± 0%     462kB ± 0%     ~             (p=0.952 n=5+5)
Sort100000Limit10_Cockroach-32             118MB ± 0%      42MB ± 0%  -64.13%          (p=0.029 n=4+4)
Sort100000Limit10Distinct_Cockroach-32     136MB ± 0%      60MB ± 0%  -55.87%          (p=0.008 n=5+5)
TrackChoices1_Cockroach-32                46.6kB ± 0%    44.0kB ± 0%   -5.68%          (p=0.008 n=5+5)
TrackChoices10_Cockroach-32               23.5kB ± 0%    16.5kB ± 0%  -29.72%          (p=0.008 n=5+5)
TrackChoices100_Cockroach-32              20.0kB ± 0%    16.0kB ± 0%  -19.99%          (p=0.008 n=5+5)
TrackChoices1000_Cockroach-32             22.7kB ± 0%    17.8kB ± 0%  -21.70%          (p=0.016 n=4+5)
InsertDistinct1_Cockroach-32               303kB ± 0%     303kB ± 0%     ~             (p=0.635 n=5+5)
InsertDistinct10_Cockroach-32              316kB ± 0%     323kB ± 2%   +2.41%          (p=0.008 n=5+5)
InsertDistinct100_Cockroach-32             473kB ± 0%     540kB ±19%  +14.19%          (p=0.032 n=5+5)
WideTable1_Cockroach-32                    170kB ± 0%     170kB ± 0%   +0.52%          (p=0.016 n=5+4)
WideTable10_Cockroach-32                   373kB ± 0%     374kB ± 0%   +0.36%          (p=0.008 n=5+5)
WideTable100_Cockroach-32                 2.36MB ± 0%    2.36MB ± 0%   +0.15%          (p=0.008 n=5+5)
WideTable1000_Cockroach-32                21.2MB ± 0%    21.2MB ± 0%     ~             (p=0.056 n=5+5)
PgbenchQuery_Cockroach-32                  277kB ± 0%     247kB ± 0%  -10.68%          (p=0.029 n=4+4)
ParallelPgbenchQuery_Cockroach-32         1.41MB ± 4%    1.33MB ± 6%     ~             (p=0.056 n=5+5)

name                                    old allocs/op  new allocs/op  delta
Bank2_Cockroach-32                         11.6k ±20%     12.5k ± 8%     ~             (p=0.421 n=5+5)
Bank4_Cockroach-32                         9.56k ±30%     9.20k ±21%     ~             (p=0.841 n=5+5)
Bank8_Cockroach-32                         6.45k ±10%     6.64k ±15%     ~             (p=0.548 n=5+5)
Bank16_Cockroach-32                        5.00k ± 4%     5.10k ±12%     ~             (p=0.690 n=5+5)
Bank32_Cockroach-32                        4.03k ±10%     3.89k ± 7%     ~             (p=0.690 n=5+5)
Bank64_Cockroach-32                        3.03k ± 4%     3.14k ± 1%     ~             (p=0.056 n=5+5)
Select1_Cockroach-32                        58.0 ± 0%      58.0 ± 0%     ~     (all samples are equal)
Select2_Cockroach-32                       1.20k ± 0%     1.19k ± 0%   -0.72%          (p=0.000 n=5+4)
Select3_Cockroach-32                       2.06k ± 0%     2.05k ± 0%     ~             (p=0.079 n=4+5)
Insert1_Cockroach-32                         300 ± 0%       300 ± 0%     ~     (all samples are equal)
Insert10_Cockroach-32                        508 ± 0%       508 ± 0%     ~     (all samples are equal)
Insert100_Cockroach-32                     2.35k ± 0%     2.35k ± 0%     ~             (p=1.000 n=5+5)
Insert1000_Cockroach-32                    20.5k ± 0%     20.5k ± 0%     ~             (p=0.524 n=5+5)
Update1_Cockroach-32                         575 ± 0%       580 ± 0%   +0.87%          (p=0.008 n=5+5)
Update10_Cockroach-32                        887 ± 0%       884 ± 0%     ~             (p=0.079 n=4+5)
Update100_Cockroach-32                     3.68k ± 0%     3.59k ± 0%   -2.43%          (p=0.000 n=5+4)
Update1000_Cockroach-32                    29.7k ± 0%     28.8k ± 0%   -2.88%          (p=0.029 n=4+4)
Upsert1_Cockroach-32                       1.15k ± 0%     1.12k ± 0%   -2.79%          (p=0.008 n=5+5)
Upsert10_Cockroach-32                      1.54k ± 0%     1.40k ± 0%   -9.36%          (p=0.008 n=5+5)
Upsert100_Cockroach-32                     5.70k ± 0%     4.36k ± 0%  -23.46%          (p=0.008 n=5+5)
Upsert1000_Cockroach-32                    46.8k ± 0%     33.4k ± 0%  -28.55%          (p=0.008 n=5+5)
Delete1_Cockroach-32                         479 ± 0%       466 ± 0%   -2.71%          (p=0.029 n=4+4)
Delete10_Cockroach-32                        684 ± 0%       591 ± 0%  -13.71%          (p=0.008 n=5+5)
Delete100_Cockroach-32                     2.60k ± 0%     1.69k ± 0%  -34.87%          (p=0.008 n=5+5)
Delete1000_Cockroach-32                    21.6k ± 0%     12.6k ± 0%  -41.81%          (p=0.016 n=4+5)
Scan1_Cockroach-32                           200 ± 0%       200 ± 0%     ~     (all samples are equal)
Scan10_Cockroach-32                          227 ± 0%       227 ± 0%     ~     (all samples are equal)
Scan100_Cockroach-32                         425 ± 0%       425 ± 0%     ~     (all samples are equal)
Scan1000_Cockroach-32                      2.31k ± 0%     2.31k ± 0%     ~             (p=0.873 n=5+5)
Scan10000_Cockroach-32                     21.2k ± 0%     21.2k ± 0%     ~             (p=1.000 n=5+5)
Scan1000Limit1_Cockroach-32                  207 ± 0%       207 ± 0%     ~     (all samples are equal)
Scan1000Limit10_Cockroach-32                 233 ± 0%       233 ± 0%     ~     (all samples are equal)
Scan1000Limit100_Cockroach-32                430 ± 0%       430 ± 0%     ~     (all samples are equal)
Scan10000FilterLimit1_Cockroach-32           509 ± 0%       509 ± 0%     ~     (all samples are equal)
Scan10000FilterLimit10_Cockroach-32          563 ± 0%       563 ± 0%     ~     (all samples are equal)
Scan10000FilterLimit50_Cockroach-32        1.80k ± 0%     1.80k ± 0%     ~             (p=0.167 n=5+5)
Sort100000Limit10_Cockroach-32             75.1k ± 0%     60.3k ± 0%  -19.74%          (p=0.029 n=4+4)
Sort100000Limit10Distinct_Cockroach-32      135k ± 0%      120k ± 0%  -10.93%          (p=0.008 n=5+5)
TrackChoices1_Cockroach-32                   397 ± 0%       387 ± 0%   -2.52%          (p=0.008 n=5+5)
TrackChoices10_Cockroach-32                  117 ± 0%       107 ± 0%   -8.55%          (p=0.008 n=5+5)
TrackChoices100_Cockroach-32                86.0 ± 0%      77.0 ± 0%  -10.47%          (p=0.000 n=5+4)
TrackChoices1000_Cockroach-32               83.0 ± 0%      73.0 ± 0%     ~             (p=0.079 n=4+5)
InsertDistinct1_Cockroach-32               1.90k ± 0%     1.90k ± 0%     ~             (p=0.079 n=5+5)
InsertDistinct10_Cockroach-32              1.97k ± 0%     2.06k ± 0%   +4.62%          (p=0.016 n=5+4)
InsertDistinct100_Cockroach-32             3.32k ± 0%     3.90k ±22%  +17.51%          (p=0.032 n=5+5)
WideTable1_Cockroach-32                    1.96k ± 0%     1.96k ± 0%   +0.41%          (p=0.016 n=5+4)
WideTable10_Cockroach-32                   4.01k ± 0%     4.02k ± 0%   +0.27%          (p=0.008 n=5+5)
WideTable100_Cockroach-32                  24.0k ± 0%     24.0k ± 0%   +0.13%          (p=0.008 n=5+5)
WideTable1000_Cockroach-32                  223k ± 0%      224k ± 0%     ~             (p=0.690 n=5+5)
PgbenchQuery_Cockroach-32                  2.89k ± 0%     2.85k ± 0%   -1.28%          (p=0.029 n=4+4)
ParallelPgbenchQuery_Cockroach-32          13.9k ± 4%     13.8k ± 7%     ~             (p=0.841 n=5+5)

This change is Reviewable

@RaduBerinde
Copy link
Member

Nice!

Do you know where the InsertDistinct100 slowdown comes from? Is it that we were able to do a CPut before and now we first need to do a Get?

Review status: 0 of 15 files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.

sql/testdata/select_non_covering_index, line 25 [r1] (raw file):

----
0  /t/b/2/1            NULL    PARTIAL
0  /t/primary/1/b/c/d  /2/3/4  ROW

Regarding these testcases which aren't specific to families - it's not reasonable to have to update them any time we change the heuristics. We should instead update the "create table" statements in these tests to explicitly set one family per column.

Comments from Reviewable

@tbg
Copy link
Member

tbg commented Jul 7, 2016

Also interested in InsertDistinct. It doesn't do anything but insert batches of (out-of-order) rows in a simple table with only a three-element composite primary key.

@danhhz
Copy link
Contributor Author

danhhz commented Jul 7, 2016

Happily, INSERT still doesn't require a get. The Insert benchmark is unchanged, which bears that out.

There's an allocation uptick in InsertDistinct, which is probably the cause of the time slowdown. I could pprof it again and see if I can find where that's coming from, but I really can't imaging what's different

Review status: 0 of 15 files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.

Comments from Reviewable

@RaduBerinde
Copy link
Member

+34.41% is significant so it would be good to investigate. Can be done later (doesn't need to hold up this PR) but in that case please file an issue.

Review status: 0 of 15 files reviewed at latest revision, 1 unresolved discussion, some commit checks failed.

Comments from Reviewable

@danhhz
Copy link
Contributor Author

danhhz commented Jul 8, 2016

This got really bizarre. When I run only the InsertDistinct benchmark (either on my laptop or gcloud), there is no change. I've done this several times. When I run the 15 minute big suite of sql benchmarks, it repros. Some sort of benchmark bleeding?

Review status: 0 of 15 files reviewed at latest revision, 1 unresolved discussion, some commit checks pending.

sql/testdata/select_non_covering_index, line 25 [r1] (raw file):

Previously, RaduBerinde wrote…

Regarding these testcases which aren't specific to families - it's not reasonable to have to update them any time we change the heuristics. We should instead update the "create table" statements in these tests to explicitly set one family per column.

Done, though I left table and alter_table since I'd like to see those changes. This also surfaced the inability to force a new family with ALTER TABLE ADD, so I created an issue: https://github.com//issues/7697

Comments from Reviewable

@RaduBerinde
Copy link
Member

... wow

I notice there is a lot of variation (±15%) - can you post all the results of InsertDistinct100_Cockroach in the raw output? I wonder if it's just a single run that's superslow

What variation do you get if run more iterations (say 100) of InsertDistinct100_Cockroach?

BTW are you using test.count or are you running make bench multiple times?

Review status: 0 of 15 files reviewed at latest revision, 1 unresolved discussion, some commit checks pending.

Comments from Reviewable

@danhhz
sql: turn on column family heuristics
0ac6d56 
name                            old time/op    new time/op    delta
Bank2_Cockroach-32                12.2ms ±44%    12.9ms ±35%     ~           (p=0.529 n=10+10)
Bank4_Cockroach-32                9.32ms ±52%    9.66ms ±39%     ~           (p=0.579 n=10+10)
Bank8_Cockroach-32                4.44ms ±52%    4.82ms ±26%     ~           (p=0.190 n=10+10)
Bank16_Cockroach-32               3.09ms ±38%    2.68ms ± 8%     ~             (p=0.113 n=9+9)
Bank32_Cockroach-32               2.00ms ±26%    2.09ms ±21%     ~            (p=0.549 n=9+10)
Bank64_Cockroach-32               1.53ms ±14%    1.51ms ±34%     ~           (p=0.315 n=10+10)
Insert1_Cockroach-32               517µs ± 2%     513µs ± 2%     ~           (p=0.165 n=10+10)
Insert10_Cockroach-32              671µs ± 2%     671µs ± 2%     ~           (p=0.912 n=10+10)
Insert100_Cockroach-32            1.82ms ± 3%    1.80ms ± 3%     ~           (p=0.247 n=10+10)
Insert1000_Cockroach-32           12.2ms ± 3%    12.1ms ± 4%     ~            (p=0.065 n=10+9)
Update1_Cockroach-32               722µs ± 1%     725µs ± 3%     ~           (p=0.853 n=10+10)
Update10_Cockroach-32             1.15ms ± 2%    1.13ms ± 3%     ~           (p=0.089 n=10+10)
Update100_Cockroach-32            4.51ms ± 2%    4.44ms ± 4%     ~            (p=0.095 n=9+10)
Update1000_Cockroach-32           35.5ms ±10%    33.4ms ± 3%   -6.11%        (p=0.000 n=10+10)
Upsert1_Cockroach-32              1.77ms ± 2%    1.71ms ± 1%   -3.33%          (p=0.000 n=9+9)
Upsert10_Cockroach-32             2.21ms ± 2%    1.95ms ± 2%  -11.85%         (p=0.000 n=10+9)
Upsert100_Cockroach-32            6.85ms ± 2%    4.50ms ± 2%  -34.35%        (p=0.000 n=10+10)
Upsert1000_Cockroach-32           54.6ms ± 5%    29.8ms ± 2%  -45.48%         (p=0.000 n=9+10)
Delete1_Cockroach-32               738µs ± 2%     691µs ± 2%   -6.30%        (p=0.000 n=10+10)
Delete10_Cockroach-32             1.28ms ± 3%    0.86ms ± 1%  -32.94%        (p=0.000 n=10+10)
Delete100_Cockroach-32            6.37ms ± 7%    2.33ms ± 1%  -63.43%         (p=0.000 n=10+9)
Delete1000_Cockroach-32           64.8ms ± 7%    17.6ms ± 6%  -72.79%        (p=0.000 n=10+10)
TrackChoices1_Cockroach-32         607µs ± 5%     593µs ± 2%     ~            (p=0.068 n=10+8)
TrackChoices10_Cockroach-32        127µs ± 2%     112µs ± 5%  -11.13%        (p=0.000 n=10+10)
TrackChoices100_Cockroach-32      73.0µs ± 4%    60.4µs ± 2%  -17.19%        (p=0.000 n=10+10)
TrackChoices1000_Cockroach-32     67.3µs ± 4%    55.2µs ± 4%  -17.97%         (p=0.000 n=10+9)
InsertDistinct1_Cockroach-32      1.63ms ± 2%    1.62ms ± 2%     ~             (p=0.190 n=9+9)
InsertDistinct10_Cockroach-32     1.16ms ± 3%    1.15ms ± 3%     ~           (p=0.579 n=10+10)
InsertDistinct100_Cockroach-32    1.54ms ± 3%    1.55ms ± 3%     ~            (p=0.447 n=9+10)
WideTable1_Cockroach-32           2.32ms ± 2%    2.31ms ± 2%     ~           (p=0.353 n=10+10)
WideTable10_Cockroach-32          3.69ms ± 2%    3.64ms ± 1%     ~            (p=0.065 n=10+9)
WideTable100_Cockroach-32         16.3ms ± 2%    16.3ms ± 4%     ~           (p=0.393 n=10+10)
WideTable1000_Cockroach-32         138ms ± 1%     137ms ± 2%     ~            (p=0.095 n=9+10)

name                            old alloc/op   new alloc/op   delta
Bank2_Cockroach-32                1.07MB ±35%    1.15MB ±34%     ~           (p=0.315 n=10+10)
Bank4_Cockroach-32                 976kB ±31%     971kB ±25%     ~           (p=0.684 n=10+10)
Bank8_Cockroach-32                 573kB ± 7%     619kB ±19%     ~            (p=0.211 n=9+10)
Bank16_Cockroach-32                431kB ± 6%     420kB ± 4%     ~            (p=0.133 n=9+10)
Bank32_Cockroach-32                327kB ±19%     334kB ±14%     ~           (p=1.000 n=10+10)
Bank64_Cockroach-32                262kB ± 3%     259kB ± 4%     ~           (p=0.218 n=10+10)
Insert1_Cockroach-32              30.2kB ± 0%    30.2kB ± 0%     ~            (p=0.091 n=10+7)
Insert10_Cockroach-32             75.5kB ± 0%    75.5kB ± 0%     ~           (p=0.956 n=10+10)
Insert100_Cockroach-32             491kB ± 0%     491kB ± 0%     ~            (p=0.887 n=7+10)
Insert1000_Cockroach-32           4.17MB ± 0%    4.17MB ± 0%     ~            (p=0.780 n=10+9)
Update1_Cockroach-32              52.5kB ± 0%    52.3kB ± 0%   -0.52%        (p=0.000 n=10+10)
Update10_Cockroach-32              123kB ± 0%     120kB ± 0%   -1.68%          (p=0.000 n=7+9)
Update100_Cockroach-32             799kB ± 0%     776kB ± 0%   -2.83%          (p=0.000 n=9+9)
Update1000_Cockroach-32           6.56MB ± 0%    6.36MB ± 0%   -3.01%          (p=0.000 n=8+8)
Upsert1_Cockroach-32               105kB ± 0%      99kB ± 0%   -5.90%        (p=0.000 n=10+10)
Upsert10_Cockroach-32              174kB ± 0%     130kB ± 0%  -24.93%         (p=0.000 n=9+10)
Upsert100_Cockroach-32             946kB ± 0%     557kB ± 0%  -41.11%         (p=0.000 n=9+10)
Upsert1000_Cockroach-32           7.91MB ± 1%    4.50MB ± 0%  -43.13%         (p=0.000 n=10+8)
Delete1_Cockroach-32              45.8kB ± 0%    44.4kB ± 0%   -2.95%         (p=0.000 n=9+10)
Delete10_Cockroach-32             71.4kB ± 0%    54.5kB ± 0%  -23.64%         (p=0.000 n=9+10)
Delete100_Cockroach-32             311kB ± 0%     158kB ± 1%  -49.08%        (p=0.000 n=10+10)
Delete1000_Cockroach-32           3.19MB ± 0%    1.07MB ± 0%  -66.47%          (p=0.000 n=8+9)
TrackChoices1_Cockroach-32        47.3kB ± 0%    44.6kB ± 0%   -5.63%         (p=0.000 n=10+8)
TrackChoices10_Cockroach-32       23.8kB ± 0%    16.7kB ± 0%  -29.65%          (p=0.000 n=8+9)
TrackChoices100_Cockroach-32      20.1kB ± 0%    16.1kB ± 0%  -19.93%          (p=0.000 n=9+6)
TrackChoices1000_Cockroach-32     22.7kB ± 0%    17.7kB ± 0%  -21.74%         (p=0.000 n=9+10)
InsertDistinct1_Cockroach-32       306kB ± 0%     306kB ± 0%     ~             (p=0.620 n=9+9)
InsertDistinct10_Cockroach-32      319kB ± 0%     319kB ± 0%     ~           (p=0.971 n=10+10)
InsertDistinct100_Cockroach-32     476kB ± 0%     476kB ± 0%     ~            (p=0.905 n=9+10)
WideTable1_Cockroach-32            172kB ± 0%     172kB ± 0%     ~            (p=0.251 n=9+10)
WideTable10_Cockroach-32           378kB ± 0%     378kB ± 0%     ~            (p=0.278 n=9+10)
WideTable100_Cockroach-32         2.38MB ± 0%    2.38MB ± 0%     ~            (p=0.661 n=9+10)
WideTable1000_Cockroach-32        21.2MB ± 0%    21.2MB ± 0%     ~            (p=0.315 n=10+9)

name                            old allocs/op  new allocs/op  delta
Bank2_Cockroach-32                 11.7k ±36%     12.4k ±33%     ~           (p=0.353 n=10+10)
Bank4_Cockroach-32                 10.7k ±29%     10.6k ±24%     ~           (p=0.796 n=10+10)
Bank8_Cockroach-32                 6.49k ± 8%     7.01k ±17%     ~            (p=0.211 n=9+10)
Bank16_Cockroach-32                4.98k ± 7%     4.88k ± 4%     ~            (p=0.315 n=9+10)
Bank32_Cockroach-32                3.80k ±18%     3.88k ±12%     ~           (p=0.912 n=10+10)
Bank64_Cockroach-32                3.09k ± 3%     3.07k ± 4%     ~           (p=0.698 n=10+10)
Insert1_Cockroach-32                 302 ± 0%       303 ± 0%     ~           (p=0.450 n=10+10)
Insert10_Cockroach-32                510 ± 0%       510 ± 0%     ~     (all samples are equal)
Insert100_Cockroach-32             2.35k ± 0%     2.35k ± 0%     ~           (p=0.873 n=10+10)
Insert1000_Cockroach-32            20.5k ± 0%     20.5k ± 0%     ~           (p=0.643 n=10+10)
Update1_Cockroach-32                 577 ± 0%       582 ± 0%   +0.94%        (p=0.000 n=10+10)
Update10_Cockroach-32                888 ± 0%       886 ± 0%   -0.23%        (p=0.000 n=10+10)
Update100_Cockroach-32             3.68k ± 0%     3.59k ± 0%   -2.41%         (p=0.000 n=9+10)
Update1000_Cockroach-32            29.7k ± 0%     28.8k ± 0%   -2.87%          (p=0.000 n=9+8)
Upsert1_Cockroach-32               1.16k ± 0%     1.13k ± 0%   -2.75%          (p=0.001 n=7+7)
Upsert10_Cockroach-32              1.55k ± 0%     1.41k ± 0%   -9.32%         (p=0.000 n=8+10)
Upsert100_Cockroach-32             5.69k ± 0%     4.36k ± 0%  -23.34%        (p=0.000 n=10+10)
Upsert1000_Cockroach-32            46.7k ± 0%     33.4k ± 0%  -28.44%         (p=0.000 n=10+8)
Delete1_Cockroach-32                 480 ± 0%       468 ± 0%   -2.71%        (p=0.000 n=10+10)
Delete10_Cockroach-32                685 ± 0%       591 ± 0%  -13.66%         (p=0.000 n=8+10)
Delete100_Cockroach-32             2.60k ± 0%     1.69k ± 0%  -34.86%         (p=0.000 n=9+10)
Delete1000_Cockroach-32            21.6k ± 0%     12.6k ± 0%  -41.83%         (p=0.000 n=8+10)
TrackChoices1_Cockroach-32           399 ± 0%       389 ± 0%   -2.60%         (p=0.000 n=10+8)
TrackChoices10_Cockroach-32          118 ± 0%       107 ± 0%   -9.32%        (p=0.000 n=10+10)
TrackChoices100_Cockroach-32        87.0 ± 0%      76.0 ± 0%  -12.64%         (p=0.000 n=10+8)
TrackChoices1000_Cockroach-32       83.0 ± 0%      73.0 ± 0%  -12.05%        (p=0.000 n=10+10)
InsertDistinct1_Cockroach-32       1.91k ± 0%     1.91k ± 0%     ~            (p=0.137 n=8+10)
InsertDistinct10_Cockroach-32      1.98k ± 0%     1.97k ± 0%     ~            (p=0.845 n=10+9)
InsertDistinct100_Cockroach-32     3.32k ± 0%     3.32k ± 0%     ~             (p=0.386 n=9+9)
WideTable1_Cockroach-32            1.97k ± 0%     1.97k ± 0%     ~            (p=0.961 n=8+10)
WideTable10_Cockroach-32           4.03k ± 0%     4.03k ± 0%     ~           (p=0.515 n=10+10)
WideTable100_Cockroach-32          24.0k ± 0%     24.0k ± 0%     ~           (p=0.669 n=10+10)
WideTable1000_Cockroach-32          224k ± 0%      224k ± 0%   -0.01%          (p=0.042 n=9+9)
@RaduBerinde
Copy link
Member

:lgtm:

Review status: 0 of 16 files reviewed at latest revision, all discussions resolved, some commit checks pending.

Comments from Reviewable

@danhhz
Copy link
Contributor Author

danhhz commented Jul 8, 2016

Running the benchmark in a loop with count=1 instead of using count=5 seems to have cleared up the InsertDistinct benchmark. Updated the commit message with the new results

Review status: 0 of 16 files reviewed at latest revision, all discussions resolved, all commit checks successful.

Comments from Reviewable

@danhhz danhhz merged commit a2c2244 into cockroachdb:master Jul 8, 2016
@danhhz danhhz deleted the family branch July 8, 2016 19:37
@ajwerner ajwerner mentioned this pull request Jan 8, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@RaduBerinde RaduBerinde

Labels
docs-done
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@danhhz @RaduBerinde @tbg @jseldess