Skip to content

Commit

Permalink
[SPARK-31606][SQL] Reduce the perf regression of vectorized parquet r…
Browse files Browse the repository at this point in the history 
…eader caused by datetime rebase

### What changes were proposed in this pull request?

Push the rebase logic to the lower level of the parquet vectorized reader, to make the final code more vectorization-friendly.

### Why are the changes needed?

Parquet vectorized reader is carefully implemented, to make it more likely to be vectorized by the JVM. However, the newly added datetime rebase degrade the performance a lot, as it breaks vectorization, even if the datetime values don't need to rebase (this is very likely as dates before 1582 is rare).

### Does this PR introduce any user-facing change?

no

### How was this patch tested?

Run part of the `DateTimeRebaseBenchmark` locally. The results:
before this patch
```
[info] Load dates from parquet:                  Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
[info] ------------------------------------------------------------------------------------------------------------------------
[info] after 1582, vec on, rebase off                     2677           2838         142         37.4          26.8       1.0X
[info] after 1582, vec on, rebase on                      3828           4331         805         26.1          38.3       0.7X
[info] before 1582, vec on, rebase off                    2903           2926          34         34.4          29.0       0.9X
[info] before 1582, vec on, rebase on                     4163           4197          38         24.0          41.6       0.6X

[info] Load timestamps from parquet:             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
[info] ------------------------------------------------------------------------------------------------------------------------
[info] after 1900, vec on, rebase off                     3537           3627         104         28.3          35.4       1.0X
[info] after 1900, vec on, rebase on                      6891           7010         105         14.5          68.9       0.5X
[info] before 1900, vec on, rebase off                    3692           3770          72         27.1          36.9       1.0X
[info] before 1900, vec on, rebase on                     7588           7610          30         13.2          75.9       0.5X
```

After this patch
```
[info] Load dates from parquet:                  Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
[info] ------------------------------------------------------------------------------------------------------------------------
[info] after 1582, vec on, rebase off                     2758           2944         197         36.3          27.6       1.0X
[info] after 1582, vec on, rebase on                      2908           2966          51         34.4          29.1       0.9X
[info] before 1582, vec on, rebase off                    2840           2878          37         35.2          28.4       1.0X
[info] before 1582, vec on, rebase on                     3407           3433          24         29.4          34.1       0.8X

[info] Load timestamps from parquet:             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
[info] ------------------------------------------------------------------------------------------------------------------------
[info] after 1900, vec on, rebase off                     3861           4003         139         25.9          38.6       1.0X
[info] after 1900, vec on, rebase on                      4194           4283          77         23.8          41.9       0.9X
[info] before 1900, vec on, rebase off                    3849           3937          79         26.0          38.5       1.0X
[info] before 1900, vec on, rebase on                     7512           7546          55         13.3          75.1       0.5X
```

Date type is 30% faster if the values don't need to rebase, 20% faster if need to rebase.
Timestamp type is 60% faster if the values don't need to rebase, no difference if need to rebase.

Closes apache#28406 from cloud-fan/perf.

Lead-authored-by: Wenchen Fan <wenchen@databricks.com>
Co-authored-by: Maxim Gekk <max.gekk@gmail.com>
Signed-off-by: HyukjinKwon <gurwls223@apache.org>
  • Loading branch information
@cloud-fan @MaxGekk @huaxingao
2 people authored and huaxingao committed May 4, 2020
1 parent 065871c commit 931c0bc
Show file tree
Hide file tree
Showing 8 changed files with 295 additions and 160 deletions.
4 changes: 4 additions & 0 deletions sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RebaseDateTime.scala
View file
Expand Up @@ -71,6 +71,8 @@ object RebaseDateTime {
-719164, -682945, -646420, -609895, -536845, -500320, -463795,
-390745, -354220, -317695, -244645, -208120, -171595, -141427)

final val lastSwitchJulianDay: Int = julianGregDiffSwitchDay.last

// The first days of Common Era (CE) which is mapped to the '0001-01-01' date in Julian calendar.
private final val julianCommonEraStartDay = julianGregDiffSwitchDay(0)

Expand Down Expand Up @@ -416,6 +418,8 @@ object RebaseDateTime {
// in the interval: [julianGregDiffSwitchMicros(i), julianGregDiffSwitchMicros(i+1))
private val julianGregRebaseMap = loadRebaseRecords("julian-gregorian-rebase-micros.json")

final val lastSwitchJulianTs: Long = julianGregRebaseMap.values.map(_.switches.last).max

/**
* An optimized version of [[rebaseJulianToGregorianMicros(ZoneId, Long)]]. This method leverages
* the pre-calculated rebasing maps to save calculation. If the rebasing map doesn't contain
Expand Down
104 changes: 52 additions & 52 deletions sql/core/benchmarks/DateTimeRebaseBenchmark-jdk11-results.txt
View file
Expand Up @@ -2,93 +2,93 @@
Rebasing dates/timestamps in Parquet datasource
================================================================================================

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save dates to parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 21171 21171 0 4.7 211.7 1.0X
before 1582, noop 11036 11036 0 9.1 110.4 1.9X
after 1582, rebase off 34321 34321 0 2.9 343.2 0.6X
after 1582, rebase on 33269 33269 0 3.0 332.7 0.6X
before 1582, rebase off 22016 22016 0 4.5 220.2 1.0X
before 1582, rebase on 23338 23338 0 4.3 233.4 0.9X
after 1582, noop 20073 20073 0 5.0 200.7 1.0X
before 1582, noop 10985 10985 0 9.1 109.9 1.8X
after 1582, rebase off 32245 32245 0 3.1 322.4 0.6X
after 1582, rebase on 31434 31434 0 3.2 314.3 0.6X
before 1582, rebase off 21590 21590 0 4.6 215.9 0.9X
before 1582, rebase on 22963 22963 0 4.4 229.6 0.9X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load dates from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off, rebase off 12791 13089 287 7.8 127.9 1.0X
after 1582, vec off, rebase on 13203 13271 81 7.6 132.0 1.0X
after 1582, vec on, rebase off 3709 3764 49 27.0 37.1 3.4X
after 1582, vec on, rebase on 5082 5114 29 19.7 50.8 2.5X
before 1582, vec off, rebase off 13059 13153 87 7.7 130.6 1.0X
before 1582, vec off, rebase on 14211 14236 27 7.0 142.1 0.9X
before 1582, vec on, rebase off 3687 3749 72 27.1 36.9 3.5X
before 1582, vec on, rebase on 5449 5497 56 18.4 54.5 2.3X
after 1582, vec off, rebase off 12815 12858 40 7.8 128.1 1.0X
after 1582, vec off, rebase on 13030 13167 148 7.7 130.3 1.0X
after 1582, vec on, rebase off 3705 3712 6 27.0 37.1 3.5X
after 1582, vec on, rebase on 3788 3791 3 26.4 37.9 3.4X
before 1582, vec off, rebase off 12873 12943 61 7.8 128.7 1.0X
before 1582, vec off, rebase on 14072 14165 80 7.1 140.7 0.9X
before 1582, vec on, rebase off 3694 3708 15 27.1 36.9 3.5X
before 1582, vec on, rebase on 4403 4484 81 22.7 44.0 2.9X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save timestamps to parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 2831 2831 0 35.3 28.3 1.0X
before 1582, noop 2816 2816 0 35.5 28.2 1.0X
after 1582, rebase off 15543 15543 0 6.4 155.4 0.2X
after 1582, rebase on 18391 18391 0 5.4 183.9 0.2X
before 1582, rebase off 15747 15747 0 6.4 157.5 0.2X
before 1582, rebase on 18846 18846 0 5.3 188.5 0.2X
after 1900, noop 3032 3032 0 33.0 30.3 1.0X
before 1900, noop 3043 3043 0 32.9 30.4 1.0X
after 1900, rebase off 15634 15634 0 6.4 156.3 0.2X
after 1900, rebase on 18233 18233 0 5.5 182.3 0.2X
before 1900, rebase off 15820 15820 0 6.3 158.2 0.2X
before 1900, rebase on 19921 19921 0 5.0 199.2 0.2X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load timestamps from parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off, rebase off 16126 16216 78 6.2 161.3 1.0X
after 1582, vec off, rebase on 18277 18453 165 5.5 182.8 0.9X
after 1582, vec on, rebase off 5030 5067 42 19.9 50.3 3.2X
after 1582, vec on, rebase on 8553 8583 43 11.7 85.5 1.9X
before 1582, vec off, rebase off 15828 15872 39 6.3 158.3 1.0X
before 1582, vec off, rebase on 18899 18959 103 5.3 189.0 0.9X
before 1582, vec on, rebase off 4961 5009 43 20.2 49.6 3.3X
before 1582, vec on, rebase on 9099 9140 40 11.0 91.0 1.8X
after 1900, vec off, rebase off 14987 15008 18 6.7 149.9 1.0X
after 1900, vec off, rebase on 17500 17628 210 5.7 175.0 0.9X
after 1900, vec on, rebase off 5030 5036 7 19.9 50.3 3.0X
after 1900, vec on, rebase on 5066 5109 44 19.7 50.7 3.0X
before 1900, vec off, rebase off 15094 15213 121 6.6 150.9 1.0X
before 1900, vec off, rebase on 18098 18175 101 5.5 181.0 0.8X
before 1900, vec on, rebase off 5008 5012 4 20.0 50.1 3.0X
before 1900, vec on, rebase on 8803 8848 55 11.4 88.0 1.7X


================================================================================================
Rebasing dates/timestamps in ORC datasource
================================================================================================

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save dates to ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 21026 21026 0 4.8 210.3 1.0X
before 1582, noop 11040 11040 0 9.1 110.4 1.9X
after 1582 28171 28171 0 3.5 281.7 0.7X
before 1582 18955 18955 0 5.3 189.5 1.1X
after 1582, noop 19593 19593 0 5.1 195.9 1.0X
before 1582, noop 10581 10581 0 9.5 105.8 1.9X
after 1582 27843 27843 0 3.6 278.4 0.7X
before 1582 19435 19435 0 5.1 194.4 1.0X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load dates from ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off 10876 10931 49 9.2 108.8 1.0X
after 1582, vec on 3900 3913 20 25.6 39.0 2.8X
before 1582, vec off 11165 11174 12 9.0 111.6 1.0X
before 1582, vec on 4208 4214 7 23.8 42.1 2.6X
after 1582, vec off 10395 10507 119 9.6 103.9 1.0X
after 1582, vec on 3921 3945 22 25.5 39.2 2.7X
before 1582, vec off 10762 10860 127 9.3 107.6 1.0X
before 1582, vec on 4194 4226 41 23.8 41.9 2.5X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Save timestamps to ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, noop 2924 2924 0 34.2 29.2 1.0X
before 1582, noop 2820 2820 0 35.5 28.2 1.0X
after 1582 22228 22228 0 4.5 222.3 0.1X
before 1582 22590 22590 0 4.4 225.9 0.1X
after 1900, noop 3003 3003 0 33.3 30.0 1.0X
before 1900, noop 3016 3016 0 33.2 30.2 1.0X
after 1900 21804 21804 0 4.6 218.0 0.1X
before 1900 23920 23920 0 4.2 239.2 0.1X

OpenJDK 64-Bit Server VM 11.0.6+10-post-Ubuntu-1ubuntu118.04.1 on Linux 4.15.0-1063-aws
OpenJDK 64-Bit Server VM 11.0.7+10-post-Ubuntu-2ubuntu218.04 on Linux 4.15.0-1063-aws
Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz
Load timestamps from ORC: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative
------------------------------------------------------------------------------------------------------------------------
after 1582, vec off 13591 13658 59 7.4 135.9 1.0X
after 1582, vec on 7399 7488 126 13.5 74.0 1.8X
before 1582, vec off 14065 14096 30 7.1 140.7 1.0X
before 1582, vec on 7950 8127 249 12.6 79.5 1.7X
after 1900, vec off 14112 14128 17 7.1 141.1 1.0X
after 1900, vec on 7347 7459 134 13.6 73.5 1.9X
before 1900, vec off 15170 15192 27 6.6 151.7 0.9X
before 1900, vec on 8280 8312 52 12.1 82.8 1.7X


0 comments on commit 931c0bc

Please sign in to comment.