Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[VL] enable test 'as map of case class - reorder fields by name' #4028

Merged
merged 1 commit into from
Dec 13, 2023
Merged

[VL] enable test 'as map of case class - reorder fields by name' #4028

merged 1 commit into from
Dec 13, 2023

Conversation

zhli1142015
Copy link
Contributor

What changes were proposed in this pull request?

UT change. Complex type support for map functions is added in facebookincubator/velox#7560.

-- Project[expressions: (n1_1:MAP<INTEGER,ROW<b:INTEGER,a:VARCHAR>>, map(1,row_constructor(try_cast "n0_0" as INTEGER,"a")))] -> n1_1:MAP<INTEGER,ROW<b:INTEGER,a:VARCHAR>>
  -- ValueStream[] -> n0_0:BIGINT

@github-actions GitHub Actions
Copy link

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/oap-project/gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:

@github-actions GitHub Actions
Copy link

Run Gluten Clickhouse CI

@zhli1142015
Copy link
Contributor Author

cc @rui-mo , thanks.

rui-mo
rui-mo approved these changes Dec 13, 2023
View reviewed changes
Copy link
Contributor

@rui-mo rui-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. We can merge it once CI passes.

@zhli1142015
Copy link
Contributor Author

Run Gluten Clickhouse CI

@zhli1142015
Copy link
Contributor Author

Looks CK validation failure is not related to this change. CC @baibaichen , @loneylee .
Thanks.

20:51:47  All tests passed.
20:51:49  [INFO] ------------------------------------------------------------------------
20:51:49  [INFO] Reactor Summary for Gluten Parent Pom 1.1.0-SNAPSHOT:
20:51:49  [INFO] 
20:51:49  [INFO] Gluten Parent Pom .................................. SUCCESS [  1.182 s]
20:51:49  [INFO] Gluten UI .......................................... SUCCESS [ 25.657 s]
20:51:49  [INFO] Gluten Shims ....................................... SUCCESS [  2.456 s]
20:51:49  [INFO] Gluten Shims Common ................................ SUCCESS [ 19.941 s]
20:51:49  [INFO] Gluten Shims for Spark 3.3 ......................... SUCCESS [ 23.826 s]
20:51:49  [INFO] Gluten Core ........................................ SUCCESS [01:29 min]
20:51:49  [INFO] Gluten Backends ClickHouse ......................... SUCCESS [56:30 min]
20:51:49  [INFO] Gluten Celeborn .................................... SUCCESS [  0.614 s]
20:51:49  [INFO] Gluten Celeborn Common ............................. SUCCESS [  6.327 s]
20:51:49  [INFO] Gluten Celeborn Clickhouse ......................... SUCCESS [01:04 min]
20:51:49  [INFO] Gluten Celeborn Package ............................ SUCCESS [  0.453 s]
20:51:49  [INFO] Gluten Package ..................................... SUCCESS [  5.385 s]
20:51:49  [INFO] Gluten Substrait Spark ............................. SUCCESS [ 32.892 s]
20:51:49  [INFO] Gluten Unit Test ................................... SUCCESS [  0.312 s]
20:51:49  [INFO] Gluten Unit Test Common ............................ SUCCESS [ 13.022 s]
20:51:49  [INFO] Gluten Unit Test Spark33 ........................... SUCCESS [40:47 min]
20:51:49  [INFO] ------------------------------------------------------------------------
20:51:49  [INFO] BUILD SUCCESS
20:51:49  [INFO] ------------------------------------------------------------------------
20:51:49  [INFO] Total time:  01:42 h
20:51:49  [INFO] Finished at: 2023-12-13T12:51:49Z
20:51:49  [INFO] ------------------------------------------------------------------------
[Pipeline] echo
20:51:49  enter finally
[Pipeline] sh
20:51:50  + ls -l
20:51:50  total 138692
20:51:50  -rw-r--r--  1 root root      4147 Dec 13 10:33 CONTRIBUTING.md
20:51:50  -rw-r--r--  1 root root     94051 Dec 13 10:33 LICENSE
20:51:50  -rw-r--r--  1 root root     10761 Dec 13 10:33 README.md
20:51:50  -rw-r--r--  1 root root       431 Dec 13 10:33 SECURITY.md
20:51:50  -rw-r--r--  1 root root    325088 Dec 13 10:33 TPP.txt
20:51:50  drwxr-xr-x 15 root root       247 Dec 13 10:33 apache-celeborn-0.3.0-incubating-bin
20:51:50  -rw-r--r--  1 root root 141533645 Dec 13 10:33 apache-celeborn-0.3.0-incubating-bin.tgz
20:51:50  drwxr-xr-x  9 root root       255 Dec 13 11:59 backends-clickhouse
20:51:50  drwxr-xr-x  4 root root        49 Dec 13 10:33 backends-velox
20:51:50  drwxr-xr-x  5 root root       124 Dec 13 10:33 cpp
20:51:50  drwxr-xr-x  3 root root        95 Dec 13 10:33 cpp-ch
20:51:50  drwxr-xr-x  3 root root      4096 Dec 13 10:33 dev
20:51:50  drwxr-xr-x  6 root root       282 Dec 13 10:33 docs
20:51:50  drwxr-xr-x  4 root root        49 Dec 13 10:33 ep
20:51:50  drwxr-xr-x  6 root root        81 Dec 13 10:33 gluten-celeborn
20:51:50  drwxr-xr-x  5 root root        98 Dec 13 11:12 gluten-core
20:51:50  drwxr-xr-x  3 root root        32 Dec 13 10:33 gluten-data
20:51:50  drwxr-xr-x  3 root root        32 Dec 13 10:33 gluten-delta
20:51:50  drwxr-xr-x  3 root root        32 Dec 13 10:33 gluten-iceberg
20:51:50  drwxr-xr-x  4 root root        46 Dec 13 11:09 gluten-ui
20:51:50  drwxr-xr-x  6 root root        80 Dec 13 10:33 gluten-ut
20:51:50  -rw-r--r--  1 root root      1441 Dec 13 10:33 mkdocs.yml
20:51:50  drwxr-xr-x  2 root root        21 Dec 13 10:33 package
20:51:50  -rw-r--r--  1 root root     28076 Dec 13 10:33 pom.xml
20:51:50  drwxr-xr-x  6 root root        80 Dec 13 10:33 shims
20:51:50  drwxr-xr-x  3 root root        29 Dec 13 10:33 substrait
20:51:50  drwxr-xr-x  5 root root        56 Dec 13 10:33 tools
[Pipeline] sh
20:51:51  + pwd
20:51:51  /home/jenkins/agent/workspace/gluten/gluten-ci/gluten2
[Pipeline] sh
20:51:51  + mkdir -p /maven-log/5272/3.3.1/backends-clickhouse
[Pipeline] sh
20:51:52  + mkdir -p /maven-log/5272/3.3.1/spark33
[Pipeline] sh
20:51:53  + pwd
20:51:53  + cp -r /home/jenkins/agent/workspace/gluten/gluten-ci/gluten2/backends-clickhouse /maven-log/5272/3.3.1/backends-clickhouse
[Pipeline] sh
20:52:25  + pwd
20:52:25  + cp -r /home/jenkins/agent/workspace/gluten/gluten-ci/gluten2/gluten-ut/spark33 /maven-log/5272/3.3.1/spark33
[Pipeline] sh
20:52:34  + ls -l /maven-log/5272/3.3.1/backends-clickhouse
20:52:34  total 4
20:52:34  drwxr-xr-x 9 1000 1000 6144 Dec 13 12:52 backends-clickhouse
[Pipeline] sh
20:52:35  + ls -l /maven-log/5272/3.3.1/spark33
20:52:35  total 4
20:52:35  drwxr-xr-x 5 1000 1000 6144 Dec 13 12:52 spark33
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // dir
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Declarative: Post Actions)
[Pipeline] container
[Pipeline] {
[Pipeline] script
[Pipeline] {
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // timestamps
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Setting status of 39ad0fb04089de90d737cada840941d3b7ffe5a7 to FAILURE with url https://opencicd.kyligence.com/job/gluten/job/gluten-ci/5272/ and message: 'Something went really wrong. Investigate!
 '
Using context: Gluten ClickHouse CI
Finished: FAILURE

@zhli1142015 zhli1142015 merged commit 8bb8c14 into apache:main Dec 13, 2023
16 of 17 checks passed
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_4028_time.csv log/native_master_12_12_2023_8c6f1643e_time.csv difference percentage
q1 33.03 33.75 0.711 102.15%
q2 25.38 24.88 -0.500 98.03%
q3 37.85 37.71 -0.141 99.63%
q4 38.56 37.86 -0.704 98.18%
q5 72.54 72.83 0.297 100.41%
q6 5.31 7.05 1.737 132.68%
q7 84.86 86.50 1.641 101.93%
q8 87.74 86.24 -1.503 98.29%
q9 126.02 125.50 -0.519 99.59%
q10 44.54 44.68 0.143 100.32%
q11 20.20 20.21 0.016 100.08%
q12 24.55 25.06 0.513 102.09%
q13 47.42 46.58 -0.838 98.23%
q14 18.52 19.99 1.471 107.94%
q15 28.78 27.55 -1.231 95.72%
q16 15.89 15.86 -0.027 99.83%
q17 102.87 103.27 0.399 100.39%
q18 150.12 151.33 1.209 100.81%
q19 14.17 13.89 -0.275 98.06%
q20 28.70 28.68 -0.017 99.94%
q21 228.46 229.12 0.661 100.29%
q22 14.04 13.95 -0.090 99.36%
total 1249.54 1252.49 2.952 100.24%

@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_master_12_13_2023_time.csv log/native_master_12_12_2023_8c6f1643e_time.csv difference percentage
q1 35.15 33.75 -1.403 96.01%
q2 25.05 24.88 -0.169 99.33%
q3 37.75 37.71 -0.044 99.88%
q4 39.49 37.86 -1.634 95.86%
q5 71.76 72.83 1.075 101.50%
q6 7.80 7.05 -0.743 90.46%
q7 87.10 86.50 -0.605 99.31%
q8 84.82 86.24 1.423 101.68%
q9 122.63 125.50 2.874 102.34%
q10 47.06 44.68 -2.380 94.94%
q11 20.42 20.21 -0.209 98.98%
q12 25.75 25.06 -0.691 97.32%
q13 46.78 46.58 -0.206 99.56%
q14 14.73 19.99 5.253 135.65%
q15 27.69 27.55 -0.147 99.47%
q16 15.51 15.86 0.348 102.24%
q17 102.08 103.27 1.186 101.16%
q18 151.81 151.33 -0.476 99.69%
q19 12.69 13.89 1.198 109.44%
q20 27.12 28.68 1.566 105.77%
q21 229.47 229.12 -0.342 99.85%
q22 14.16 13.95 -0.208 98.53%
total 1246.83 1252.49 5.666 100.45%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rui-mo rui-mo rui-mo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@zhli1142015 @GlutenPerfBot @rui-mo