Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[GLUTEN-4106][VL] Initialize velox global memory manager #4111

Merged
merged 2 commits into from
Dec 20, 2023

Conversation

ulysses-you
Copy link
Contributor

@ulysses-you ulysses-you commented Dec 19, 2023

What changes were proposed in this pull request?

Fixes: #4106

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)

(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

Copy link

#4106

Comment on lines 396 to 398
instance_ = std::unique_ptr<VeloxBackend>(new gluten::VeloxBackend(conf));
// initialize the global memory manager for current process
facebook::velox::memory::MemoryManager::initialize({});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! And should we move the statement to function VeloxBackend::init?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, moved

Copy link
Contributor

@zhouyuan zhouyuan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Member

@zhztheplayer zhztheplayer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@ulysses-you ulysses-you merged commit 6de797b into apache:main Dec 20, 2023
15 of 16 checks passed
@ulysses-you ulysses-you deleted the MemoryManager branch December 20, 2023 07:45
@GlutenPerfBot
Copy link
Contributor

===== Performance report for TPCH SF2000 with Velox backend, for reference only ====

query log/native_4111_time.csv log/native_master_12_19_2023_521cad261_time.csv difference percentage
q1 32.79 32.55 -0.239 99.27%
q2 25.17 25.18 0.014 100.05%
q3 37.60 37.70 0.097 100.26%
q4 39.87 39.88 0.009 100.02%
q5 72.39 71.42 -0.971 98.66%
q6 6.79 5.41 -1.379 79.68%
q7 86.43 87.04 0.610 100.71%
q8 85.42 86.34 0.926 101.08%
q9 126.93 122.28 -4.648 96.34%
q10 45.88 44.36 -1.525 96.68%
q11 20.09 20.12 0.028 100.14%
q12 24.19 26.74 2.554 110.56%
q13 47.89 46.35 -1.534 96.80%
q14 14.91 17.44 2.531 116.98%
q15 28.12 29.09 0.967 103.44%
q16 15.34 15.70 0.360 102.35%
q17 102.65 102.87 0.223 100.22%
q18 152.17 149.94 -2.232 98.53%
q19 13.44 12.78 -0.660 95.09%
q20 27.24 28.17 0.926 103.40%
q21 226.77 227.31 0.536 100.24%
q22 13.93 13.95 0.022 100.16%
total 1246.01 1242.62 -3.386 99.73%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Velox memory manager throw exception
4 participants