Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AVX2 Support #7

Closed
lmangani opened this issue Apr 15, 2023 · 6 comments
Closed

AVX2 Support #7

lmangani opened this issue Apr 15, 2023 · 6 comments
Labels
question Further information is requested

Comments

@lmangani
Copy link
Contributor

lmangani commented Apr 15, 2023

Great mod and very interesting binding approach! We're also playing with clickhouse-local custom builds towards a slimmer execution engine.

Question: Is there a specific reason for AVX2 to be enabled for Linux systems? This greatly reduces the scope of usage (ie: lambdas, virtualized cpus, etc, fail with Instruction check fail. The CPU does not support AVX2 instruction set.)

Having this disabled (or possibly multiple versions) could broaden compatibility scope enormously. Also the library doesn't seem to be stripped and the distributed .so has 100M+ of trimmable fat.

Thanks!

@lmangani lmangani added the question Further information is requested label Apr 15, 2023
@auxten
Copy link
Member

auxten commented Apr 15, 2023

Thank you for your suggestion. Before, I just thought that most Linux servers should have AVX2 support for performance considerations. Didn't realize usage scenarios like lambdas, virtualized cpus. I'm going to fix this in v0.5.0.

@lmangani
Copy link
Contributor Author

Thanks! If you would like to support AWS Lambdas and other virtualized system the following patch is also required:

sed -i '/Cannot set thread name/c\' /ClickHouse/src/Common/setThreadName.cpp

Would you accept PRs to the builder?

@auxten
Copy link
Member

auxten commented Apr 15, 2023

PR is welcome.

@auxten
Copy link
Member

auxten commented Apr 15, 2023

The strip issue is mainly because currently chdb is really new. I need debug info if any crash happens.
The shared lib is built in release mode but with -g flag which means not so much(increased the size by approximately 20%) debug info included.

@lmangani
Copy link
Contributor Author

That makes sense, although the resulting file is really huge. chdb x86_64 library as installed by pip3

-rwxr-xr-x   1 root staff 527420816 Apr 15 15:37 _chdb.cpython-37m-x86_64-linux-gnu.so*

after strip

-rwxr-xr-x   1 root staff 411978416 Apr 15 15:55 _chdb.cpython-37m-x86_64-linux-gnu.so*

@lmangani
Copy link
Contributor Author

PR open, moving discussion in thread: #9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants