An official OS X Homebrew formula/tap or Docker image

[hatarist]

---

Update (not about Homebrew) from @alexey-milovidov:

How to run ClickHouse on Mac (x86):

wget 'https://builds.clickhouse.com/master/macos/clickhouse'
chmod a+x ./clickhouse
./clickhouse

How to run ClickHouse on Mac (M1):

wget 'https://builds.clickhouse.com/master/macos-aarch64/clickhouse'
chmod a+x ./clickhouse
./clickhouse

---

A while ago I tried to create a formula for clickhouse.
It worked for me on 10.11.5 and 1.1.54074, but it wasn't accepted by the maintainers.

Their notes were:

This [ClickHouse?] and Boost need to be build using Clang.
Are unixodbc / glib / libtool / gettext linked to at runtime?
more convenient make install
Why max_open_files isn't default? (I'm not sure what's the default value for Linux builds)

See Homebrew/homebrew-core#7222 for more details.

If it's not possible/takes a lot of work to make it compatible with the current state of the project, I guess, there are two things that could be optimal, either:

1. Create a bottled-only formula (package) - precompile Clickhouse for multiple OS X versions and publish it to the official homebrew repository. The contents of the formula would be just like https://github.com/excitoon/homebrew-clickhouse/blob/master/clickhouse-client.rb. Not sure if maintainers would accept it.

or:

2. Create a tap (repository) inside of the Yandex namespace (a yandex/homebrew-clickhouse github repository) so it won't depend on homebrew's requirements/guidelines.

[pramitchoudhary]

Recently tried building the latest on Mac (10.13.6) but got the following error
steps:

1. cmake .. -DCMAKE_CXX_COMPILER=g++-8 -DCMAKE_C_COMPILER=gcc-8
2. ninja

also tried,
cmake .. -DCMAKE_CXX_COMPILER=g++-8 -DCMAKE_C_COMPILER=gcc-8 -DENABLE_EMBEDDED_COMPILER=0 -DUSE_INTERNAL_LLVM_LIBRARY=0

[2880/3324] FAILED: dbms/src/Functions/CMakeFiles/clickhouse_functions.dir/FunctionsGeo.cpp.o /usr/local/bin/g++-8 -DBOOST_SYSTEM_NO_DEPRECATED -DLZ4_DISABLE_DEPRECATE_WARNINGS=1 -DPOCO_STATIC -DPOCO_UNBUNDLED_ZLIB -DUNALIGNED_OK -DWITH_GZFILEOP -DX86_64 -DZLIB_COMPAT -I../dbms/src -Idbms/src -I../libs/libcommon/include -Ilibs/libcommon/include -I../contrib/zlib-ng -Icontrib/zlib-ng -I../contrib/cityhash102/include -I../contrib/croaring -I../libs/consistent-hashing -I../libs/consistent-hashing-sumbur -I../contrib/libfarmhash -I../contrib/murmurhash/include -I../contrib/simdjson/include -isystem ../contrib/libdivide -isystem ../contrib/libmetrohash/src -isystem /usr/local/opt/icu4c/include -isystem ../contrib/base64/include -isystem ../contrib/lz4/lib -isystem ../contrib/hyperscan/src -isystem ../contrib/rapidjson/include -isystem ../contrib/pdqsort -isystem ../contrib/libpcg-random/include -isystem ../contrib/double-conversion -isystem ../contrib/re2 -isystem ../contrib/poco/Foundation/include -isystem ../contrib/boost -isystem ../contrib/poco/Net/include -isystem ../contrib/poco/Util/include -isystem ../contrib/poco/XML/include -isystem ../contrib/poco/JSON/include -isystem contrib/re2_st -isystem ../contrib/ssl/include -fdiagnostics-color=always -pipe -msse4.1 -msse4.2 -mpopcnt -fno-omit-frame-pointer -Wall -Wnon-virtual-dtor -Wextra -Werror -O2 -g -DNDEBUG -O3 -fno-tree-loop-distribute-patterns -g0 -std=c++17 -MD -MT dbms/src/Functions/CMakeFiles/clickhouse_functions.dir/FunctionsGeo.cpp.o -MF dbms/src/Functions/CMakeFiles/clickhouse_functions.dir/FunctionsGeo.cpp.o.d -o dbms/src/Functions/CMakeFiles/clickhouse_functions.dir/FunctionsGeo.cpp.o -c ../dbms/src/Functions/FunctionsGeo.cpp ../dbms/src/Functions/FunctionsGeo.cpp: In member function 'virtual void DB::FunctionGeohashEncode::executeImpl(DB::Block&, const ColumnNumbers&, size_t, size_t)': ../dbms/src/Functions/FunctionsGeo.cpp:339:117: error: no matching function for call to 'min(const size_t&, long long unsigned int)' arguments.size() == 3 ? block.getByPosition(arguments[2]).column->get64(0) : GEOHASH_MAX_TEXT_LENGTH); ^ In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/memory:62, from ../dbms/src/Functions/IFunction.h:3, from ../dbms/src/Functions/FunctionFactory.h:3, from ../dbms/src/Functions/FunctionsGeo.cpp:1: /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/stl_algobase.h:195:5: note: candidate: 'template<class _Tp> constexpr const _Tp& std::min(const _Tp&, const _Tp&)' min(const _Tp& __a, const _Tp& __b) ^~~ /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/stl_algobase.h:195:5: note: template argument deduction/substitution failed: ../dbms/src/Functions/FunctionsGeo.cpp:339:117: note: deduced conflicting types for parameter 'const _Tp' ('long unsigned int' and 'long long unsigned int') arguments.size() == 3 ? block.getByPosition(arguments[2]).column->get64(0) : GEOHASH_MAX_TEXT_LENGTH); ^ In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/memory:62, from ../dbms/src/Functions/IFunction.h:3, from ../dbms/src/Functions/FunctionFactory.h:3, from ../dbms/src/Functions/FunctionsGeo.cpp:1: /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/stl_algobase.h:243:5: note: candidate: 'template<class _Tp, class _Compare> constexpr const _Tp& std::min(const _Tp&, const _Tp&, _Compare)' min(const _Tp& __a, const _Tp& __b, _Compare __comp) ^~~ /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/stl_algobase.h:243:5: note: template argument deduction/substitution failed: ../dbms/src/Functions/FunctionsGeo.cpp:339:117: note: deduced conflicting types for parameter 'const _Tp' ('long unsigned int' and 'long long unsigned int') arguments.size() == 3 ? block.getByPosition(arguments[2]).column->get64(0) : GEOHASH_MAX_TEXT_LENGTH); ^ In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/algorithm:62, from ../dbms/src/Core/Field.h:4, from ../dbms/src/Functions/IFunction.h:7, from ../dbms/src/Functions/FunctionFactory.h:3, from ../dbms/src/Functions/FunctionsGeo.cpp:1: /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/stl_algo.h:3450:5: note: candidate: 'template<class _Tp> constexpr _Tp std::min(std::initializer_list<_Tp>)' min(initializer_list<_Tp> __l) ^~~ /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/stl_algo.h:3450:5: note: template argument deduction/substitution failed: ../dbms/src/Functions/FunctionsGeo.cpp:339:117: note: mismatched types 'std::initializer_list<_Tp>' and 'long unsigned int' arguments.size() == 3 ? block.getByPosition(arguments[2]).column->get64(0) : GEOHASH_MAX_TEXT_LENGTH); ^ In file included from /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/algorithm:62, from ../dbms/src/Core/Field.h:4, from ../dbms/src/Functions/IFunction.h:7, from ../dbms/src/Functions/FunctionFactory.h:3, from ../dbms/src/Functions/FunctionsGeo.cpp:1: /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/stl_algo.h:3456:5: note: candidate: 'template<class _Tp, class _Compare> constexpr _Tp std::min(std::initializer_list<_Tp>, _Compare)' min(initializer_list<_Tp> __l, _Compare __comp) ^~~ /usr/local/Cellar/gcc@8/8.3.0/include/c++/8.3.0/bits/stl_algo.h:3456:5: note: template argument deduction/substitution failed: ../dbms/src/Functions/FunctionsGeo.cpp:339:117: note: mismatched types 'std::initializer_list<_Tp>' and 'long unsigned int' arguments.size() == 3 ? block.getByPosition(arguments[2]).column->get64(0) : GEOHASH_MAX_TEXT_LENGTH); ^ [216/651] Building CXX object dbms/src/Functions/CMakeFiles/clickhouse_functions.dir/FunctionsJSON.cpp.o

[touseefrafique]

@pramitchoudhary Were you able to get rid of this issue

[alexey-milovidov]

Now we have pre-built ClickHouse binary for OS X.

It does not solve the issue completely, but now we build it for every commit and every pull request in our CI with cross-compilation. This helps to ensure that native OS X build does not break after some commit. You can download the binary and use it directly as a client or server.

How to download:

• go to the list of commits: https://github.com/ClickHouse/ClickHouse/commits/master
• find commit with green "ClickHouse build check";
• click on details link of the check, example: https://clickhouse-builds.s3.yandex.net/7558/faf1b070ac982f51aa0fc0efac1e3ad4c0d9416e/report.html
• look at clang-8-darwin at the bottom;
• download the clickhouse binary.

There are still many things to do:

1. It does not support SSL, ICU and readline.
2. Publish download link on official site.
3. Add .tgz package instead of plain binary.

[alexey-milovidov]

Pre-built binaries are published in docs: https://clickhouse.tech/docs/en/getting-started/install/#from-binaries-non-linux

[develar]

No darwin artifacts anymore on "ClickHouse build check", and link on page https://clickhouse.tech/docs/en/getting-started/install/#from-binaries-non-linux for macOS is broken.

[develar]

Is it possible to build macOS for release branches? https://github.com/ClickHouse/ClickHouse/tree/21.2

ClickHouse special build check — Not ran for backport or release branches

Build from master is not good to use clickhouse to test/develop app on macOS since better to use the same version as in production. And Docker is not a solution because in this case a lot of RAM and CPU is needed to be allocated to VM.

Each update and build on macOS it is a lottery and in most cases will be some compilation error :)

[develar]

If someone cannot build stable 21.2 clickhouse on macOS — see #21128 (comment)

[marklit]

I would like to amplify the need for Homebrew-based installation support on macOS. On Debian systems, I can install a 4-day-old build of ClickHouse with a single command. On any Apple hardware, I have the choice of spending hours compiling ClickHouse, even on recent hardware, or using a 3rd-party and often out-of-date build.

Allowing developers to run brew install clickhouse and get a feel for the software's interfaces and behaviours will do a lot for adoption. I've been blogging about ClickHouse since 2017, the posts are popular with my readership. Around 33% of those readers are on an Apple-built laptop, letting them see this software runs great without the need of dedicated infrastructure will do wonders for adoption.

MySQL and PostgreSQL can be installed via Brew without issue. OmniSci has a single installer for macOS as well. It would be great to see ClickHouse provide official, recent and stable builds for Intel and M1-based macOS devices.

[alex-zaitsev]

@traceon is working on that

[alexey-milovidov]

@marklit

Mac x86:

wget 'https://builds.clickhouse.tech/master/macos/clickhouse'
chmod a+x ./clickhouse
./clickhouse

Mac M1:

wget 'https://builds.clickhouse.tech/master/macos-aarch64/clickhouse'
chmod a+x ./clickhouse
./clickhouse

[marklit]

That works. Cheers @alexey-milovidov

[qoega]

@marklit Totally agree with you!
Last time when we tried to submit formula to allow brew install clickhouse we met restrictions due to excessive code vendoring as we have all dependencies in git submodules and brew follows Debian like way with all libraries dynamically linked.

[traceon]

clickhouse-cpp was accepted. clickhouse-odbc is on its way to adding. I'll start with clickhouse-client package, and see how it goes. Ideally, we will be able to also push clickhouse-server trough. Will try to do native builds (bottles) and not casks.

[traceon]

@qoega are you going to retry your this Homebrew/homebrew-core#79378 attempt, or should I go ahead and make clickhouse-client/clickhouse-server as I planned?

[traceon]

FYI: you can do brew install clickhouse-odbc now.

[alexey-milovidov]

@qoega Do I understand correctly that:

1. The users can easily run ClickHouse on Mac as following: https://clickhouse.com/#quick-start

2. Homebrew does not accept binary packages and require dynamic linking with versions of the libraries from Homebrew itself.

3. We cannot allow this type of build, because we are using custom fixed versions of the libraries with patches. Otherwise minor differencies in libraries will lead to bugs, including security issues, and also we will open ourself to supply chain attacks.

Then why do we need Homebrew at all? Maybe close this issue?

[traceon]

Then why do we need Homebrew at all?

Oh, maybe because this issue is just slightly less popular that the entire Roadmap 2021?

[alexey-milovidov]

Is brew install clickhouse really better than
wget 'https://builds.clickhouse.com/master/macos/clickhouse' && chmod +x clickhouse?

[den-crane]

I think you should make an official docker build for M1 and encourage developers to use docker not Homebrew.

[den-crane]

It's kinda stupid to use Homebrew, because CH won't be a service under MacOs anyway.
People how voted for Homebrew don't know that MacOs build is lame and not fully functional.

[alexey-milovidov]

Ok.

[marklit]

Package management is about bringing in new users. There really is no other competitive and free columnar store on macOS. Being able to simply tell people in blog posts to run brew install clickhouse without needing to complicate it by asking which CPU architecture they're running and having the client in their PATH ready to go will make introducing new people to the software much easier.

I run Clickhouse on my MBP daily. It works fine for what I need and if I had to replace it I'd struggle. I've done benchmarks against PostgreSQL for needle-in-haystack searches and it ran circles around PG. If there are features missing I personally haven't missed them and I don't think that's justification for dismissing making ClickHouse install like all other CLI-centric software.

Also, on macOS, if you launch the server and close the Terminal it'll remain running in the background, even after reboots so it effectively is a service.

For reasons I won't get into here, I never recommend people wrap their databases in Docker containers if there is any viable alternative. I promise I'll never publish anything instructing people to do so with ClickHouse.

[traceon]

@den-crane it is not a problem to setup it as a service. It is missing some third-party stuff only because devs who integrated each of them didn't care to spend some more time in adapting the build scripts to it, that's the only reason I know of.

[develar]

It's kinda stupid to use Homebrew, because CH won't be a service under MacOs anyway.

I don't need it as a service. I need a prebuilt binary for macOS that matches my production version to test the app locally. Currently, prebuilt binary is provided only for master and so, it is not useful for my use case. Building from sources it is always a lottery due to numerous pitfalls and new compilation errors. Wasting time.

[develar]

For reasons I won't get into here, I never recommend people wrap their databases in Docker containers if there is any viable alternative

Regular docker setup is not suitable for Clickhouse. For toy apps maybe, but for any production-like usage, you often have to allocate more memory, CPU and so on. Changing Docker settings is possible, but it is annoying and time-wasting. Of course, you can have a separate minikube for Clickhouse, but it is complicated (e.g. volumes are not supported correctly and so on).

[traceon]

Altinity started maintaining a Homebrew Tap with ClickHouse (build by the formula using vanilla Clang 13).

1. Add the Tap to your Homebrew installation (needs to be executed only once):

brew tap altinity/clickhouse

2. Install ClickHouse:

brew install clickhouse

3. Read the Caveats section after the installation completes!
4. The recommended way of running the installed ClickHouse server is by using brew services:

brew services start clickhouse
brew services stop clickhouse

Note 1: All ClickHouse tools are available, including the client: clickhouse client --help.

Note 2: the installation doesn't require sudo and will deploy ClickHouse under the standard Homebrew prefix. That's why running the server is better to be done via brew services ... so that all command-line options are passed properly.

Note 3: Homebrew is available for Linux too!

For Intel-based macOS Bug Sur and above a precompiled binary will be downloaded. For other platforms (x86_64 Linux, or Apple silicon-based macOS) the install will start a lengthy build process, you may want to append --verbose to the above command to see the progress: brew install --verbose clickhouse. Eventually, bottles (precompiled binaries) will be added for those platforms too, as well as possible support for arm64 and power Linux by the formula will be investigated.

In order to install the latest version from the tip of the master branch (by building from sources), add --HEAD to the above command: brew install --verbose --HEAD clickhouse.

By default, the clickhouse formula will install the latest stable version. In the future, various other versioned formulae will become available for different versions and traits.

Please, report any problems during usage, compilation, or installation in the Issues section of the Tap.

[develar]

Downloading https://github.com/Altinity/homebrew-clickhouse/releases/download/clickhouse-21.11.4.14/clickhouse-21.11.4.14.big_sur.bottle.tar.gz

Awesome! I can test app locally using the same version as in production. Works great for me.