Skip to content
Modified version of MariaDB for ColumnStore
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
BUILD Renamed compile-pentium scripts to compile-pentium32 Apr 16, 2018
Docs Misc. typos Apr 5, 2018
client Merge 10.2 into 10.3 Nov 7, 2018
cmake port changes for mysql_version cmake to fix columnstore RPM packaging Nov 20, 2018
dbug Misc. typos Apr 5, 2018
debian MCOL-1385 Modify MariaDB 10.3 to work with CS Nov 20, 2018
extra Merge 10.2 into 10.3 Nov 6, 2018
include MDEV-16697: Fix difference between 32bit/windows and 64bit systems in… Nov 7, 2018
libmariadb @ 334964f C/C 3.0.7 Nov 12, 2018
libmysqld Merge 10.2 into 10.3 Nov 6, 2018
libservices Remove compiler warnings Jan 30, 2018
man Merge 10.2 into 10.3 Sep 6, 2018
mysql-test Updated list of unstable tests for 10.3.11 release Nov 16, 2018
mysys Merge 10.2 into 10.3 Nov 7, 2018
mysys_ssl Windows : fix broken build with OpenSSL Sep 8, 2018
pcre fix bundled pcre unit tests Aug 13, 2018
plugin Merge 10.2 into 10.3 Nov 6, 2018
randgen/conf Group commit for maria engine. Feb 12, 2010
scripts Revert "MDEV-11036 Add link wsrep_sst_rsync_wan -> wsrep_sst_rsync" Nov 20, 2018
sql-bench Added more test to sql-bench Apr 30, 2018
sql-common Merge 10.2 into 10.3 Aug 3, 2018
sql MCOL-1790 Remove case type hack Nov 21, 2018
storage Merge branch '10.2' into 10.3 Nov 15, 2018
strings Merge branch '10.2' into 10.3 Nov 15, 2018
support-files MCOL-1385 Modify MariaDB 10.3 to work with CS Nov 20, 2018
tests Merge 10.2 into 10.3 Oct 9, 2018
unittest Merge 10.2 into 10.3 Nov 6, 2018
vio Merge 10.2 into 10.3 Nov 7, 2018
win Merge branch '10.2' into 10.3 Sep 28, 2018
wsrep Prevent building WSREP without INNODB Feb 15, 2018
zlib Merge 10.1 into 10.2 Nov 6, 2018
.gitattributes Merge branch '10.0' into 10.1 May 5, 2018
.gitignore Revert "MDEV-11036 Add link wsrep_sst_rsync_wan -> wsrep_sst_rsync" Nov 20, 2018
.gitmodules Use https instead of ssh(git@) for rocksdb submodule. Nov 15, 2016
.travis.compiler.sh travis: increase ccache size for linux jobs Nov 8, 2018
.travis.yml travis: llvm repo gpg key missing - use travis defination Aug 21, 2018
BUILD-CMAKE Minor spelling fixes in code comments, docs and output Jan 12, 2018
CMakeLists.txt Change package Summary field for all packages. Nov 20, 2018
COPYING Import GPLv2,tokudb License Apr 17, 2013
COPYING.thirdparty Correct FSF address Mar 10, 2017
CREDITS Update contributors Sep 10, 2018
EXCEPTIONS-CLIENT MDEV-5645 MariaDB-5.5.35 - references are made to an "EXCEPTIONS-CLIE… May 31, 2014
INSTALL-SOURCE Update AskMonty and Atlassian references to MariaDB Mar 8, 2016
INSTALL-WIN-SOURCE Update AskMonty and Atlassian references to MariaDB Mar 8, 2016
KNOWN_BUGS.txt various documentation updates Jun 18, 2012
README.md update Readm.md Nov 26, 2018
VERSION MCOL-1385 Modify MariaDB 10.3 to work with CS Nov 20, 2018
appveyor.yml Add some hints for finding bison on its usual locations on Windows. Feb 14, 2018
config.h.cmake Merge branch '10.2' into 10.3 Sep 28, 2018
configure.cmake Merge 10.2 into 10.3 Nov 6, 2018

README.md

MariaDB ColumnStore Server (version 1.2)

This is the server part of MariaDB ColumnStore 1.2. MariaDB ColumnStore 1.2 is an GA of MariaDB ColumnStore. It is built by porting InfiniDB 4.6.7 on MariaDB 10.3 and adding entirely new features not found anywhere else.

MariaDB ColumnStore Engine (version 1.2)

MariaDB ColumnStore also requires the matching engine version. This can be found at https://github.com/mariadb-corporation/mariadb-columnstore-engine.

Always match the server engine / git branch with the engine git branch.

Currently building has only been certified on CentOS 6 and 7, Ubuntu 16.04, Debain 8, and SUSE 12.. Building on other platforms will be certified in a later release.

Issue tracking

Issue tracking of MariaDB ColumnStore happens in JIRA, https://jira.mariadb.org/browse/MCOL

The structure of this repository is:

  • Branch "master" - this is the latest released version of the source code. Each major release is tagged.
  • Branch "develop-1.0" - this is the 1.0 legacy mainline development branch.
  • Branch "develop-1.1" - this is the 1.1 mainline development branch.
  • Branch "develop" - this is the 1.2 unstable development branch.
  • Branch "mcol-xxx" - these are specific bug and feature branches. These are merged into development which is merged to master.

MariaDB ColumnStore server and the engine are in separate repositories.

Contributing

To contribute to ColumnStore please see the Contributions Documentation.

Build dependencies

Boost Libraries

MariaDB ColumnStore requires that the boost package of 1.53 or newer is installed for both building and executing

For CentOS 7, Ubuntu 16, Debian 8, SUSE 12 and other newer OS's, you can just install the boost packages via yum or apt-get.

yum install boost-devel

or

apt-get install libboost-dev-all

or

SUSEConnect -p sle-sdk/12.2/x86_64

zypper install boost-devel

For CentOS 6, you can either download and install the MariaDB Columnstore Centos 6 boost library package or install the boost source of 1.55 and build it to generate the required libraries. That means both the build and the install machines require this.

Downloading and installing the MariaDB Columnstore Centos 6 boost library package is documented here:

https://mariadb.com/kb/en/mariadb/preparing-for-columnstore-installation/#boost-libraries

Downloading and build the boost libraries:

NOTE: This means that the "Development Tools" group install be done prior to this.

yum groupinstall "Development Tools"
yum install cmake

Here is the procedure to download and build the boost source:

cd /usr/

wget http://sourceforge.net/projects/boost/files/boost/1.55.0/boost_1_55_0.tar.gz

tar zxvf boost_1_55_0.tar.gz

cd boost_1_55_0

./bootstrap.sh --with-libraries=atomic,date_time,exception,filesystem,iostreams,locale,program_options,regex,signals,system,test,thread,timer,log --prefix=/usr

./b2 install

ldconfig

For CentOS

These packages need to be install along with the group development packages:

yum groupinstall "Development Tools"
yum install bison ncurses-devel readline-devel perl-devel openssl-devel cmake libxml2-devel gperf libaio-devel libevent-devel python-devel ruby-devel tree wget pam-devel snappy-devel

For Ubuntu 16/18

apt-get install build-essential automake libboost-all-dev bison cmake libncurses5-dev libreadline-dev libperl-dev libssl-dev libxml2-dev libkrb5-dev flex libpam-dev libsnappy-dev

For Debian 8

apt-get install build-essential automake libboost-all-dev bison cmake libncurses5-dev libreadline-dev libperl-dev libssl-dev libxml2-dev libkrb5-dev flex libpam-dev libkrb5-dev libsnappy-dev

For Debian 9

apt-get install build-essential automake libboost-all-dev bison cmake libncurses5-dev libreadline-dev libperl-dev libssl1.0-dev libxml2-dev libkrb5-dev flex libpam-dev libkrb5-dev libsnappy-dev libgmp-dev
apt-get install libxml2-dev lmodern openjdk-8-jdk openssl pkg-config scons texlive-latex-base texlive-latex-base-doc thrift-compiler ttf-dejavu ttf-dejavu-extra unixodbc-dev uuid-dev xfonts-encodings xfonts-utils zlib1g-dev

For Suse 12

These packages need to be install along with the group development packages:

zypper se -t pattern devel
zypper install bison ncurses-devel readline-devel perl-devel openssl-devel cmake libxml2-devel gperf libaio-devel libevent-devel python-devel ruby-devel tree wget pam-devel snappy-devel

Building master branch

The current master branch is the released version.

Building develop branch

The develop branch is used for develop updates

Building can be done as a non-root user. If you do a "build install", it will install the binaries in /usr/local/mariadb/columnstore and the use of sudo is required.

To build the current development branch binaries only (Engine checkout inside Server):

git clone https://github.com/mariadb-corporation/mariadb-columnstore-server.git
cd mariadb-columnstore-server
git checkout develop # switch to develop code
cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mariadb/columnstore/mysql
make -jN # N is the number of concurrent build processes and should likely be the number of cores available
sudo make install
git clone https://github.com/mariadb-corporation/mariadb-columnstore-engine.git
cd mariadb-columnstore-engine
git checkout develop
cmake .
make -jN # same as above with respect to concurrent processes
sudo make install

To build the current development branch binaries and packages only (Engine checkout inside Server):

git clone https://github.com/mariadb-corporation/mariadb-columnstore-server.git
cd mariadb-columnstore-server
git checkout develop # switch to develop code
run cmake
For RPMs:
cmake . -DWITH_READLINE=1 -DRPM=centos6 -DPLUGIN_CONNECT=NO -DWITH_WSREP=OFF -DINSTALL_LAYOUT=RPM -DCMAKE_INSTALL_PREFIX=/usr/local/mariadb/columnstore/mysql  -DCPACK_RPM_EXCLUDE_FROM_AUTO_FILELIST_ADDITION=/usr/local
For DEBIANs:
cmake . -DWITH_READLINE=1 -DDEB=xenial -DPLUGIN_CONNECT=NO -DWITH_WSREP=OFF -DINSTALL_LAYOUT=DEB -DCMAKE_INSTALL_PREFIX=/usr/local/mariadb/columnstore/mysql/  -DCPACK_DEB_EXCLUDE_FROM_AUTO_FILELIST_ADDITION=/usr/local
make -jN # N is the number of concurrent build processes and should likely be the number of cores available
sudo make install
make package
git clone https://github.com/mariadb-corporation/mariadb-columnstore-engine.git
cd mariadb-columnstore-engine
git checkout develop
run cmake 
For RPMs"
cmake . -DRPM=centos6
For DEBIANs:
cmake . -DDEB=xenial
make -jN # same as above with respect to concurrent processes
sudo make install
make package

With the engine checked out in a separate location the following values need to be set by cmake command.

SERVER_BUILD_INCLUDE_DIR=Path to the server build include directory.
SERVER_SOURCE_ROOT_DIR=Path the directory the server source checked out from github.

Examples

Engine not located inside server:

git clone https://github.com/mariadb-corporation/mariadb-columnstore-server.git
git clone https://github.com/mariadb-corporation/mariadb-columnstore-engine.git
cd mariadb-columnstore-server
cmake . -DCMAKE_INSTALL_PREFIX=/usr/local/mariadb/columnstore/mysql
make -jN # N is the number of concurrent build processes and should likely be the number of cores available
sudo make install
cd ../mariadb-columnstore-engine
cmake . -DSERVER_BUILD_INCLUDE_DIR=../mariadb-columnstore-server/include -DSERVER_SOURCE_ROOT_DIR=../mariadb-columnstore-server
make -jN # same as above with respect to concurrent processes
sudo make install

Build out-of-source:

git clone https://github.com/mariadb-corporation/mariadb-columnstore-server.git
git clone https://github.com/mariadb-corporation/mariadb-columnstore-engine.git
mkdir buildServer
cd buildServer
cmake ../mariadb-columnstore-server -DCMAKE_INSTALL_PREFIX=/usr/local/mariadb/columnstore/mysql
make -jN # N is the number of concurrent build processes and should likely be the number of cores available
sudo make install
cd ..
mkdir buildEngine
cd buildEngine
cmake ../mariadb-columnstore-engine -DSERVER_BUILD_INCLUDE_DIR=../buildServer/include -DSERVER_SOURCE_ROOT_DIR=../mariadb-columnstore-server
make -jN # same as above with respect to concurrent processes
sudo make install

To build a debug version

  • Add -DCMAKE_BUILD_TYPE=debug -DPLUGIN_TOKUDB=NO to each of the cmake commands in the build process
  • Do not mix release and debug versions of server and engine

To develop a new branch/feature/pull request

  • Fork the server repo from github mariadb-corporation/mariadb-columnstore-server
  • Fork the engine report from github mariadb-corporation/mariadb-columnstore-engine
  • git checkout develop #branch in server
  • git submodule update --init
  • git branch new-branch-name (this can be in engine or server code)
  • git checkout new-branch-name
  • Edit source files
  • git commit -m 'meaningful checkin comment'
  • git push -u origin new-branch-name
  • Issue pull request for merge from new-branch-name into develop
  • MariaDB ColumnStore team will evaluate the changes and may request further development or changes before merge

Run dependencies

For CentOS

For CentOS 6 follow the install procedure for boost from the build Dependecy section above, with CentOS 7 you can just do:

yum install boost

In addition these packages need to be install:

yum install expect perl perl-DBI openssl zlib file sudo libaio rsync snappy net-tools nmap

For Ubuntu 16/18

These packages need to be installed:

apt-get install expect perl openssl file sudo libdbi-perl libboost-all-dev libreadline-dev rsync libsnappy1v5 net-tools nmap

For Debian 8

These packages need to be installed:

apt-get install expect perl openssl file sudo libdbi-perl libboost-all-dev libreadline-dev rsync libsnappy1 net-tools nmap

For Debian 9

These packages need to be installed:

apt-get install expect perl openssl file sudo libdbi-perl libboost-all-dev libreadline-dev rsync  net-tools libsnappy1v5 libreadline5 nmap

For SUSE 12

These packages need to be installed:

SUSEConnect -p sle-sdk/12.2/x86_64 zypper install boost-devel

zypper install expect perl perl-DBI openssl file sudo libaio1 rsync net-tools libsnappy1 nmap

MariaDB ColumnStore utilizes the System Logging for logging purposes

So you will want to make sure that one of these system logging packages is installed:

syslog, rsyslog, or syslog-ng

Configure and Starting of MariaDB ColumnStore

Follow the binary package install instructions in the ColumnStore Getting Starter Guide:

https://mariadb.com/kb/en/mariadb/columnstore-getting-started/

Commands to run as root user:

cd /usr/local/mariadb/columnstore/bin/
./post-install
./postConfigure
You can’t perform that action at this time.