diqube is a fast, distributed, in-memory column-store which enables you to analyze large amounts of read-only data easily
Java TypeScript HTML Thrift JavaScript ANTLR Other
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
build
diqube-buildinfo
diqube-cache
diqube-cluster
diqube-config
diqube-connection
diqube-consensus-test
diqube-consensus
diqube-context
diqube-data-serialize
diqube-data
diqube-diql
diqube-execution
diqube-executionenv
diqube-file
diqube-flatten
diqube-function
diqube-hadoop
diqube-im
diqube-itest
diqube-listeners
diqube-loader
diqube-metadata
diqube-name
diqube-optimize
diqube-permission
diqube-plan
diqube-queries
diqube-remote
diqube-server
diqube-test-util
diqube-threads
diqube-thrift
diqube-ticket
diqube-tool
diqube-ui
diqube-util
docs
third-party-licenses
.gitignore
LICENSE
README.md
THIRD-PARTY.txt
generate-third-party.sh
pom.xml

README.md

diqube - Distributed Query Base

Build Status Project Statistics

diqube is a fast, distributed, in-memory column-store which enables you to analyze large amounts of read-only data easily.

UI YouTube video

Getting started

To get started with diqube, you either need to

  • prepare a Docker image which will build & start a specific version of diqube for you or
  • install a build environment on your local machine and build and start diqube from there.

In order to prepare some data to be loaded into diqube server, you might want to read the chapter "Loading data" of the Operating a cluster guide.

Using Docker to get started

To create a Docker image, follow the instructions at diqube-docker/build-and-run.

Building manually from source

Make sure you have the prerequisites installed:

In addition to that, you need to provide the location of the Thrift compiler executable to the maven build. Do this by setting the property "thrift.executable" in your settings.xml:

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"  
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <profiles>
    <profile>
      <id>local-thrift-093</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <properties>
        <thrift.executable>/path/to/your/thrift/compiler/thrift</thrift.executable>
      </properties>
    </profile>
  </profiles>
</settings>

After everything is prepared, execute the build using

$ mvn clean install -DskipTests

Starting diqube

Execute the following after building, note that the passwords and secrets need to be changed for production use, obviously:

$ cd diqube-server

$ echo "messageIntegritySecret=CHANGE_THIS_IN_PRODUCTION" > server.properties
$ openssl genrsa -des3 -out ticket.pem -passout pass:diqube 2048
$ echo "ticketRsaPrivateKeyPemFile=`pwd`/ticket.pem" >> server.properties
$ echo "ticketRsaPrivateKeyPassword=diqube" >> server.properties
$ echo "superuser=root" >> server.properties
$ echo "superuserPassword=diqube" >> server.properties

$ java -Ddiqube.properties=server.properties -jar target/diqube-server-1-SNAPSHOT.jar

# For use in diqube-ui (see below):
$ openssl rsa -in ticket.pem -pubout -out ticket.pub.pem

The server will then start and watch the 'data' directory inside diqube-server for any new data to be loaded (.control files). There is some sample data already provided. For more information see Operating a cluster.

The diqube server itself does not contain the diqube UI. The UI is available as standard web archive (.war file) which can be deployed using a context xml like tomcat-sample-context (for additional configuration keys see DiqubeServletConfig) to an Apache Tomcat 8.0.26+. You can find the .war in diqube-ui/target after building. Please note that only Apache Tomcat 8.0.26+ is supported, as it relies on Tomcat Bug 58232 being fixed which is the case starting with 8.0.26. The UI only supports current versions of Chrome.

Documentation

See the docs directory for some documentation.

License

diqube is distributed under the terms of the GNU Affero General Public License 3.