Skip to content
A compiler for a SQL dialect and mathematical meta-system with support for heterogeneous structured data
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
api/src 117.0.0: Delete Tables#liveData Feb 13, 2019
common/src
connector/src Replace ParseInstructions with ScalarStages Feb 8, 2019
core/src Add more tests for inferred projection names Feb 6, 2019
datagen/src Fix errors from overzealous commit. Oct 26, 2018
ejson/src
foundation/src
frontend/src
impl/src
project
qscript/src Join multiple tails when computing Map provenance Feb 24, 2019
qsu/src
run/src Adapt Quasar to work with unchanged schemas for aggregated resources Mar 7, 2019
scripts
sql/src
sst/src Compress SSTs during construction Nov 20, 2018
.appveyor.yml Don't test exclusive in appveyor Dec 12, 2018
.gitignore
.jvmopts Added string deduplication tuning Jan 29, 2019
.mailmap
.travis.yml Use transferCommonResources Dec 17, 2018
CLA.md Change the project name to Quasar. Sep 11, 2015
LICENSE change to Apache 2 Jun 30, 2015
README.md Rewrote the readme Feb 8, 2019
build.sbt
fs2-gzip-version Prep fs2-gzip version for autobump Nov 15, 2018
qdata-version Bump qdata version to 8.1.3 Feb 20, 2019
sbt
sbt.bat Increase reserved code cache size to 512m. Nov 17, 2017
tectonic-version Bump tectonic version to 7.2.3 Feb 20, 2019
version.sbt

README.md

Quasar Travis Discord

Quasar is a purely-functional compiler and optimizing planner for queries expressed in terms of the Multidimensional Relational Algebra (MRA). Quasar has support for arbitrary backends, both heavyweight (full evaluation engines) and lightweight (simple reads with optional pushdown of structural operations and columnar predicates), including full classpath isolation for lightweight backends.

It's important to note that Quasar is not, in and of itself, a runnable application. It is a library which is used by the broader SlamData product, much of which is closed-source. Contributions are very much welcome, as is feedback, questions, and general conversation. Join the Discord!

Building and Testing

Quasar builds with SBT:

$ ./sbt
> test:compile
> test

If running on Windows, you may use the SBT batch file instead of the shell script.

Code Organization

Probably the most interesting part of the codebase is the optimizing query planner, which is implemented in the qsu submodule, based on data structures defined in qscript. I recommend starting by looking at the LPtoQS class, which defines a kleisli composition that clearly lays out all of the phases of the compiler. The core data structure used by the compiler is QSUGraph, which is a purely functional representation of a directed acyclic graph, which in turn represents data flow in a query.

The formulation of the query plan itself is a fixed-point data structure dictated by several pattern functors composed via coproducts. The primary such pattern functor is QScriptCore. You would generally deconstruct and interpret this query plan using general folds provided by matryoshka.

Query operations which are pushed down to the underlying data source are represented by ScalarStage, and carried via InterpretedRead. Data sources are always free to only implement a subset of the pushdown functionality.

The codebase makes extremely heavy use of Scalaz and Cats throughout (using shims to solve the impedance between them), and many high-level operations (such as datasets) are represented as fs2 streams.

Legal

Copyright © 2014 - 2019 SlamData Inc.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

You can’t perform that action at this time.