This repository has been archived by the owner. It is now read-only.
Quark: An Embedded DSL for Quasar Analytics
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
core/src/main/scala/quasar/quark
project
.gitignore
LICENSE
README.md
build.sbt
sbt
version.sbt

README.md

Quark Analytics DSL for Quasar

Quark is an embedded DSL for the Quasar Analytics compiler.

With a syntax reminiscent of Spark, Quark allows developers to easily create complex workflows on semi-structured data that can be compiled and deployed to any target infrastructure supported by the Quasar Analytics compiler.

In contrast to Spark, Quark is specialized for semi-structured data, rather just structured data. In addition, Quark does not serialize any code and has a purely-functional, declarative interface.

On the flip side, Quark does not permit arbitrary user-defined code, but rather carefully limits the space of operations to those supported by the Quasar Analytics compiler.

Build

Usage

import quasar.quark._

object Example {
  def main(args: Array[String]): Unit = {
    val dataset = Dataset.load("/prod/profiles")

    val averageAge = dataset.groupBy(_.country[Str]).map(_.age[Int]).reduceBy(_.average)

    val stream = averageAge.evalTo("/prod/profiles_2")
  }
}