Skip to content
Apache Pinot (Incubating) - A realtime distributed OLAP datastore
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config Add Pinot code style (#3705) Jan 18, 2019
contrib/pinot-druid-benchmark Auto-reformat all java source files (#3739) Jan 29, 2019
docker Adding script to build and publish docker image (#4200) Jun 13, 2019
docs Adding ORC configuration in docs (#4045) Jun 14, 2019
licenses-binary Prepare licenses for 0.2 (#4290) Jun 13, 2019
licenses Update LICENSE and NOTICE files (#3722) Jan 31, 2019
pinot-api Add support for passing headers in pinot client (#4243) Jun 5, 2019
pinot-azure-filesystem Move pinot-filesystem into pinot-common (#4155) Apr 24, 2019
pinot-broker Cleaning up getTableName() for segment metadata (#4289) Jun 11, 2019
pinot-common Add interface and implementations for the new segment assignment (#4269) Jun 14, 2019
pinot-connectors Package name changes for classes which moved from kafka->stream (#4298) Jun 10, 2019
pinot-controller Add interface and implementations for the new segment assignment (#4269) Jun 14, 2019
pinot-core PQL -> SQL enhancement - phase 1 - new Pinot Query Struct (#4216) Jun 12, 2019
pinot-distribution Remove filesystem module from pinot-distribution (#4165) Apr 25, 2019
pinot-hadoop-filesystem Move pinot-filesystem into pinot-common (#4155) Apr 24, 2019
pinot-hadoop Support legacy time type in SegmentValidationAndRetentionConfig (#4081) Apr 5, 2019
pinot-integration-tests Added tests for RealtimeSegmentValidationManager (#4306) Jun 12, 2019
pinot-minion Cleaning up getTableName() for segment metadata (#4289) Jun 11, 2019
pinot-orc Move Parquet related classes to the new package (#4073) Apr 4, 2019
pinot-parquet Move Parquet related classes to the new package (#4073) Apr 4, 2019
pinot-perf Cleaning up getTableName() for segment metadata (#4289) Jun 11, 2019
pinot-server Cleaning up getTableName() for segment zk metadata (#4288) Jun 10, 2019
pinot-tools Cleaning up getTableName() for segment metadata (#4289) Jun 11, 2019
pinot-transport Migration to log4j2 (#4139) May 7, 2019
thirdeye [TE] Use multimap in metric urn as dimension (#4313) Jun 14, 2019
.codecov.yml Update license-maven-plugin setting to correctly exclude files (#3691) Jan 15, 2019
.codecov_bash Fix codecov ignore coverage for certain files (#1054) Feb 14, 2017
.gitignore Change ControllerTest default prot to 18998 (#4259) May 31, 2019
.travis.yml Revert travis-wait config (#4084) Apr 5, 2019
.travis_install.sh Update license header (#3664) Jan 8, 2019
.travis_test.sh Update license header (#3664) Jan 8, 2019
DISCLAIMER Update pom files for preparing Apache release (#3772) Feb 1, 2019
HEADER Cleaning up the license-maven-plugin (#3706) Jan 17, 2019
LICENSE Update the display for sidebar navigation to include section names (#… Mar 6, 2019
LICENSE-binary Prepare licenses for 0.2 (#4290) Jun 13, 2019
NOTICE Update pom files for preparing Apache release (#3772) Feb 1, 2019
NOTICE-binary Prepare licenses for 0.2 (#4290) Jun 13, 2019
README.md Fixed #3960 Update README.md to fix Quick Demo link broken. (#3962) Mar 13, 2019
pom.xml Upgrade to helix-0.9 (#4324) Jun 15, 2019
rstheaderdefinition.xml Add header definition for rst files (#3860) Feb 21, 2019

README.md

Apache Pinot (incubating)

Build Status codecov.io Join the chat at https://gitter.im/linkedin/pinot license

Apache Pinot is a realtime distributed OLAP datastore, which is used to deliver scalable real time analytics with low latency. It can ingest data from offline data sources (such as Hadoop and flat files) as well as online sources (such as Kafka). Pinot is designed to scale horizontally.

These presentations on Pinot give an overview of Pinot:

Looking for the ThirdEye anomaly detection and root-cause analysis platform? Check out the Pinot/ThirdEye project

Key Features

  • A column-oriented database with various compression schemes such as Run Length, Fixed Bit Length
  • Pluggable indexing technologies - Sorted Index, Bitmap Index, Inverted Index
  • Ability to optimize query/execution plan based on query and segment metadata
  • Near real time ingestion from Kafka and batch ingestion from Hadoop
  • SQL like language that supports selection, aggregation, filtering, group by, order by, distinct queries on fact data
  • Support for multivalued fields
  • Horizontally scalable and fault tolerant

Because of the design choices we made to achieve these goals, there are certain limitations present in Pinot:

  • Pinot is not a replacement for database i.e it cannot be used as source of truth store, cannot mutate data
  • Not a replacement for search engine i.e full text search, relevance not supported
  • Query cannot span across multiple tables

Pinot works very well for querying time series data with lots of Dimensions and Metrics. Example - Query (profile views, ad campaign performance, etc.) in an analytical fashion (who viewed this profile in the last weeks, how many ads were clicked per campaign).

Instructions to build Pinot

More detailed instructions can be found at Quick Demo section in the documentation.

# Clone a repo
$ git clone https://github.com/apache/incubator-pinot.git
$ cd incubator-pinot

# Build Pinot
$ mvn clean install -DskipTests -Pbin-dist

# Run Quck Demo
$ cd pinot-distribution/target/apache-pinot-incubating-<version>-SNAPSHOT-bin
$ bin/quick-start-offline.sh

Getting Involved

Documentation

Check out Pinot documentation for a complete description of Pinot's features.

License

Apache Pinot is under Apache License, Version 2.0

You can’t perform that action at this time.