Skip to content
DuckDB is an embeddable SQL OLAP Database Management System
C++ C TSQL JavaScript Python CMake Other
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
benchmark Adding the IMDB-based Join Order Benchmark [JOB] from Leis et al. (#283) Sep 9, 2019
examples updated dplyr example Oct 6, 2019
logo first part of prepared statements and transactions for python api Oct 13, 2019
src Correlated window function test cases and only fetch types of window … Oct 17, 2019
test Test case with binding failures inside a window function Oct 17, 2019
third_party Adding the IMDB-based Join Order Benchmark [JOB] from Leis et al. (#283) Sep 9, 2019
tools cleanup with exceptions , test with pandas df Oct 16, 2019
.clang-format Rework SelectStatement -> queries are now contained in QueryNode, whi… Dec 4, 2018
.gitignore Adding the IMDB-based Join Order Benchmark [JOB] from Leis et al. (#283) Sep 9, 2019
.travis.yml Fixing Travis again (#294) Sep 26, 2019
CMakeLists.txt Format Aug 6, 2019
CODE_OF_CONDUCT.md Update CODE_OF_CONDUCT.md Mar 26, 2019
CONTRIBUTING.md Pushing the number of prepared statements into the C/C++ APIs Jun 30, 2019
Doxyfile Rework build directory somewhat, now build to build/debug and Aug 3, 2018
LICENSE Fixed bug in cross product rewriting that resulted in conditions bein… Dec 4, 2018
Makefile Adding the IMDB-based Join Order Benchmark [JOB] from Leis et al. (#283) Sep 9, 2019
README.md Update README.md Sep 26, 2019
format.py Adding the IMDB-based Join Order Benchmark [JOB] from Leis et al. (#283) Sep 9, 2019
generate_querygraph.py Add duckdb_query_graph module to Python package with support for gene… Jul 20, 2019
runsqlsmith.py Add script for running SQL smith and automatically extracting crashin… Oct 18, 2018

README.md

DuckDB, the SQLite for Analytics

Travis CodeFactor Coverage Status


Requirements

DuckDB requires CMake to be installed and a C++11 compliant compiler. GCC 4.9 and newer, Clang 3.9 and newer and VisualStudio 2017 are tested on each revision.

Compiling

Run make in the root directory to compile the sources. For development, use make debug to build a non-optimized debug versoin. You may run make unit and make allunit to verify that your version works properly after making changes.

Usage

A command line utility based on sqlite3 can be found in either build/release/tools/shell/shell (release, the default) or build/debug/tools/shell/shell (debug).

Embedding

As DuckDB is an embedded database, there is no database server to launch or client to connect to a running server. However, the database server can be embedded directly into an application using the C or C++ bindings. The main build process creates the shared library build/release/src/libduckdb.[so|dylib|dll] that can be linked against. A static library is built as well.

For examples on how to embed DuckDB into your application, see the examples folder.

Benchchmarks

After compiling, benchmarks can be executed from the root directory by executing ./build/release/benchmark/benchmark_runner.

Standing on the Shoulders of Giants

DuckDB is implemented in C++ 11, should compile with GCC and clang, uses CMake to build and Catch2 for testing. DuckDB uses some components from various Open-Source databases and draws inspiration from scientific publications. Here is an overview:

  • Parser: We use the PostgreSQL parser that was repackaged as a stand-alone library. The translation to our own parse tree is inspired by Peloton.
  • Shell: We have adapted the SQLite shell to work with DuckDB.
  • Tests: We use the SQL Logic Tests from SQLite to test DuckDB.
  • Query fuzzing: We use SQLsmith to generate random queries for additional testing.
  • Date Math: We use the date math component from MonetDB.
  • SQL Window Functions: DuckDB's window functions implementation uses Segment Tree Aggregation as described in the paper "Efficient Processing of Window Functions in Analytical SQL Queries" by Viktor Leis, Kan Kundhikanjana, Alfons Kemper and Thomas Neumann.
  • Execution engine: The vectorized execution engine is inspired by the paper "MonetDB/X100: Hyper-Pipelining Query Execution" by Peter Boncz, Marcin Zukowski and Niels Nes.
  • Optimizer: DuckDB's optimizer draws inspiration from the papers "Dynamic programming strikes back" by Guido Moerkotte and Thomas Neumman as well as "Unnesting Arbitrary Queries" by Thomas Neumann and Alfons Kemper.
  • Concurrency control: Our MVCC implementation is inspired by the paper "Fast Serializable Multi-Version Concurrency Control for Main-Memory Database Systems" by Thomas Neumann, Tobias Mühlbauer and Alfons Kemper.
  • Regular Expression: DuckDB uses Google's RE2 regular expression engine.

Other pages

You can’t perform that action at this time.