Skip to content
Quickstep project
C++ CMake C Python Shell Objective-C Other
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
benchmarks/tpch Added instructions to run TPC-H. (#6) Aug 25, 2019
build QUICKSTEP-40: Fix Copyright notice to confirm to Apache. Aug 7, 2016
catalog Fix and rearrange all CMakeLists.txt so that they are ready to be pro… Apr 27, 2018
cli Fix a bug in the optimization rule that adds FilterJoins. (#5) Jun 30, 2019
cmake Refactored out proto GenerateCPP function Apr 6, 2017
compression Fix and rearrange all CMakeLists.txt so that they are ready to be pro… Apr 27, 2018
expressions Add virtual destructors where they are needed to remove compilations … Oct 11, 2018
parser Minor bug fixes and refactors. May 9, 2018
query_execution Fix the inclusion guard of ForemanSingleNode.hpp file Jun 22, 2018
query_optimizer Fix a bug in the optimization rule that adds FilterJoins. (#5) Jun 30, 2019
relational_operators Fix a bug of filter side in HashJoinOperator Jun 5, 2018
release Added script for release license audit Mar 17, 2017
storage Fix a bug of filter side in HashJoinOperator Jun 5, 2018
third_party Update gperftools version from 2.5 to 2.7 (#2) May 10, 2019
threading Fix and rearrange all CMakeLists.txt so that they are ready to be pro… Apr 27, 2018
transaction Update Transaction.hpp Aug 26, 2019
types Fix and rearrange all CMakeLists.txt so that they are ready to be pro… Apr 27, 2018
utility Fix a bug in the optimization rule that adds FilterJoins. (#5) Jun 30, 2019
yarn Fix and rearrange all CMakeLists.txt so that they are ready to be pro… Apr 27, 2018
.gitattributes Init release scripts Mar 1, 2017
.gitignore Make the third party directory leaner. Jan 28, 2017
.gitmodules Make the third party directory leaner. Jan 28, 2017
.travis.yml In .travis.yml, update Ubuntu distribution to trusty and clang versio… Jun 17, 2019
BUILDING.md Refactor building.md Feb 9, 2017
CMakeLists.txt Fix glog build issue for Mac OS X 10.14. Oct 23, 2018
CREDITS.md QUICKSTEP-40: Fix Copyright notice to confirm to Apache. Aug 7, 2016
DEV_README.md Added header to scripts and yaml files, updated NOTICE Mar 8, 2017
DISCLAIMER QUICKSTEP-1: Add incubation disclaimer to README.md. Jun 8, 2016
Doxyfile Initial commit of Quickstep Jan 16, 2016
GENERATOR_FUNCTIONS.md QUICKSTEP-40: Fix Copyright notice to confirm to Apache. Aug 7, 2016
HISTORY.md Added header to scripts and yaml files, updated NOTICE Mar 8, 2017
LICENSE Update license file to summarize 3rd party Mar 17, 2017
NOTICE Update license file to summarize 3rd party Mar 17, 2017
README.md Update README.md Aug 24, 2019
WORKING_WITH_AN_IDE.md IDE Documentation fixes Jan 11, 2018
autofix_cmakelists.py Fix a bug of filter side in HashJoinOperator Jun 5, 2018
cyclic_dependency.py Fixed the root path check in the cyclic_dependency.py. Oct 3, 2017
empty_src.cpp QUICKSTEP-40: Fix Copyright notice to confirm to Apache. Aug 7, 2016
lint_everything.py Upgrade cpplint Dec 21, 2017
validate_cmakelists.py Fixed the root path check in the validate_cmakelists script. Sep 29, 2017

README.md

Quickstep

Travis Widget

What is Quickstep?

Quickstep is high-performance database engine designed to exploit the full potential of hardware that is packed in modern computing boxes (servers and laptops). This version targets single-node in-memory environments. If your data spills overs the memory limit Quickstep will still work, so you don't have to obsessively worry about the in-memory part. Also, if your working set fits in memory then Quickstep will transparently and automatically figure that out, and cache that hot set to deliver in-memory performance.

Why Quickstep?

Did you know that the hardware that you have in your laptop was spread across a small cluster just a decade ago? (PS: Hopefully you are not using a very old laptop!) If you look at a high-end server box, then that packs compute and storage power that was a full rack about 5 years ago! And, the way hardware technology is going, that box is going to become even more powerful in the future. In fact, it is likely that the computing power in each box is going to grow faster than other hardware components (e.g. networking) in data centers. So, if you care about performance and/or total operating costs, paying attention to single box performance is likely to be super important in the long run.

In other words there is a small data center in an individual compute boxes today! Quickstep aims to allow you to fully exploit the potential of that data center that is hidden in each individual box today. We call this the scaling-in approach, and it complements a scaling-out approach. But without scaling-in, you are overpaying (by a lot!) when you run your data service.

What are the key ingredients?

Modern computing boxes contain a large number of computing cores and large main memory configuration. Quickstep allows you to fully exploit these hardware resources using novel data processing, data storage, and query processing methods that include:

  1. A unique decoupling of data-flow from control-flow for query execution that allows for unlimited intra and inter-query parallelism. Thus, using all the processing core effectively.

  2. A template meta-programming framework that provides fast vectorized query execution. Thus, using each processor cycle very efficiently.

  3. A hybrid data storage architecture that includes columnar and row-store. Yes, this may surprise some of you, but sometimes a row-store beats a column-store!

And, it is open source!

Giving it a spin

  1. Checkout the code: git clone https://github.com/UWQuickstep/quickstep.git
  2. Then, go to the code directory: cd quickstep
  3. Initialize the dependencies: git submodule init
  4. Checkout the dependencies: git submodule update
  5. Download additional third-party dependencies and apply patches:
    cd third_party && ./download_and_patch_prerequisites.sh && cd ../
  6. Go into the build directory: cd build
  7. Create the Makefile: cmake -D CMAKE_BUILD_TYPE=Release ..
  8. Build: make -j4. Note you may replace the 4 with the number of cores on your machine.
  9. Start quickstep: ./quickstep_cli_shell --initialize_db=true. You can now fire SQL queries. To quit, you can type in quit; Your data is stored in the directory qsstor. Note the next time you start Quickstep, you can omit the --initialize_db flag (as the database has already been initialized), and simply start Quickstep as: ./quickstep_cli_shell. There are also a number of optional flags that you can specify, and to see the full list, you can type in: ./quickstep_cli_shell --help
  10. Next let us load some data and fire some queries. A few points to note: The SQL surface of Quickstep is small (it will grow over time). The traditional SQL CREATE TABLE and SELECT statements work. The data types that are supported include INTEGER, FLOAT, DOUBLE, VARCHAR, CHAR, DATE, and DATETIME. Quickstep also does not have support for NULLS or keys (yet). Let us create two tables by typing into the Quickstep shell (which you opened in the step above), the following SQL command:
CREATE TABLE Weather (cid INTEGER, recordDate DATE, highTemperature FLOAT, lowTemperature FLOAT);

and then,

CREATE TABLE City (cid Integer, name VARCHAR(80), state CHAR(2));
  1. Next, let us insert some tuples in these two tables.

    INSERT INTO City VALUES (1, 'Madison', 'WI');
    INSERT INTO City VALUES (2, 'Palo Alto', 'CA');
    INSERT INTO Weather VALUES (1, '2015-11-1', 50, 30);
    INSERT INTO Weather VALUES (1, '2015-11-2', 51, 32);
    INSERT INTO Weather VALUES (2, '2015-11-1', 60, 50);
    
  2. We can now issue SQL queries such as: a. Find all weather records for California:

SELECT * FROM WEATHER W, City C WHERE C.cid = W.cid AND C.state = 'CA';

b. Find the min and max temperature for each city, printing the cid:

SELECT cid, MIN(lowTemperature), MAX(highTemperature) FROM Weather GROUP BY cid;

c. Find the min and max temperature for each city using a nested query, and printing thie city name:

SELECT * FROM City C, (SELECT cid, MIN(lowTemperature), MAX(highTemperature) FROM Weather GROUP BY cid) AS T WHERE C.cid = T.cid;
  1. Quickstep also supports a COPY TABLE command. If you want to try that, then from a separate shell file type in the following:

    echo "3|2015-11-3|49|29" > /tmp/tmp.tbl
    echo "3|2015-11-4|48|28" >> /tmp/tmp.tbl
    echo "3|2015-11-5|47|27" >> /tmp/tmp.tbl
    

    Then, load this new data by typing the following SQL in the Quickstep shell:

    COPY Weather FROM '/tmp/tmp.tbl' WITH (DELIMITER '|');
    

    Now, you have loaded three more tuples into the Weather table, and you can fire the SQL queries above again against this modified database.

    Remember, to quit Quickstep, you can type in quit; into the Quickstep shell.

Additional pointers

  1. For other build options, see the more comprehensive build guide.
  2. To get started as a developer, you should start with the code organization guide.

Licensing

Quickstep is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

You can’t perform that action at this time.