Skip to content
The official home of the Presto distributed SQL query engine for big data
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github
.mvn Remove maven-buildtime-extension Oct 5, 2017
presto-accumulo Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-array Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-atop Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-base-jdbc Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-benchmark-driver Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-benchmark Change lookupFunction to take String instead of QualifiedName as func… Apr 17, 2019
presto-benchto-benchmarks
presto-blackhole Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-cassandra Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-cli
presto-client Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-docs Add a warning to Release 0.218 about FULL OUTER JOIN regression Apr 10, 2019
presto-elasticsearch Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-example-http Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-geospatial-toolkit Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-geospatial Change lookupFunction to take String instead of QualifiedName as func… Apr 17, 2019
presto-hive-hadoop2 Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-hive Change min_by, max_by aggregations intermediate type Apr 23, 2019
presto-jdbc Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-jmx Rename PrestoNode to InternalNode Apr 18, 2019
presto-kafka Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-kudu
presto-local-file Prefer Files.newInputStream/Files.newOutputStream Apr 10, 2019
presto-main Change qdigest aggregations intermediate type Apr 23, 2019
presto-matching Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-memory-context Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-memory Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-ml Change lookupFunction to take String instead of QualifiedName as func… Apr 17, 2019
presto-mongodb Add support for readPreferenceTags in mongodb Apr 18, 2019
presto-mysql
presto-orc Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-parquet Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-parser Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-password-authenticators Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-plugin-toolkit Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-postgresql Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-product-tests Move presto-server to assembly plugin Apr 24, 2019
presto-proxy Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-raptor
presto-rcfile Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-record-decoder Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-redis
presto-redshift Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-resource-group-managers Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-server-rpm Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-server
presto-session-property-managers Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-spi Change min_by, max_by aggregations intermediate type Apr 23, 2019
presto-sqlserver Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-teradata-functions
presto-testing-server-launcher Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-tests Change qdigest aggregations intermediate type Apr 23, 2019
presto-thrift-connector-api Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-thrift-connector Check partition handle argument in {Raptor, Thrift}SplitSource Apr 13, 2019
presto-thrift-testing-server Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-tpcds Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-tpch Prepare for the next development iteration - 0.220-SNAPSHOT Apr 8, 2019
presto-verifier Support JdbcErrorCode to Verifier exception classifier Apr 23, 2019
src Prefer String.toLowerCase(ENGLISH) over String.toLowerCase() Jul 9, 2018
.gitattributes Migrate index page for WebUI to new javascript workflow Oct 18, 2018
.gitignore Migrate index page for WebUI to new javascript workflow Oct 18, 2018
.travis.yml Run exchange materialization tests in a separate build Apr 17, 2019
CODE_OF_CONDUCT.md OSS Automated Fix: Addition of Code of Conduct Mar 28, 2019
CONTRIBUTING.md
LICENSE Add LICENSE file Oct 1, 2013
README.md Update references to Presto website Jan 23, 2019
mvnw Update Maven wrapper to 3.5.0 Apr 12, 2017
pom.xml Use nexus-staging-maven-plugin Apr 24, 2019

README.md

Presto Build Status

Presto is a distributed SQL query engine for big data.

See the User Manual for deployment instructions and end user documentation.

Requirements

  • Mac OS X or Linux
  • Java 8 Update 151 or higher (8u151+), 64-bit. Both Oracle JDK and OpenJDK are supported.
  • Maven 3.3.9+ (for building)
  • Python 2.4+ (for running with the launcher script)

Building Presto

Presto is a standard Maven project. Simply run the following command from the project root directory:

./mvnw clean install

On the first build, Maven will download all the dependencies from the internet and cache them in the local repository (~/.m2/repository), which can take a considerable amount of time. Subsequent builds will be faster.

Presto has a comprehensive set of unit tests that can take several minutes to run. You can disable the tests when building:

./mvnw clean install -DskipTests

Running Presto in your IDE

Overview

After building Presto for the first time, you can load the project into your IDE and run the server. We recommend using IntelliJ IDEA. Because Presto is a standard Maven project, you can import it into your IDE using the root pom.xml file. In IntelliJ, choose Open Project from the Quick Start box or choose Open from the File menu and select the root pom.xml file.

After opening the project in IntelliJ, double check that the Java SDK is properly configured for the project:

  • Open the File menu and select Project Structure
  • In the SDKs section, ensure that a 1.8 JDK is selected (create one if none exist)
  • In the Project section, ensure the Project language level is set to 8.0 as Presto makes use of several Java 8 language features

Presto comes with sample configuration that should work out-of-the-box for development. Use the following options to create a run configuration:

  • Main Class: com.facebook.presto.server.PrestoServer
  • VM Options: -ea -XX:+UseG1GC -XX:G1HeapRegionSize=32M -XX:+UseGCOverheadLimit -XX:+ExplicitGCInvokesConcurrent -Xmx2G -Dconfig=etc/config.properties -Dlog.levels-file=etc/log.properties
  • Working directory: $MODULE_DIR$
  • Use classpath of module: presto-main

The working directory should be the presto-main subdirectory. In IntelliJ, using $MODULE_DIR$ accomplishes this automatically.

Additionally, the Hive plugin must be configured with location of your Hive metastore Thrift service. Add the following to the list of VM options, replacing localhost:9083 with the correct host and port (or use the below value if you do not have a Hive metastore):

-Dhive.metastore.uri=thrift://localhost:9083

Using SOCKS for Hive or HDFS

If your Hive metastore or HDFS cluster is not directly accessible to your local machine, you can use SSH port forwarding to access it. Setup a dynamic SOCKS proxy with SSH listening on local port 1080:

ssh -v -N -D 1080 server

Then add the following to the list of VM options:

-Dhive.metastore.thrift.client.socks-proxy=localhost:1080

Running the CLI

Start the CLI to connect to the server and run SQL queries:

presto-cli/target/presto-cli-*-executable.jar

Run a query to see the nodes in the cluster:

SELECT * FROM system.runtime.nodes;

In the sample configuration, the Hive connector is mounted in the hive catalog, so you can run the following queries to show the tables in the Hive database default:

SHOW TABLES FROM hive.default;

Code Style

We recommend you use IntelliJ as your IDE. The code style template for the project can be found in the codestyle repository along with our general programming and Java guidelines. In addition to those you should also adhere to the following:

  • Alphabetize sections in the documentation source files (both in table of contents files and other regular documentation files). In general, alphabetize methods/variables/sections if such ordering already exists in the surrounding code.
  • When appropriate, use the Java 8 stream API. However, note that the stream implementation does not perform well so avoid using it in inner loops or otherwise performance sensitive sections.
  • Categorize errors when throwing exceptions. For example, PrestoException takes an error code as an argument, PrestoException(HIVE_TOO_MANY_OPEN_PARTITIONS). This categorization lets you generate reports so you can monitor the frequency of various failures.
  • Ensure that all files have the appropriate license header; you can generate the license by running mvn license:format.
  • Consider using String formatting (printf style formatting using the Java Formatter class): format("Session property %s is invalid: %s", name, value) (note that format() should always be statically imported). Sometimes, if you only need to append something, consider using the + operator.
  • Avoid using the ternary operator except for trivial expressions.
  • Use an assertion from Airlift's Assertions class if there is one that covers your case rather than writing the assertion by hand. Over time we may move over to more fluent assertions like AssertJ.
  • When writing a Git commit message, follow these guidelines.

Building the Web UI

The Presto Web UI is composed of several React components and is written in JSX and ES6. This source code is compiled and packaged into browser-compatible Javascript, which is then checked in to the Presto source code (in the dist folder). You must have Node.js and Yarn installed to execute these commands. To update this folder after making changes, simply run:

yarn --cwd presto-main/src/main/resources/webapp/src install

If no Javascript dependencies have changed (i.e., no changes to package.json), it is faster to run:

yarn --cwd presto-main/src/main/resources/webapp/src run package

To simplify iteration, you can also run in watch mode, which automatically re-compiles when changes to source files are detected:

yarn --cwd presto-main/src/main/resources/webapp/src run watch

To iterate quickly, simply re-build the project in IntelliJ after packaging is complete. Project resources will be hot-reloaded and changes are reflected on browser refresh.

You can’t perform that action at this time.