Skip to content

Commit

Permalink
fixing presto pinot doc for pinot catalog configs and add session pro…
Browse files Browse the repository at this point in the history 
…perty pinot.topn_large
  • Loading branch information
@xiangfu0 @highker
xiangfu0 authored and highker committed Feb 23, 2022
1 parent 59d092c commit afd4d2f
Showing 1 changed file with 50 additions and 49 deletions.
99 changes: 50 additions & 49 deletions presto-docs/src/main/sphinx/connector/pinot.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,61 +37,61 @@ Catalog Properties

The following catalog configuration properties are available:

======================================================== ============================================================
Property Name Description
======================================================== ============================================================
``pinot.controller-urls`` Pinot controller urls.
``pinot.controller-rest-service`` Alternative rest endpoint for Pinot controller requests.
``pinot.rest-proxy-url`` Pinot rest proxy url.
``pinot.allow-multiple-aggregations`` Push down a Pinot query with multiple aggregation functions, default is true.
``pinot.limit-large-for-segment`` Cap the number of rows returned when pushing down non-aggregation segment query, default is 2147483647.
``pinot.topn-large`` Cap the TOP/LIMIT value when pushing down broker query, default is 10000.
``pinot.thread-pool-size`` Parameter to init Pinot server query client, default is 30.
``pinot.min-connections-per-server`` Parameter to init Pinot server query client, default is 10.
``pinot.max-connections-per-server`` Parameter to init Pinot server query client, default is 30.
``pinot.max-backlog-per-server`` Parameter to init Pinot server query client, default is 30.
``pinot.idle-timeout`` Parameter to init Pinot server query client, default is 5 minutes.
``pinot.connection-timeout`` Connection Timeout to talk to Pinot servers, default is 1 minute.
``pinot.metadata-expiry`` Pinot metadata cache expiration time, default is 2 minutes.
``pinot.estimated-size-in-bytes-for-non-numeric-column`` Estimated byte size for non-numeric column, default is 20.
``pinot.service-header-param`` RPC service service header key, default is "RPC-Service".
``pinot.caller-header-param`` RPC service caller header key, default is "RPC-Caller".
``pinot.caller-header-value`` RPC service caller header value, default is "presto".
``pinot.forbid-broker-queries`` No broker request pushing down, default is false.
``pinot.forbid-segment-queries`` No segment query pushing down, fail the query if broker query pushing down is not possible, default is false.
``pinot.rest-proxy-service-for-query`` Use rest proxy endpoint for Pinot broker requests, default is false.
``pinot.use-date-trunc`` Use the new UDF dateTrunc in pinot that is more presto compatible, default is false.
``pinot.num-segments-per-split`` Number of segments of the same host per split, default is 1.
``pinot.ignore-empty-responses`` Ignore empty or missing pinot server responses, default is false.
``pinot.fetch-retry-count`` Retry count for retriable pinot data fetch calls, default is 2.
``pinot.non-aggregate-limit-for-broker-queries`` Max limit for non aggregate queries to the pinot broker, default is 25000.
``pinot.infer-date-type-in-schema`` Infer Pinot DAYS epoch column to Presto DATE type, default is true.
``pinot.infer-timestamp-type-in-schema`` Infer Pinot SECONDS epoch column to Presto TIMESTAMP type, default is true.
``pinot.mark-data-fetch-exceptions-as-retriable`` Retry Pinot request when failure, default is true.
``pinot.use-pinot-sql-for-broker-queries`` Use Pinot SQL syntax, otherwise PQL syntax, default is true.
``pinot.pushdown-topn-broker-queries`` Allow pushing down query pattern to broker: aggregation + groupBy + orderBy, default is false.
``pinot.use-streaming-for-segment-queries`` Use gRPC endpoint for pinot server queries, default is false.
``pinot.streaming-server-grpc-max-inbound-message-bytes`` Max inbound message bytes when init gRPC client, default is 128MB.
``pinot.use-proxy-for-broker-request`` Sending broker query to Pinot Proxy, default is false.
``pinot.use-proxy-grpc-endpoint`` Sending segment query to Pinot Proxy gRPC endpoint, default is false.
``pinot.proxy-grpc-host`` Pinot proxy gRPC host.
``pinot.proxy-grpc-port`` Pinot proxy gRPC port.
``pinot.use-https-for-controller`` Use https for controller requests, default is false.
``pinot.use-https-for-broker`` Use https for broker requests, default is false.
``pinot.use-https-for-proxy`` Use https for proxy requests, default is false.
``pinot.override-distinct-count-function`` Override 'distinctCount' function name, default is "distinctCount".
``pinot.extra-http-headers`` Extra headers when sending HTTP based pinot requests to Pinot controller/broker.
``pinot.extra-grpc-metadata`` Extra metadata when sending gRPC based pinot requests to Pinot broker/server/proxy.
======================================================== ============================================================
========================================================== =============================================================================================================
Property Name Description
========================================================== =============================================================================================================
``pinot.controller-urls`` Pinot controller urls.
``pinot.controller-rest-service`` Alternative rest endpoint for Pinot controller requests.
``pinot.rest-proxy-url`` Pinot rest proxy url.
``pinot.allow-multiple-aggregations`` Push down a Pinot query with multiple aggregation functions, default is true.
``pinot.limit-large-for-segment`` Cap the number of rows returned when pushing down non-aggregation segment query, default is 2147483647.
``pinot.topn-large`` Cap the TOP/LIMIT value when pushing down broker query, default is 10000.
``pinot.thread-pool-size`` Parameter to init Pinot server query client, default is 30.
``pinot.min-connections-per-server`` Parameter to init Pinot server query client, default is 10.
``pinot.max-connections-per-server`` Parameter to init Pinot server query client, default is 30.
``pinot.max-backlog-per-server`` Parameter to init Pinot server query client, default is 30.
``pinot.idle-timeout`` Parameter to init Pinot server query client, default is 5 minutes.
``pinot.connection-timeout`` Connection Timeout to talk to Pinot servers, default is 1 minute.
``pinot.metadata-expiry`` Pinot metadata cache expiration time, default is 2 minutes.
``pinot.estimated-size-in-bytes-for-non-numeric-column`` Estimated byte size for non-numeric column, default is 20.
``pinot.service-header-param`` RPC service service header key, default is "RPC-Service".
``pinot.caller-header-param`` RPC service caller header key, default is "RPC-Caller".
``pinot.caller-header-value`` RPC service caller header value, default is "presto".
``pinot.forbid-broker-queries`` No broker request pushing down, default is false.
``pinot.forbid-segment-queries`` No segment query pushing down, fail the query if broker query pushing down is not possible, default is false.
``pinot.rest-proxy-service-for-query`` Use rest proxy endpoint for Pinot broker requests, default is false.
``pinot.use-date-trunc`` Use the new UDF dateTrunc in pinot that is more presto compatible, default is false.
``pinot.num-segments-per-split`` Number of segments of the same host per split, default is 1.
``pinot.ignore-empty-responses`` Ignore empty or missing pinot server responses, default is false.
``pinot.fetch-retry-count`` Retry count for retriable pinot data fetch calls, default is 2.
``pinot.non-aggregate-limit-for-broker-queries`` Max limit for non aggregate queries to the pinot broker, default is 25000.
``pinot.infer-date-type-in-schema`` Infer Pinot DAYS epoch column to Presto DATE type, default is true.
``pinot.infer-timestamp-type-in-schema`` Infer Pinot SECONDS epoch column to Presto TIMESTAMP type, default is true.
``pinot.mark-data-fetch-exceptions-as-retriable`` Retry Pinot request when failure, default is true.
``pinot.use-pinot-sql-for-broker-queries`` Use Pinot SQL syntax, otherwise PQL syntax, default is true.
``pinot.pushdown-topn-broker-queries`` Allow pushing down query pattern to broker: aggregation + groupBy + orderBy, default is false.
``pinot.use-streaming-for-segment-queries`` Use gRPC endpoint for pinot server queries, default is false.
``pinot.streaming-server-grpc-max-inbound-message-bytes`` Max inbound message bytes when init gRPC client, default is 128MB.
``pinot.use-proxy-for-broker-request`` Sending broker query to Pinot Proxy, default is false.
``pinot.use-proxy-grpc-endpoint`` Sending segment query to Pinot Proxy gRPC endpoint, default is false.
``pinot.proxy-grpc-host`` Pinot proxy gRPC host.
``pinot.proxy-grpc-port`` Pinot proxy gRPC port.
``pinot.use-https-for-controller`` Use https for controller requests, default is false.
``pinot.use-https-for-broker`` Use https for broker requests, default is false.
``pinot.use-https-for-proxy`` Use https for proxy requests, default is false.
``pinot.override-distinct-count-function`` Override 'distinctCount' function name, default is "distinctCount".
``pinot.extra-http-headers`` Extra headers when sending HTTP based pinot requests to Pinot controller/broker. E.g. k1:v1,k2:v2.
``pinot.extra-grpc-metadata`` Extra metadata when sending gRPC based pinot requests to Pinot broker/server/proxy. E.g. k1:v1,k2:v2.
========================================================== =============================================================================================================

Session Properties
^^^^^^^^^^^^^^^^^^

The following session properties are available:

======================================================== ============================================================
======================================================== ==================================================================
Property Name Description
======================================================== ============================================================
======================================================== ==================================================================
``pinot.forbid_broker_queries`` Forbid queries to the broker.
``pinot.forbid_segment_queries`` Forbid segment queries.
``pinot.ignore_empty_responses`` Ignore empty or missing pinot server responses.
Expand All @@ -105,7 +105,8 @@ Property Name Description
``pinot.num_segments_per_split`` Number of segments of the same host per split.
``pinot.limit_larger_for_segment`` Server query selection limit for large segment.
``pinot.override_distinct_count_function`` Override distinct count function to another function name.
======================================================== ============================================================
``pinot.topn_large`` Cap the TOP/LIMIT value when pushing down broker query.
======================================================== ==================================================================

Map Pinot Schema to Presto Schema
---------------------------------
Expand Down

0 comments on commit afd4d2f

Please sign in to comment.