Add release notes for 0.280

[wanglinsong]

Missing Release Notes

Amit Dutta

• Fixing a typo. #19072 Fixing a typo. (Merged by: Michael Shang)
• [native] Remove deprecated.legacy-date-timestamp-to-varchar-coercion option from native e2e tests. #19010 [native] Remove deprecated.legacy-date-timestamp-to-varchar-coercion … (Merged by: Amit Dutta)

Ben Tony Joe

• Introduce new Keyword FETCH FIRST N ROWS ONLY in Presto Query #18968 Introduce new Keyword FETCH FIRST N ROWS ONLY in Presto Query (Merged by: Rebecca Schlussel)

Karteek Murthy Samba Murthy

• [Native][Issue-3593] Handle Task::updateBroadcastOutputBuffer failure gracefully with error log message #18939 [Native][Issue-3593] Handle Task::updateBroadcastOutputBuffer failure gracefully with error log message (Merged by: Aditi Pandit)
• [Native] Fix race condition that throws "Output buffers not found" #18776 [Native] Fix race condition that throws "Output buffers not found" (Merged by: Aditi Pandit)

Linsong Wang

• 960289f Update stable release branch development version to 0.280.1-SNAPSHOT

Michael Shang

• Update README on velox and prestissimo #19022 Update README on velox and prestissimo (Merged by: Michael Shang)

Paul Meng

• Revert "[issue-19030] add dependency-check-maven plugin to detect CPE… #19103 Revert "[issue-19030] add dependency-check-maven plugin to detect CPE… (Merged by: Timothy Meehan)

Shrinidhi Joshi

• Presto-on-Spark: Fix max drivers config for Native Task  #19061 Presto-on-Spark: Fix max drivers config for Native Task (Merged by: Pranjal Shankhdhar)

Zhongjun Jin (Mark)

• Prestissimo: Fix logic deserializing LongDecimal-typed Data Encoded in INT128_ARRAY in bytestream read #18838 Prestissimo: Fix logic deserializing LongDecimal-typed Data Encoded in INT128_ARRAY in bytestream read (Merged by: Maria Basmanova)

xiaoxmeng

• [native]Set promise out of ExchangeQueue lock (#4111) #19101 [native]Set promise out of ExchangeQueue lock (Move the SQL query below the Summary block #4111) (Merged by: Xiaoxuan)

枢木

• [native] Fix crash in window queries without ORDER BY clause #18895 [native] Fix crash in window queries without ORDER BY clause (Merged by: Maria Basmanova)

Extracted Release Notes

• Implement Spilling for TopNRowNumber Operator #18400 (Author: Shrinidhi Joshi): Implement Spilling for TopNRowNumber Operator
  • ...
  • ...
  • ...
  • ...
• Reimplementing REMOVE_NULLS as sql function #18661 (Author: Avinash Jain): Reimplementing REMOVE_NULLS as sql function
  • Add :func:remove_nulls to remove null elements from the given array.
• Fix wall time printed on CLI #18743 (Author: Anant Aneja): Fix wall time printed on CLI
  • Report both client-side and server-side latency in presto-cli result summary. With this change, the client-side latency is the wall time presto-cli spends for this query to finish, but no longer includes the time spent by user paging through results. The server side latency is acquired from the Presto server which represents the wall time the engine uses to finish executing this query.
• Use less restrictive column access checks when using transform and cardinality functions #18840 (Author: Pranjal Shankhdhar): Use less restrictive column access checks when using transform and cardinality functions
  • Use less restrictive column access checks when using transform and cardinality functions.
• Add S3 Select pushdown for JSON files #18902 (Author: dnnanuti): Add S3 Select pushdown for JSON files
  • Add Amazon S3 Select pushdown for JSON files.
• Fix bugs in duplicate items in UNNEST expression #18909 (Author: feilong-liu): Fix bugs in duplicate items in UNNEST expression
  • Fix the bug where duplicate items in UNNEST leads to query compilation failure.
• Fix blocking resource groups when query queue full #18910 (Author: Ruslan Mardugalliamov): Fix blocking resource groups when query queue full
  • Fix blocking resource groups when query queue full.
• Fix find_first function for NULL value #18952 (Author: feilong-liu): Fix find_first function for NULL value
  • Fix output of find_first function for NULL Now it will throw exception if the found matched value is NULL.
• Add range support for window function #18953 (Author: feilong-liu): Add range support for window function
  • Add range with offset expression support to window function.
• Add UDF find_first_index #18962 (Author: feilong-liu): Add UDF find_first_index
  • Add UDF find_first_index Returns the index of the first element which satisfies the condition.
• Fix merge window optimizers #18967 (Author: feilong-liu): Fix merge window optimizers
  • Fix bug in GatherAndMergeWindows optimization rule.
• Add invoked functions to query completed event #18980 (Author: Zac): Add invoked functions to query completed event
  • Add the names of the functions invoked by the query to the completed event. This can be enabled with the log_invoked_function_names_enabled session property or the log-invoked-function-names-enabled configuration property.
• Upgrade Airlift to 0.207 #19004 (Author: Jingmei Huang): Upgrade Airlift to 0.207
  • Upgrade Airlift to 0.207.
• Add Optimizer for validating and optimizing #19050 (Author: Rohit Jain): Add Optimizer for validating and optimizing
  • Added a new runtime metric optimizerTimeNanos to measure time taken by optimizers. The time taken by optimizers has been removed from runtime metric logicalPlannerTimeNanos.
• Prefilter GROUPBY LIMIT queries #19051 (Author: Sreeni Viswanadha): Prefilter GROUPBY LIMIT queries
  • Added a new optimization for filtering large tables with LIMIT number of keys for queries that do simple GROUP BY LIMIT with no ORDER BY. Added a boolean session param: prefilter_for_groupby_limit that can enable this feature.
• Fix race condition in InternalResourceManager initialization #19067 (Author: Rebecca Schlussel): Fix race condition in InternalResourceManager initialization
  • Fix a bug where if a query was submitted before the resource group configuration had been loaded, all queries to the cluster would fail with a NullPointerException.
• Add pluggable QueryAnalyzer interface #19093 (Author: Rohit Jain): Add pluggable QueryAnalyzer interface
  • Added a pluggable QueryAnalyzer interface for analyzing queries and creating logical plans. This interface can be implemented to add additional analyzers.

All Commits

• 960289f Update stable release branch development version to 0.280.1-SNAPSHOT (Linsong Wang)
• 9a8f91c [native] Update logging for Presto split events. (Sergey Pershin)
• 1210243 [native]Set promise out of ExchangeQueue lock (xiaoxmeng)
• c5b3bed Add pluggable QueryAnalyzer interface (Rohit Jain)
• b5df262 Revert "[issue-19030] add dependency-check-maven plugin to detect CPEs and CVEs" (Paul Meng)
• 74b0085 consider GDA as glacier as well (Lisheng)
• 545754f add GDA to glacier list (Lisheng)
• c8c8b82 consider GDA as glacier as well (Lisheng)
• 8d38a61 Update docker base image to centos:stream8 (Linsong Wang)
• 9cd6d43 Fix SSL error on web ui checks (Pranjal Shankhdhar)
• d4bba71 [native] Convert Presto's groupedExecutionScanNodes to Velox. (Sergey Pershin)
• 15a496c Fix resource loading issue in presto-plugin-toolkit test code (Michael Shang)
• 94b161c Optimization for prefiltering group by limit keys (Sreeni Viswanadha)
• fce77f5 Make ResourceGroupId segments immutable (Arun Thirupathi)
• 0fd0d6e Add range support for window function (feilong-liu)
• 3c7e006 [Native]] Log message in TaskManager on updateBroadcastOuputBuffer() (Karteek Murthy Samba Murthy)
• 7a6d02e Fixing a typo. (Amit Dutta)
• cd6a84d Fix race condition in InternalResourceManager initialization (Rebecca Schlussel)
• 6ea5f16 [issue-19030] add dependency-check-maven plugin to detect CPEs and CVEs (Linsong Wang)
• 7da651c Instrument the configureGroup failure (Arun Thirupathi)
• 458e54c Presto-on-Spark: Fix max drivers config for Native Task (Shrinidhi Joshi)
• c79a269 Add Optimizer for validating and optimizing (Rohit Jain)
• cf84fbf Remove Travis CI badge (Ajay George)
• d9cab46 Fix a presto native test case (Michael Shang)
• a580bba Report function names based on the kinds (Zac)
• 4a71c63 Refactor to remove duplicate LogicalPlanner constructor (Ajay George)
• 0786aa6 [native] Reflect Velox Facebook components versions (Miłosz Linkiewicz)
• ea617d1 Fix find_first function for NULL values (feilong-liu)
• 917d253 Fix name of find first function class (feilong-liu)
• 9b66dd0 [native] Advance velox version and fix a unit test case there (Michael Shang)
• 6169f3a [natvie] Fix data path in unit tests (Michael Shang)
• 68d953e Fix parquet tests to allow method level parallel runs (Anant Aneja)
• 66a5c9f Revert "Upgrade alluxio to 2.9.0" (Arun Thirupathi)
• 5342eea [native] Allow ssh-keyscan to fail (Miłosz Linkiewicz)
• aee410d Add getColumns to MetadataResolver (Rohit Jain)
• 63ca3e0 Make file formats in Hive integration tests customizable (Sergii Druzkin)
• c237612 Use separate properties for children of exchange node (singcha)
• 0b7f532 Add query completion event for non-shuffle stages (singcha)
• a86c2a6 Bring back TestLocalExecutionPlanner (Michael Shang)
• bbaaf25 Refactor SqlQueryExecution for readability (Ajay George)
• ab88363 Update README on velox and prestissimo (Update README on velox and prestissimo #19022) (Michael Shang)
• 959c3ee [native] Fix link order of Thrift and FBThrift libraries (Deepak Majeti)
• de74c0b Fix some test failures in presto-main (Michael Shang)
• cd1cf8d Fix merge window optimizers (feilong-liu)
• 1a8f6ce Fix incorrect catalog session property name in Hive Connector Documentation (Jalpreet Singh Nanda (:imjalpreet))
• 77c0949 Add writer's hostname to ORC file metadata (Sergii Druzkin)
• a1eae0a Introduce adaptive query executor (Konstantinos Karanasos)
• c6a3e8b [native] Add e2e tests for binary functions (Ivan Sadikov)
• 5d8b74d Update alluxio cache.dir config in the test (Chunxu Tang)
• 087f4ff Add FunctionAndTypeResolver (Rohit Jain)
• cc793ac [native] Use Native Parquet Reader by default (Deepak Majeti)
• 6c4459c Avoid TaskRunner destruction on cancel (James Petty)
• c4a2590 Stop RetryDriver when interrupted (James Petty)
• fc20aa0 Stop RetryDriver on S3 AbortedException (James Petty)
• b21715e [native] Fix deserializing of encoded INT128_ARRAY in byte-stream read (Zhongjun Jin (Mark))
• 529bd72 [native] Fix Cast expressions on Varbinary (Ivan Sadikov)
• 2a9bbcb Add invoked functions to query completed event (Zac)
• e0980a1 Remove expired nodes in ResourceGroupStates (Jingmei Huang)
• 0325d3d Upgrade Airlift to 0.207 (Jingmei Huang)
• 4be06e2 Add UDF find_first_index (feilong-liu)
• 9c52ddf [native] Add an e2e test for shuffle function (Darren Fu)
• 96d9c51 [native] Remove deprecated.legacy-date-timestamp-to-varchar-coercion from native e2e test. (Amit Dutta)
• f8b3b84 [native] Advance Velox version. (Sergey Pershin)
• 12372c4 Use query id to name memory pool (xiaoxmeng)
• 612248a [native] Advance Velox version (tanjialiang)
• 1c713e7 [native] Allow http server port reuse (tanjialiang)
• 3b151ed Add http reuse port property to Spark Native system property (tanjialiang)
• 6d84a19 Add shuffle name property to system property (tanjialiang)
• d542786 Fix a build failure caused by git version update (Linsong Wang)
• db349c0 [native] Change UnsaferowShuffleTest to adapt to new ShuffleRead operator (tanjialiang)
• e46a5fa Refactor VeloxQueryPlanConverter to support batch conversion (tanjialiang)
• 441b2d4 Add ShuffleReadOperator and its plan node (tanjialiang)
• 282851a [native] Advance Velox version (tanjialiang)
• 60803f5 Review Changes (Ben Tony Joe)
• 2284af4 Introduce new Keyword FETCH FIRST N ROWS ONLY in Presto Query (Ben Tony Joe)
• 8e4be4c fix failure while registering query completion (singcha)
• 3dfbbdc Move analysis to presto-analyzer (Rohit Jain)
• ba19543 Move getTableHandle to metadataResolver (Rohit Jain)
• c8addd6 Move TableMetadata to presto-spi (Rohit Jain)
• 45d1956 Move AllowAllAccessControl to presto-spi (Rohit Jain)
• b1a1f2a HBO Canonicalization support for DistinctLimitNode (Pranjal Shankhdhar)
• 54b3032 HBO Canonicalization support for TopNRowNumberNode (Pranjal Shankhdhar)
• e882367 HBO Canonicalization support for TopNNode (Pranjal Shankhdhar)
• 94dd94a HBO Canonicalization support for RowNumberNode (Pranjal Shankhdhar)
• 031b4c9 HBO Canonicalization support for SemiJoinNode (Pranjal Shankhdhar)
• e57077f [Native] Advance Velox version (Ge Gao)
• 12012ee Fix broken URLs in BigQuery documentation (Vladimir Ozerov)
• 297582f [native] Let TaskManager/TaskResource catch clause catch exact type without losing details (tanjialiang)
• afae141 Add JSON support for Hive connector with S3 Select pushdown (dnnanuti)
• 54a2c27 Support reading partitions different from Table format (Arun Thirupathi)
• 4203c20 HBO Canonicalization support for MarkDistinct/AssignUniqueId/EnforceSingleRow (Pranjal Shankhdhar)
• 4e8a858 HBO Canonicalization support for SortNode (Pranjal Shankhdhar)
• 6adb5ee HBO Canonicalization support for ValuesNode (Pranjal Shankhdhar)
• dfa19ac HBO Canonicalization support for WindowNode (Pranjal Shankhdhar)
• 270f619 Nits in CanonicalPlanGenerator (Pranjal Shankhdhar)
• 9b488fb Use per-query cache for HBO stats (Pranjal Shankhdhar)
• 2f7ea25 Fix wall time printed on CLI (Anant Aneja)
• 6b1dc67 [native] Add log details when getResult() on a failed PrestoTask (tanjialiang)
• cdfc5d2 [native] Advance Velox version. (Sergey Pershin)
• 77fb254 Implement Spilling for TopNRowNumberOperator (Shrinidhi Joshi)
• b6a2899 Move AccessControl to presto-spi (Rohit Jain)
• 5c0ba40 Move TransactionId to presto-common (Rohit Jain)
• 8649714 [native] nits refactorings (tanjialiang)
• a050350 [native] Add and use "use-mmap-allocator" system property. (Sergey Pershin)
• 3bd0fcd Fix ShuffleManager for getting the shuffleHandle (MJ Deng)
• 5334129 Inject shuffle write info to native execution entities and wwitch to BatchTaskUpdateRequest (MJ Deng)
• 04a2562 Add PrestoSparkNativeTaskRdd for native execution (tanjialiang)
• da55329 [native] Add support for Date Tablescan filters (Aditi Pandit)
• fb2e12a [prestissimo] Advance Velox version (tanjialiang)
• b1c17da Refactor S3 Select related classes before JSON support (dnnanuti)
• 7bf109c Fix setting PartititioningHandle in IterativePlanFragmenter (Rebecca Schlussel)
• e90dd3b Skip access check argument in cardinality function (Pranjal Shankhdhar)
• 053658a Support subfield acl checks using transform (Pranjal Shankhdhar)
• 1856b27 Fix failing Timezone test (dnnanuti)
• f2808c2 Fix blocking resource groups when query queue full (Ruslan Mardugalliamov)
• 9e01c81 Table writer 4: Refactor table writer API usage (Ge Gao)
• 63a0071 Return the original type for null RLE block (Zac)
• 4f8f67a Add an e2e test for Repeat function (Zac)
• 096fa33 Advance Velox version (Zac)
• 1387be7 Refactor local shuffle info (tanjialiang)
• dfefa5c Add PrestoSparkShuffleDescriptor to contain shuffle data from Spark (MJ Deng)
• a7827c9 Fix bugs in duplicate items in UNNEST expression (Feilong Liu)
• 8e6d0e2 [Native] Fix race condition that throws Output buffer not found (Karteek Murthy Samba Murthy)
• fe8deb9 [native] Fix crash in window queries without ORDER BY clause (枢木)
• 46aa056 Reduce logging overheads in TaskExecutor (James Petty)
• 7346b4f Remove redundant System.nanoTime call (James Petty)
• 4903016 Avoid synchronizing in TaskHandle#isDestroyed() (James Petty)
• ac2d452 Re-implementing remove_nulls with sql function (Avinash Jain)

[yingsu00]

Many of the commits need release notes. E.g.
[native] Remove deprecated.legacy-date-timestamp-to-varchar-coercion … #19010 (Merged by: Amit Dutta @amitkdutta )
Introduce new Keyword FETCH FIRST N ROWS ONLY in Presto Query #18968 (Merged by: Rebecca Schlussel @rschlussel )

Also I think the two fixes by @karteekmurthys and @aditi-pandit better have release notes too, since it changes the way log is written and therefore is considered user-facing.

Would you please update your release notes following the following guidelines please?
https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines

[rschlussel]

The fetch first release note can be

* Add ANSI SQL compliant ``FETCH FIRST n ROWS ONLY`` syntax as an alternative way to express ``LIMIT n``