Release notes for 318

[sopel39]

Dain Sundstrom

• all checked
• 08-07 10:41:52 2597804 Fix distinct/groupby on UUID
• 08-15 16:04:47 b619848 Add missing doAs around Hive create empty bucket files

David Phillips

• all checked
• 08-02 15:21:45 16a1eeb Update gcs version
• 08-06 22:53:25 0c8706b Extract caching Hive metastore to separate package
• 08-06 22:55:04 12328e3 Improve error messages for loading various plugin services
• 08-07 18:13:20 e82c2d5 Add Iceberg connector
• 08-13 16:24:02 466edbc Remove Google Sheets Connector's config file
• 08-13 18:20:56 181a7c6 Should ignore all Exception from close
• 08-14 09:32:45 3de21b8 Add limit pushdown to Information Schema Connector
• 08-14 22:38:36 e02a12c Add Google Sheets connector documentation
• 08-15 10:51:23 f82073c Fix duplicate field IDs when creating Iceberg tables
• 08-15 15:41:18 efc2bc1 Improve error reporting for Sheets connector
• 08-15 21:46:13 67443c1 Move product tests defaults to presto-product-tests
• 08-16 10:05:22 d463bb0 Fix warnings in PrestoS3FileSystem
• 08-16 10:09:14 d1306b5 Add hive.s3 option hive.s3.signer-class
• 08-16 15:43:46 2718f1a Add last_day_of_month function
• 08-16 16:05:25 ab0382c Code cleanup in utility classes
• 08-16 16:05:25 fb5c165 Remove unnecessary lenient treatment of null value
• 08-16 16:11:34 8c65b0a Add support for requester pays bucket in PrestoS3FileSystem

Grzegorz Kokosiński

• all checked
• 08-04 11:38:03 8e3d74d Fix a small typo in presto-benchto-benchmarks/README.md
• 08-05 03:52:49 66632e2 Minor fixes around aggregation execution
• 08-05 06:48:27 43c25ec Extract SecurityContext classes
• 08-05 06:48:27 642fda2 Cleanups in TestFileBasedAccessControl
• 08-06 00:30:33 1e4f155 Allow connector to override min schedule split batch size
• 08-07 00:40:44 1998d60 Fix formatting
• 08-07 00:42:15 4ecdfc0 bump snappy version to 1.1.7.1
• 08-07 00:42:15 962d3c4 bump kafka version to 1.1.1
• 08-09 01:36:30 92c8ca1 Add run_presto_cli_on_docker.sh and run_tempto_on_docker.sh
• 08-12 02:02:48 087f1b1 Wrap extra credentials on top of existing CredentialProvider
• 08-12 02:02:48 32bd78d Add credential provider for JDBC connectors
• 08-12 04:06:06 0d02779 Log JdbcClient invocations
• 08-12 04:06:06 8d11ca8 Move LoggingInvocationHandler to presto-plugin-toolkit
• 08-12 04:23:34 453bf6f Avoid nested try-with-resources
• 08-12 12:37:43 a911bba Adding google sheets plugin
• 08-13 06:24:39 ab127b8 Fix wrong value in gsheets.properties
• 08-16 23:03:55 8599c9b Rename totalPlanningTime to planningTime
• 08-16 23:03:55 8a9bcf0 Add analysis time to web ui query detail page
• 08-16 23:03:55 9df4d41 Collect query semantic analysis query statistic
• 08-16 23:03:55 b367996 Remove obvious comments
• 08-16 23:03:55 d258744 Remove analysisTime and distributedPlanningTime
• 08-16 23:03:55 fb7f5bd Add planning time to QuerySystemTable

Karol Sobczak

• all checked
• 08-02 01:04:55 88eb7b0 Upgrading aws-sdk version to current (1.11.602)
• 08-02 01:39:59 2e9b62d Do not unwrap LazyBlocks in PageUtils#recordMaterializedBytes
• 08-02 01:39:59 de82684 Add Block#isLoaded method
• 08-06 02:39:18 51131ff Simplify RecordPageSource
• 08-06 02:39:18 78f9a89 Make TPCH connector return lazy pages
• 08-06 02:39:18 85a658f Do not use lineitem table in semi join work processor queries
• 08-06 02:39:18 e824116 Ensure that work processor queries process less input data
• 08-06 02:39:18 e966976 Test TopN work processor query with filter condition
• 08-07 02:26:23 100197b Replace deprecated DistributedQueryRunner constructor
• 08-07 02:26:23 ad99ec9 Adding memory related session properties
• 08-09 15:29:02 4a9476d Make TestTable name really unique
• 08-09 15:29:28 bc5590c Make Kerberos config properties consistent
• 08-13 00:51:56 570d897 Remove unused method ValueInputStream#getCheckpointType
• 08-14 15:24:22 94e08b8 Make PrestoS3InputStream#read comply with read method contract on EOF

Martin Traverso

• all checked
• 08-13 14:55:55 3583d8e Remove unused variable
• 08-13 14:55:55 67bb042 Remove redundant hash computation
• 08-13 14:55:55 b5d5540 Add a PlanMatcher for matching identity projections
• 08-13 16:45:41 a7c351e Report offending array subscript when out of bounds
• 08-13 16:49:08 ce69fff Fetch SearchHits scroll by scroll as needed than all at once in Elasticsearch
• 08-15 10:00:07 93412b3 Remove redundant call
• 08-15 10:00:07 96b9a52 Disable product tests on HDP 3.1
• 08-15 10:00:07 a99d301 Remove redundant exclusion in test
• 08-15 10:00:07 ec18f1d Code cleanup
• 08-15 10:00:07 f8b79c9 Remove unused variable in test
• 08-16 15:43:25 1382c35 Remove unused field

Piotr Findeisen

• all checked
• 08-02 01:38:26 16c33b2 Update docker images to 16
• 08-02 01:38:26 2166560 Run product tests also on HDP 3.1 on "branch builds"
• 08-02 01:38:26 f907081 Open more ports in test environment
• 08-02 12:34:48 03c1e9d Fix typo in docs
• 08-03 13:31:02 7c6c316 Add "Asia/Qostanay" time zone
• 08-04 12:25:15 24a40be Pass table handles to beginQuery
• 08-05 08:06:50 28bd075 Distribute product tests more evenly on Travis
• 08-06 08:32:13 fdf498a Add support for Array of timestamps in PostgreSQL

Raghav Sethi

• all checked
• 08-13 17:15:04 2165aee Implement periodic cpu usage tracking in resource groups
• 08-13 17:15:04 4f79339 Enable resource updates in MockManagedQueryExecution

[sopel39]

Hive
----

* Improve performance of S3 Object listing. (:issue: `1232`)

#1232

[sopel39]

SPI
---

Add `Block#isLoaded` method. (:issue: `1216`)

#1216

[findepi]

* Add support for ``Asia/Qostanay`` time zone. (:issue:`1221`)

#1221

[kokosing]

SPI Changes
-------------
* Introduce ``SecurityContext`` group of classes to store security related information that is passed to plugins. (:issue:`171`)

#171

[kokosing]

SPI Changes
-------------
* Allow connector to override minimal schedule split batch size. (:issue:`1251`)

#1251

[kokosing]

Kafka Connector Changes
--------------------------
 * Update Kafka client library to ``1.1.1``. (:issue:`1112`)

#1112

[sopel39]

General
-------

* Add session properties for `query_max_memory_per_node` and
  `query_max_total_memory_per_node`. (:issue:`1212`)

#1212

[dain]

General
-------

* Fix ``DISTINCT FROM`` for ``UUID`` type. (:issue:`1180`)

#1180

[sopel39]

General
-------

* Rename Kerberos properties to make them consistent with other
  HTTP server properties. (:issue:`1270`)

#1270

[kokosing]

Jdbc-based connectors changes
---------------------------------
* Add credential provider for JDBC connectors. Thanks to that user can pass RDBMS credentials in separate file, by using ``credential-provider.type=FILE`` and ``connection-credential-file=credentials.properties``. (:issue:`1124`)

#1124

[kokosing]

Jdbc-based connectors changes
-----------------------------
- Allow logging all calls to ``JdbcClient``. This can be enabled by turning on ``DEBUG`` logging for ``io.prestosql.plugin.jdbc.JdbcClient``. (:issue:`1274`)

#1274)

[kokosing]

General Changes
-----------------
- Introduce Google Sheets Connector. (:issue:`1030`)

#1030

[phd3]

General
-----------------
- Add periodic cpu usage tracking in resource groups (:issue: `1128`)

#1128

[dain]

Hive Changes
------------

* Fix query failure due to missing credentials while writing empty bucket files. (:issue: `1298`)

#1298 (fix is in PR #1303)

[electrum]

Hive Changes
---

* Improve performance when reading data from GCS. (:issue:`1200`)

#1200

[electrum]

General Changes
---

* Improve performance for queries over ``information_schema`` that have a limit. (:issue:`999`)

#999

[electrum]

General
---

* Add :func:`last_day_of_month` function. (:issue:`1295`)

#1295

[electrum]

Hive Changes
---

* Add support for reading data from S3 Requester Pays buckets. This can be enabled
  using the ``hive.s3.requester-pays.enabled`` configuration property. (:issue:`1241`)

#1241

[kokosing]

General Changes
------------------
 - Remove ``distributed_planning_time_ms`` column from ``system.runtime.queries``. (:issue:`1084`)
 - Change the semantics of ``analysis_time_ms`` column from ``system.runtime.queries``. Previously it was showing the time spent on query planning, now it  shows time spent on semantic analysis. of query. (:issue:`1084`)
 -  Add ``planning_time_ms `` column to ``system.runtime.queries`` that shows time spent on query planning. (:issue:`1084`)

#1084

[martint]

* Improve query performance when ``optimize_hash_generation`` is enabled. (:issue:`1071`)

#1071

[raghavsethi]

I would recommend amending #1238 (comment) to the following to better explain the user impact.

- Change query CPU tracking for resource groups to update periodically while 
the query is running. Previously, CPU usage would only update at query 
completion. This improves resource management fairness when using 
CPU-limited resource groups. (:issue: `1128`)

[kokosing]

General Changes
------------------
- Allow to display details of query that is in the queue or during semantic analysis. Query details are accessible from the web ui or ``system.runtime.queries``. (:issue: `1079`)
- Allow to cancel query that is in the queue or during semantic analysis. Query can be cancelled from the web ui or using builtin procedure ``system.runtime.kill_query``.(:issue: `1079`)

#1079

[findepi]

PostgreSQL Connector Changes
---
* Add support for PostgreSQL ``timestamp[]`` type. (:issue:`1023`, :issue:`1262`)

#1023, #1262

[kokosing]

Hive Connector Changes
-----
 - Support ``INSERT INTO`` bucketed unpartitioned Hive table. (:issue:`1127`)

#1127

[dain]

General
-------

* Fix ``DISTINCT FROM`` for ``UUID`` type. (:issue:`1291`)

#1291