Release notes for 355

[martint]

David Phillips

• all checked
• 03-25 08:17:04 f1ca46d Add Hive schemas into InputFormat#getSplits
• 03-25 11:48:30 ad1e9cd Add doc for optimizer.use-table-scan-node-partitioning

Grzegorz Kokosiński

• all checked
• 03-20 12:34:52 6395ff0 Do not allow to INSERT into a table with row filter
• 03-20 12:34:52 ad4ab3f Improve error message
• 03-20 12:38:42 8ffe51d Prefer TrinoException over RTE in MysqlClient
• 03-22 05:21:06 1576895 Fix flaky Kafka Schema Registry service
• 03-22 05:21:06 2460e2e Reuse testing Kafka Schema Registry between tests
• 03-24 01:41:38 e98cfb0 Replace fail+catch pattern with assertThatThrownBy in trino-phoenix5
• 03-26 01:18:46 961b568 Set proper update type for REFRESH MATERIALIZED VIEW
• 03-29 07:35:01 b830a35 Shorten product tests GHA titles

Karol Sobczak

• all checked
• 03-22 03:00:30 98103ff Use ConnectorMaterializedViewDefinition#getStorageTable in analyzer
• 03-22 03:15:38 1b43456 Fix MockConnectorTableHandle#equals
• 03-22 03:15:38 d140073 Use source table name instead of ConnectorTableHandle#toString
• 03-22 03:17:03 0ab2753 Extend mock connector with getTableProperties customization
• 03-22 03:17:03 1876761 Rename optimizer.plan-with-table-node-partitioning
• 03-22 03:17:03 2ab30be Make setting of TableScanNode#useConnectorNodePartitioning rule based
• 03-22 03:17:03 604012a Use table scan node partitioning depending on bucket-to-task ratio
• 03-22 03:17:03 b2f1f69 Introduce TableScanNode#useConnectorNodePartitioning
• 03-22 03:17:03 eef0153 Rename param names to match type
• 03-23 03:48:37 5b45231 Record projected column handles in HiveTableHandle
• 03-23 09:03:36 3796ba5 Update use-preferred-write-partitioning doc
• 03-23 09:06:11 0c4a1f8 Add HiveRedirectionsProvider
• 03-24 04:06:47 0360628 Throw IllegalStateException for unknown state
• 03-24 05:13:25 7369479 Perform dereference pushdown after table scan redirection
• 03-24 05:13:25 ea704d4 Refactor mockApplyRedirectAfterProjectionPushdown
• 03-24 08:35:41 e4f0da3 Use SessionPropertiesProvider in hive connector
• 03-24 08:35:41 edeb48b Move SessionPropertiesProvider to trino-plugin-toolkit
• 03-26 04:51:59 d7cd385 Add doc for table-scan-node-partitioning-min-bucket-to-task-ratio
• 03-30 03:29:33 d7d9ea3 Update javadoc to match rule
• 04-01 08:38:11 b7c52b5 Add HiveMaterializedViewMetadata

Piotr Findeisen

• all checked
• 03-23 02:18:36 741ff8f Check for subject being just -value / -key
• 03-23 02:22:32 32d1977 Move PostgreSQL schema init to the server
• 03-23 02:22:32 8caa1d6 Simplify cleanup
• 03-23 02:22:32 8d6b86b Run PostgreSQL queries in tpch test schema by default
• 03-23 03:07:37 1319e20 Replace TestPlanMatchingFramework#assertFails with assertThatThrownBy
• 03-23 04:41:45 2623d1e Add support for create table with MergeTree engine.
• 03-23 06:27:43 dc84dac Rename onPresto to onTrino in product tests
• 03-23 08:21:48 08ffec1 Move testInsertInPresenceOfNotSupportedColumn into BaseJdbcConnectorTest
• 03-23 08:21:48 310c895 Update comment for UPDATE
• 03-23 08:21:48 782aee3 Remove redundant flexibility
• 03-23 08:21:48 938b34a Add a test when null_count is not set
• 03-23 08:21:48 980eb8b Improve checks readability
• 03-23 14:48:28 0a1464f Fix Hive VIEW with expression array subscript
• 03-23 14:48:28 9763f1d Split array view test
• 03-23 14:48:28 cdfe1c5 Reorganize testArrayIndexingInView test
• 03-24 01:08:32 5ed2606 Remove SQL MERGE syntax documentation
• 03-24 01:58:16 4a58c7a Remove default schema from testing PostgreSQL server URL
• 03-24 02:08:37 37ddda2 Fix misleading 'Table not found' message for PostgreSQL
• 03-24 03:20:23 3414184 Remove ConnectorSplitSource.getMinScheduleSplitBatchSize
• 03-24 03:44:59 28d19e2 Move testSetSession, testResetSession to engine-only tests
• 03-24 03:44:59 79482a9 Update @flaky's pattern after toString change
• 03-24 14:13:59 01cdc47 Clarify scope of local file connector
• 03-25 13:33:45 68e1cd5 Update Phoenix5 connector to 5.1.1
• 03-26 09:16:17 83bc4b8 Show spilled data in EXPLAIN ANALYZE
• 03-26 10:45:23 89e22cc Fix requireNonNull call sites
• 03-26 13:39:43 87d8d06 Test cached getTableStatistics with query-based handle
• 03-26 13:39:43 8edcd7c Cache statistics for JdbcTableHandle after complex pushdowns
• 03-26 13:39:43 e66516f Track referenced tables in JdbcTableHandle
• 03-27 10:29:00 3f3a91e Keep JdbcColumnHadle with JdbcTableHandle.sortOrder
• 03-27 10:29:00 a674c45 Use SQL standard syntax for TopN in PostgreSQL
• 03-27 10:29:00 d2751de Extract reusable TopNFunction leveraging SQL standard syntax
• 03-30 04:47:45 093abcf Use existing connection in assertMetadataCalls
• 03-30 04:47:45 266e001 Add compatibility with misbehaving client applications
• 03-30 04:47:45 28233c8 Count getTableHandle calls in metadata calls tests
• 03-30 04:47:45 2ac1cec Remove redundant suppression
• 03-30 05:00:44 4ff5421 Support LATERAL VIEW json_tuple AS ... in Hive VIEW
• 03-30 05:00:44 670cf1d Add description for max-recursion-depth config
• 03-31 03:02:15 7d3fb06 Use TEXTFILE for a table in test
• 03-31 07:24:47 b56e4fb Retry testExplainAnalyzeReportSpilledDataSize
• 04-01 06:26:10 9a337a2 Prevent join pushdown happening too early

Pratham

• all checked
• 03-31 14:37:28 08fe286 Remove redundant metastore calls while getting materialized view

Praveen Krishna

• all checked
• 03-23 04:27:13 9a3683d Use String formatting
• 03-23 04:27:13 d4e6a29 Use unique table name for TestKuduIntegrationDecimalColumns
• 03-23 23:51:09 2836a76 Mark a table if it is directly referenced by query
• 03-25 20:05:49 a2e8cbc Record input/output details when creating view/materialized view
• 03-27 19:01:04 a646c22 Create a new network for each instance of TestingKafka
• 03-31 00:26:53 5a038f8 Record output column details to EventListener
• 03-31 00:26:53 ec68911 Make mock connector create/insert/update friendly
• 03-31 02:55:51 1f12a7e Record output for CREATE TABLE

Łukasz Osipiuk

• all checked
• 03-22 08:33:20 0ca1b66 Make TopN over limit test deterministic
• 03-23 10:39:00 b84bf44 Allow creating HdfsContext without schema
• 03-24 14:45:05 27cbb53 Use queryId passed via Action
• 03-24 14:45:05 9b6fb64 Add queryId to Action in SemiTransactionalHiveMetastore
• 03-24 14:45:05 fcd7f32 Simpify HdfsContext
• 03-26 03:21:34 cf68c64 Test VIEW with Hive-specific syntax in non-legacy mode only
• 03-26 03:21:34 f5b12b0 Clean view tests

[kokosing]

General
* Do not allow to INSERT into a table with row filter. ({issue}`7346`)

#7346

[sopel39]

SPI:
* Engine now uses `ConnectorMaterializedViewDefinition#storageTable`
  to determine materialized view storage table ({issue}`7319`)

#7319

[sopel39]

General:
* Rename `optimizer.plan-with-table-node-partitioning` to `optimizer.use-table-scan-node-partitioning`. ({issue}`7257`)

#7257

[sopel39]

General:
* Planning with table scan node partitioning is now triggered automatically
  when ratio between table buckets and possible table scan tasks exceeds or is equal to
  `optimizer.table-scan-node-partitioning-min-bucket-to-task-ratio`. This helps to improve
  query parallelism when table bucket count is small compared to number of nodes. ({issue}`7257`)

#7257

[findepi]

Hive
* Fix incorrect query results when reading from a Hive view that subscripts an array. ({issue}`7271`)

#7271 #7272

[findepi]

General
* Include information about {doc}`/admin/spill` in {doc}`/sql/explain-analyze`. ({issue}`7427`)

-- i am using rst syntax for bullet content (but not for the link), as i don't know how to link to Spill docs correctly from markdown.
for EXPLAIN ANALYZE this can go as `EXPLAIN ANALYZE` as well.
@mosabua can you help me?

#7427

[mosabua]

@findepi I have not tried this out at all myself but we are using the Myst parser and the docs for it are here
https://myst-parser.readthedocs.io/en/latest/using/syntax.html

Specifically this should work
https://myst-parser.readthedocs.io/en/latest/using/syntax.html#targets-and-cross-referencing

[martint]

@findepi, take a look at https://github.com/trinodb/trino/blame/master/docs/src/main/sphinx/release/release-352.md#L5

[findepi]

JDBC driver
* Add `assumeLiteralNamesInMetadataCallsForNonConformingClients` parameter that can be used as a workaround for
  applications that do not escape schema or table names when passing them to `DatabaseMetaData` methods
  as schema or table name patterns. ({issue}`7438`)

#7438 #7424 (the issue won't be helpful when linked to from RNs, thus linking to PR)

[findepi]

Hive
* Support Hive VIEW containing `LATERAL VIEW json_tuple(...) AS ...` syntax. ({issue}`7242`)

#7242 #7420

[sopel39]

General:
* Report tables referenced in materialized views to even listener. ({issue}`7483`)

#7483

[Praveen2112]

General:
* Report columns which are created/inserted/updated to event listener. ({issue}`7425`)

#7425

[sopel39]

General:
* Add support for configuring multiple password authentication plugins. ({issue}`7151`)

#7151

[findepi]

ClickHouse
* Support creating tables with MergeTree storage engine. ({issue}`7135`)

#7135

[findepi]

General
* Improve interaction between Join pushdown and {doc}`/optimizer/cost-based-optimizations.rst`. ({issue}`7331`)

#7331

[findepi]

Hive
* Fix incorrect results returned when querying `$file_modified_time` hidden column. ({issue}`7511`)

#7511

[kokosing]

SQL Server
--- 
* Avoid metadata JDBC query to read table properties during SELECT statement analysis. ({issue}`6975`

Phoenix
--- 
* Avoid metadata JDBC query to read table properties during SELECT statement analysis. ({issue}`6975`

#6975

[findepi]

Phoenix
* Improve performance of queries with `ORDER BY ... LIMIT` clause, when the computation 
  can be pushed down to the underlying database. ({issue}`7490`)

#7490

[findepi]

PostgreSQL
* Improve performance of queries with `ORDER BY ... LIMIT` clause when sorting on  `char` or `varchar` columns
  and the computation can be pushed down to the underlying database. ({issue}`7494`)

#7494

[Praveen2112]

General
* Report lineage details for columns which are created or inserted or updated to event listener. ({issue}`7465`)

#7465