Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add release notes for 0.272 #17564

Merged
merged 1 commit into from Apr 5, 2022
Merged

Add release notes for 0.272 #17564

merged 1 commit into from Apr 5, 2022

Conversation

asjadsyed
Copy link
Member

@asjadsyed asjadsyed commented Apr 1, 2022

Missing Release Notes

Amit Dutta

  • #17363 Throw proper error code when timezone is invalid from packDateTimeWit… (Merged by: Timothy Meehan)
  • 3913a1a Enable Retrying max requests queued exception.

George Wang

  • #17132 Add TPCDS schema to HiveQueryRunner (Merged by: Ying)

Ke Wang

  • #17281 Add streaming aggregation (Merged by: James Sun)

Sergey Smirnov

  • #17391 Thriftify ServerInfo (Merged by: Timothy Meehan)

Tim Meehan

  • #17378 Add adaptive execution policy (Merged by: Timothy Meehan)
  • 290bc4a Fix buffer size calculation in TDigest

Todd Gao

  • #17218 Add lark sheets connector (Merged by: Beinan)

Extracted Release Notes

  • #16756 (Author: v-jizhang): Add a new JDBC driver parameter - timeZoneID
    • Add a new JDBC driver parameter - timeZoneID :doc:/installation/jdbc :issue:16680.
  • #16983 (Author: panyliu): Support iceberg connector concurrent insertion
    • Support concurrent insertion from the same presto cluster or mutiple presto clusters which sharing the same metastore.
  • #17083 (Author: v-jizhang): Ensure Parameters are in proper order for queries having WITH clause
    • Ensure Parameters are in proper order for queries having WITH clause.
  • #17244 (Author: Sagar Sumit): Improve listing performance of Hudi tables
    • Upgrade Hudi version to 0.10.1.
    • Support metadata-based listing and bootstrap for Hudi tables.
  • #17284 (Author: Xinli shang): Skip reading Parquet pages using Column Indexes feature.
    • Add support to use Parquet page-level statistics to skip pages.
    • Introduce a flag "hive.parquet-column-index-filter-enabled" to turn on/off this feature. By default it is off.
  • #17288 (Author: Rongrong Zhong): Enable cost based optimization by default
    • Enable cost based optimization by default.
  • #17295 (Author: v-jizhang): Fix parquet data page offset calculation in parquet writer
    • Fix parquet data page offset calculation in parquet writer.
  • #17323 (Author: Xiang Fu): [Pinot Connector] Support querying Pinot JSON type
    • Support querying Pinot JSON type.
  • #17327 (Author: David N Perkins): Bump Alluxio client from 2.7.0 to 2.7.3
    • Bumped Alluxio client from 2.7.0 to 2.7.3.
  • #17393 (Author: Josh Soref): Spelling
    • Fix the spelling of the write concern option JOURNAL_SAFE for the property mongodb.write-concern.
  • #17399 (Author: Darren Fu): Add secure_random functions
    • Add secure_random() and secure_random(lower, upper) with uniform distribution.

All Commits

  • 7cb1fef Bump up sphinx version to fix docs build (James Sun)
  • 290bc4a Fix buffer size calculation in TDigest (Tim Meehan)
  • 3913a1a Enable Retrying max requests queued exception. (Amit Dutta)
  • 8479f26 Fix presto on spark tests being skipped (Ariel Weisberg)
  • e7f1401 Kill queries aggressively under high task pressure (Mayank Garg)
  • 1e64daf Expose ORC writer compression level as session property (Sergii Druzkin)
  • 028ef8e Fix TestPrestoQueries (Ariel Weisberg)
  • 8750895 Add lark sheets connector (Todd Gao)
  • 36cfca4 Support isOrderable trait for distinct types (Pranjal Shankhdhar)
  • 3c800f0 Add tests for DictionaryBlock size methods (James Petty)
  • 2a79ba1 Make DictionaryBlock sizeInBytes calculation consistent (James Petty)
  • f0ea5b9 Add more benchmark for block position sizes (James Petty)
  • 0280165 Record that the result of DictionaryBlock#copyPositions is compact (James Petty)
  • 4871cee Opportunistically record computed dictionary size and uniqueIds (James Petty)
  • e4f477c Add support for fixed width blocks computing their size more efficiently (James Petty)
  • bb0ee5c Avoid redundant metastore calls for non-system Iceberg tables (Chunxu Tang)
  • 7e138de Support distinct type in hive (Pranjal Shankhdhar)
  • 40a79cd Make TestPinotSegmentPageSource single threaded (Neerad Somanchi)
  • f63dfbc Add runtimeStats to track read calls to underlying storage (Nikhil Collooru)
  • 9faf7d5 Support directly writing bytes to VariableWidthBlockBuilder (Arunachalam Thirupathi)
  • 0233d34 Write data directly to OutputBuffer (Arunachalam Thirupathi)
  • 807cb33 Avoid Slice allocation for Statistics (Arunachalam Thirupathi)
  • 733d444 Generalize storage format in file metastore (Todd Gao)
  • 2823595 Add some tests on serialization for file metastore (Todd Gao)
  • 9a50892 Propagate session's connector properties in analyzeView (mengdilin)
  • 85a5aa5 Add operators for distinct types (Pranjal Shankhdhar)
  • 5c6efc7 Fix spelling (Josh Soref)
  • 7c1b764 Fix out-of-date links in Cassandra connector doc (v-jizhang)
  • f11dcb6 Fix incorrect type mapping in Cassandra documentation (v-jizhang)
  • 1329da0 Fix typo in Cassandra doc (v-jizhang)
  • 3020df1 implement secure_random function with uniform distribution (Darren Fu)
  • 7b212bb Reduce hadoop configuration construction time (zhangyanbing)
  • 0ebe1de Fix small inefficiency in StripeReader (Sergii Druzkin)
  • 6e88468 Add OrcReaderOptions builder (Sergii Druzkin)
  • 5eae5e5 Add adaptive execution policy (Tim Meehan)
  • ba99b74 Add Thrift serde support for ServerInfo (Sergey Smirnov)
  • 668d495 get raw filesystem should consider CachingFileSystem (guojianhua)
  • 1788358 Add casts and operators for distinct types (Pranjal Shankhdhar)
  • 8982bab Refactor lookupCast(). Use Type instead of TypeSignature (Pranjal Shankhdhar)
  • ef1fd25 Improve listing performance of Hudi tables (Sagar Sumit)
  • a9e6b6e Fix TestMemoryManager#testQueryMemoryPerNodeLimit flaky test (Swapnil Tailor)
  • fce3301 Disable flaky TestDistributedQueryResource.testGetAllQueryInfoForLimits (Masha Basmanova)
  • 143c6f6 Allow to pick a subset of TPC-DS tables to create in HiveQueryRunner (Masha Basmanova)
  • c862457 Export DispatchManager counters (Mayank Garg)
  • de18be1 Add TPCDS schema to HiveQueryRunner (George Wang)
  • 172763f refactor logic to support create TPC tables for HiveQueryRunner (George Wang)
  • 7830670 Throw proper error code when timezone is invalid from packDateTimeWithZone function. (Amit Dutta)
  • 241adf1 Add join table reference check in JOIN ON clause (Ruslan Mardugalliamov)
  • 761cd55 Fix cleanup workflow reference (Josh Soref)
  • 8c50171 Align intermediate types for AVG aggregate function (Masha Basmanova)
  • 4350f22 Handle the scalar subqueries for unnest (Maksim Dmitriyevich Podkorytov)
  • e5c35a6 Fix cleanup CI (Josh Soref)
  • e91cbb0 Add min compressible size to OrcOutputBuffer (Sergii Druzkin)
  • 25b2f25 Ensure Parameters are in proper order for queries having WITH clause (v-jizhang)
  • dd673a9 Fix bug in OrcOutputBuffer leading to severe undercompression of large blobs (Sergii Druzkin)
  • ae9f559 Refactor iceberg multi-catalog smoke tests (Chunxu Tang)
  • 2081ec9 Add getDefaultBlockSize and getDefaultReplication to CachingFileSystem (Chen)
  • bcf13da Refactor null handling in ORC Dictionary writers (Arunachalam Thirupathi)
  • f7774e5 Fixing TestServerInfoResource (Swapnil Tailor)
  • 2ebf7ea Add distinct type definition (Pranjal Shankhdhar)
  • e01d5ca Add ORC writer flush policy (Sergii Druzkin)
  • b6eb0a0 Bumped Alluxio client to 2.7.3 (David N Perkins)
  • 79370de Add a new JDBC driver parameter timeZoneId (v-jizhang)
  • 3509718 Support optional row numbers in OrcSelectiveReader (Harsha Rastogi)
  • aab1acf Add default implementation to HiveMetastore to fix the build (Beinan Wang)
  • 7b83d6a Skip synthetic members to enable code coverage (Guy Moore)
  • 39b6cea Remove the junit dependency to fix the build (Beinan Wang)
  • b1402dc Create an abstract class for metadata of multiple iceberg catalogs (Chunxu Tang)
  • e3c9af6 Enable cost based optimization by default (Rongrong Zhong)
  • ca90676 Improve memory accounting for broadcast table (Arjun Gupta)
  • 5edaef8 check field runtimeStats to prevent accessing undefined properties (ahouzheng)
  • 2fb2c31 Add ability to do streaming for partial aggregation (Ke Wang)
  • f66c85b Add ability to do streaming aggregation for hive table scans (Ke Wang)
  • 39d20a8 Add ability to disable splitting file in hive connector (Ke Wang)
  • afd4d2f fixing presto pinot doc for pinot catalog configs and add session property pinot.topn_large (Xiang Fu)
  • 59d092c Support iceberg connector concurrent insertion (panyliu)
  • fe1e84d Fixing Distributed Query Runner (Swapnil Tailor)
  • 3cb1ee2 Skip reading Parquet pages using Column Indexes feature (Xinli shang)
  • c911689 Add unit test for ThreadResource (Alan Xu)
  • c07106b ThreadResource supports JSON serde at present. (Alan Xu)
  • 6b5da9a Support json type in PinotSegmentPageSource (Xiang Fu)
  • 4d8aad1 Fix parquet data page offset calculation in parquet writer (v-jizhang)

Copy link
Contributor

@NikhilCollooru NikhilCollooru left a comment

Please follow the guidelines listed here for the wording ..
https://github.com/prestodb/presto/wiki/Release-Notes-Guidelines#order-of-changes

@asjadsyed asjadsyed requested a review from NikhilCollooru Apr 1, 2022
@NikhilCollooru
Copy link
Contributor

@NikhilCollooru NikhilCollooru commented Apr 1, 2022

Please squash all the commits to one .

@asjadsyed asjadsyed marked this pull request as draft Apr 2, 2022
@asjadsyed asjadsyed marked this pull request as ready for review Apr 4, 2022
presto-docs/src/main/sphinx/release/release-0.272.rst Outdated Show resolved Hide resolved
presto-docs/src/main/sphinx/release/release-0.272.rst Outdated Show resolved Hide resolved
presto-docs/src/main/sphinx/release/release-0.272.rst Outdated Show resolved Hide resolved
presto-docs/src/main/sphinx/release/release-0.272.rst Outdated Show resolved Hide resolved

Hive Changes
____________
* Fix Parquet data page offset calculation in Parquet writer.
Copy link
Contributor

@tdcmeehan tdcmeehan Apr 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to figure out what this means. Perhaps "Fix error in Parquet writer".

presto-docs/src/main/sphinx/release/release-0.272.rst Outdated Show resolved Hide resolved

Hive Changes
____________
* Fix offset calculation in Parquet writer.
Copy link
Contributor

@highker highker Apr 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a user impact for this item? Like what query would fail before without the fix?

Copy link
Member Author

@asjadsyed asjadsyed Apr 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on #17295: Trying to write a file larger than ~2 GB will results in an integer overflow exception

Copy link
Member Author

@asjadsyed asjadsyed Apr 5, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated wording to

Fix integer overflow exception in Parquet writer when writing files larger than ~2 GB.

presto-docs/src/main/sphinx/release/release-0.272.rst Outdated Show resolved Hide resolved
@asjadsyed asjadsyed requested a review from highker Apr 5, 2022
highker
highker approved these changes Apr 5, 2022
@highker highker merged commit f07faff into prestodb:master Apr 5, 2022
41 checks passed
@rohanpednekar
Copy link

@rohanpednekar rohanpednekar commented Apr 26, 2022

Folks, how long it will take to get this release published on prestodb.io? or is this blocked?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants