Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release notes for 302 #125

Closed
electrum opened this Issue Jan 31, 2019 · 17 comments

Comments

6 participants
@electrum
Copy link
Member

electrum commented Jan 31, 2019

Dain Sundstrom

  • all checked
  • 01-31 15:18:24 10546ea Reformat ParquetTester
  • 01-31 15:18:24 bf01fcd Add dynamic batch sizing in Parquet reader
  • 01-31 15:27:30 99bd6ba Rename Context method to match field names
  • 01-31 15:27:30 bacf59f Add physical* and network* metrics
  • 01-31 15:27:30 e81eefe Remove unused methods
  • 01-31 16:30:15 1fad470 Disable ConnectorIdentity#equals and #hashCode
  • 01-31 16:30:15 4a4bf3f Improve ConnectorIdentity#toString
  • 02-04 14:45:12 6903d9b Add CLI option --trace-token to set the trace token #56.
  • 02-04 16:55:23 f7fe35c Fix starvation while waiting for minimum number of workers
  • 02-05 11:41:04 d6fb5e0 Add peak task user memory usage to query stats
  • 02-05 12:02:33 d7d6166 Add spilled data to query stats, CLI, and Web UI
  • 02-05 12:32:15 7ff3b40 Allow function implementation choices to have different dependencies
  • 02-06 09:06:30 275bb36 Remove system pool entirely

David Phillips

  • all checked
  • 01-31 17:15:52 0b55a32 Remove unused class ConnectorAnalyzeMetadata
  • 02-02 01:31:23 3fc4aff Use class reference for BlockMissingException
  • 02-02 16:17:58 02711a8 Move ScheduledSplit to execution package
  • 02-02 16:17:58 6082254 Move OutputBuffers to execution.buffer package
  • 02-02 16:17:58 8ca92f2 Move TaskSource to execution package
  • 02-02 16:37:56 d983340 Cleanup code in ORC module
  • 02-04 14:26:47 81d77ca Fix grammar in error messages for CDF functions
  • 02-04 15:30:50 20fb7d8 Document hive.recursive-directories
  • 02-05 14:04:02 c2f23eb Rename StageExecutionStrategy to StageExecutionDescriptor
  • 02-05 15:32:28 8ba4f26 Rename methods in StageExecutionDescriptor
  • 02-05 17:04:29 30f7812 Improve performance for positioned reads on S3
  • 02-05 17:04:29 5c57d92 Throw IOException for unrecoverable S3 errors
  • 02-05 22:18:08 0a543f6 Add option not to delete target directory on failure
  • 02-05 22:18:08 14cc065 Remove redundant null check
  • 02-06 11:07:04 4575f98 Introduce DynamicConfigurationProvider
  • 02-06 11:07:04 4daf538 Add support for extra credential in JDBC driver
  • 02-06 11:07:04 50224b3 Add HiveGcsModule
  • 02-06 11:07:04 70f733f Rename HdfsConfigurationUpdater to HdfsConfigurationInitializer
  • 02-06 11:07:04 9a09af1 Add credentials in Identity and ConnectorIdentity
  • 02-06 11:07:04 de7b516 Add support for extra credential in Presto CLI
  • 02-06 11:07:04 fa02de3 Add header for extra credential

Grzegorz Kokosiński

  • all checked
  • 02-01 02:13:30 1d62598 Remove PlanNodeMatcher
  • 02-01 02:13:30 431e230 Static import DEFAULT_MATCHER
  • 02-01 02:13:30 4efe280 Add context parameter to custom pattern matching functions
  • 02-01 02:13:30 5d57936 Make Matcher to return the Stream of Match
  • 02-01 02:13:30 6375f4c Remove Matcher, move match to Pattern
  • 02-01 02:13:30 642330b Make Pattern#previous to be Optional
  • 02-01 02:13:30 68e22ec Add missing requireNonNull to matching related constructors
  • 02-01 02:13:30 70800a9 Remove unused variable
  • 02-01 02:13:30 87ed0f3 Remove value from Match
  • 02-01 02:13:30 e0d02ad Inline Patterns implementations from DefaultMatcher
  • 02-01 12:34:13 5d384b2 Add Elasticsearch connector
  • 02-04 13:38:30 b59e583 Make Git quiet when running tests in Travis
  • 02-05 01:32:25 34858db Handle S3 empty directories as directories

Karol Sobczak

  • all checked
  • 02-01 10:11:38 ca11249 Add physical* and internalNetwork* stats to QueryStatistics
  • 02-04 06:45:39 c3b113e Do not reset configuration properties to default values
  • 02-04 06:45:39 fce297a Use passed featuresConfig in LocalQueryRunner
  • 02-04 09:00:10 1a867a6 Expose plan node stats and cost in event listener
  • 02-04 09:00:10 552cf2a Document that some QueryStatistics field are not backward compatible
  • 02-05 02:22:38 c3675ee Fix Javadoc for QueryStatistics#planNodeStatsAndCosts
  • 02-06 04:41:11 0a9b804 Revoke memory after initial output page has been produced in tests
  • 02-06 04:41:11 1492b3f Return spill future directly
  • 02-06 04:41:11 5fef5aa Allow memory revoke only during operator finish phase
  • 02-06 04:41:11 79744f2 Convert revocable memory to user memory on hash aggregation finish
  • 02-06 04:41:11 e69b668 Produce more than single page in testHashAggregation

Martin Traverso

  • all checked
  • 02-04 11:37:33 d5dfc99 Preserve quotedness in QualifiedName
  • 02-04 16:46:08 7569071 Add approx_distinct for boolean
  • 02-04 21:10:28 890a6a0 Clarify handling of constant properties
  • 02-04 21:10:28 c9f8a23 Propagate partition properties for full outer join
  • 02-05 06:56:16 12c9b50 Support boolean type in EXPLAIN IO
  • 02-05 06:56:16 efc95d6 Rename classes to match coding style
  • 02-05 23:24:22 7567baf Add SphericalGeography type

Piotr Findeisen

  • all checked
  • 02-01 00:24:27 0c72574 Remove TODO related to stats caching
  • 02-01 00:24:27 363a3c3 Capture TaskManagerConfig on a field in LocalQueryRunner
  • 02-01 00:24:27 dc1d66f Add Exchange before GroupId to improve Partial Aggregation
  • 02-01 03:24:50 eddafb5 Fix merge conflict
  • 02-01 07:16:18 1116011 Rename VarcharType#getLengthSafe to #getBoundedLength
  • 02-01 07:16:18 8af7888 Remove deprecated VarcharType#getLength method
  • 02-01 07:16:58 27a1b0e Minor code fixes in AbstractTestParquetReader
  • 02-01 07:16:58 d7210b7 Fix information_schema.columns queries for Kafka, Redis
  • 02-02 13:10:02 ddf2421 Document note about LDAPS and JDK Upgrade
  • 02-04 05:57:10 a873894 Remove redundant usages of StringBuilder
  • 02-04 05:57:10 c290b17 Import String.join statically
  • 02-04 08:57:57 656be2c Make service principal hostname configurable in Kerberos auth
  • 02-04 12:12:52 157ad3a Fix inconsistencies in hostname configuration property for Kerberos auth
  • 02-05 06:14:47 1fa2cba Document RecordCursor#getReadTimeNanos
  • 02-05 06:14:47 c48c8b6 Remove incorrect getReadTimeNanos() implementations
  • 02-05 07:45:50 0ebbea6 Check overlapping on Domain directly in TupleDomainParquetPredicate
  • 02-05 07:45:50 1f4caa8 Make #getDomain return nonnull value
  • 02-05 07:45:50 7296700 Assume Domain from file cannot be isNone()
  • 02-05 07:45:50 7d2f5a9 Use known Presto Type instead of reconstructing
  • 02-05 07:45:50 8e3bf91 Remove redundant else
  • 02-05 07:45:50 972fdec Remove unused ParquetTypeUtils#getPrestoType method
@findepi

This comment has been minimized.

Copy link
Member

findepi commented Feb 1, 2019

General
---
* Improve query performance for certain queries involving ``ROLLUP``.

#105

@findepi

This comment has been minimized.

Copy link
Member

findepi commented Feb 1, 2019

Kafka
---
* Fix query failure when reading ``information_schema.columns`` without an equality condition on ``table_name``.

Redis
---
* Fix query failure when reading ``information_schema.columns`` without an equality condition on ``table_name``.

#120

@kokosing

This comment has been minimized.

Copy link
Member

kokosing commented Feb 4, 2019

Elasticsearch Connector
------
 * Add Elasticsearch Connector

#118

@findepi

This comment has been minimized.

Copy link
Member

findepi commented Feb 4, 2019

Security
---
* Make service principal hostname configurable in Kerberos authenticator.

#146 #153

@martint

This comment has been minimized.

Copy link
Member

martint commented Feb 4, 2019

* Preserve table name quoting in the output of ``SHOW CREATE VIEW``.
* Fix failure when preparing statements that contain a quoted reserved word as a table name.

#80

@dain

This comment has been minimized.

Copy link
Member

dain commented Feb 4, 2019

CLI
---
Add CLI option ``--trace-token`` to set the trace token to be sent with all queries.

#56

@martint

This comment has been minimized.

Copy link
Member

martint commented Feb 5, 2019

* Add support for ``boolean`` type to :func:`approx_distinct`.

#82

@dain

This comment has been minimized.

Copy link
Member

dain commented Feb 5, 2019

Fix backup of queries blocked waiting for minimum number of workers.

#155

@martint

This comment has been minimized.

Copy link
Member

martint commented Feb 5, 2019

* Add support for boolean columns to ``EXPLAIN`` with type ``IO``.

#157

@findepi

This comment has been minimized.

Copy link
Member

findepi commented Feb 5, 2019

Hive
---
* Fix Parquet predicate pushdown for ``smallint``, ``tinyint`` types.

#131, #123

@dain

This comment has been minimized.

Copy link
Member

dain commented Feb 6, 2019

General Changes
---------------

* Fix cluster starvation when wait for minimum number of workers is enabled. (:issue:`155`)

CLI Changes
-----------

* Add ``--trace-token`` option to set the trace token. (:issue:`117`)
* Add spilled data size debug output. (:issue:`161`)

Web UI Changes
--------------

* Add spilled data size to query details page. (:issue:`161`)

Hive Connector Changes
----------------------

* Reduce GC pressure from Parquet reader by constraining the maximum column read size. (:issue:`58`)

SPI Changes
-----------

* Include query peak task user memory in ``QueryCreatedEvent`` and ``QueryCompletedEvent``. (:issue:`163`)
@electrum

This comment has been minimized.

Copy link
Member Author

electrum commented Feb 6, 2019

Hive Connector Changes
----------------------

* Reduce network utilization and latency for S3 when reading ORC or Parquet. (:issue:`142`)
@martint

This comment has been minimized.

Copy link
Member

martint commented Feb 6, 2019

* Add SphericalGeography type and related functions.

#166

@kokosing

This comment has been minimized.

Copy link
Member

kokosing commented Feb 6, 2019

Hive Connector
-----------
* Allow to create table with `external_location` property pointing to an empty S3 directory that was created from AWS web console.

#75

@dain

This comment has been minimized.

Copy link
Member

dain commented Feb 6, 2019

General Changes
---------------

* Remove deprecated system pool. (:issue:`168`)
@electrum

This comment has been minimized.

Copy link
Member Author

electrum commented Feb 6, 2019

Security Changes
----------------

* Add support for client-provided extra credentials that can be utilized by connectors.

Hive Connector Changes
----------------------

* Add support for Google Cloud Storage (GCS). Credentials can be provided globally using
  the ``hive.gcs.json-key-file-path`` configuration property, or as a client-provided
  extra credential named ``hive.gcs`` if the ``hive.gcs.use-access-token`` is enabled.
@sopel39

This comment has been minimized.

Copy link
Member

sopel39 commented Feb 6, 2019

General Changes
---------------

* Fix memory revoking when aggregation operator is producing output pages.

SPI Changes
-----------

* Include physical and internal network input data size in ``QueryCompletedEvent``.
* Include plan node stats and cost in ``QueryCompletedEvent``.

@electrum electrum closed this Feb 7, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.