Skip to content
  • v0.8.1
  • a2fa76d
  • Compare
    Choose a tag to compare
    Search for a tag
  • v0.8.1
  • a2fa76d
  • Compare
    Choose a tag to compare
    Search for a tag

@ravisharda ravisharda released this Jan 14, 2021 · 115 commits to master since this release

This release contains several bug fixes and performance improvements over 0.8.0:

Segment Store

  • Issue #5294: Bookkeeper Client may cache old DNS entries for too long
  • Issue #5269: Fixed a deadlock in StreamSegmentReadIndex

Controller

  • Issue #5341: Fix thread leak in the Controller
  • Issue #5416: Transaction commit order should take 'limit' elements from an ordered list

Client

  • Issue #5137: Remove dependency on netty-transport-native-epoll
  • Issue #5414: Ensure getClientConnection throws an exception incase the connection pool is closed

General

  • Issue #4775: Add snapshot stage for all release branches

To see all issues resolved in this release, refer to the attached commits.txt file.

Assets 33
Pre-release

@ravisharda ravisharda released this Jan 14, 2021 · 115 commits to master since this release

This release contains several bug fixes and performance improvements over 0.8.0:

Segment Store

  • Issue #5294: Bookkeeper Client may cache old DNS entries for too long
  • Issue #5269: Fixed a deadlock in StreamSegmentReadIndex

Controller

  • Issue #5341: Fix thread leak in the Controller
  • Issue #5416: Transaction commit order should take 'limit' elements from an ordered list

Client

  • Issue #5137: Remove dependency on netty-transport-native-epoll
  • Issue #5414: Ensure getClientConnection throws an exception incase the connection pool is closed

General

  • Issue #4775: Add snapshot stage for all release branches

To see all issues resolved in this release, refer to the attached commits.txt file.

Assets 33
Pre-release

@ravisharda ravisharda released this Dec 17, 2020 · 115 commits to master since this release

This release contains several bug fixes and performance improvements over 0.8.0:

Segment Store

  • Issue #5294: Bookkeeper Client may cache old DNS entries for too long
  • Issue #5269: Fixed a deadlock in StreamSegmentReadIndex

Controller

  • Issue #5341: Fix thread leak in the Controller
  • Issue #5416: Transaction commit order should take 'limit' elements from an ordered list

Client

  • Issue #5137: Remove dependency on netty-transport-native-epoll
  • Issue #5414: Ensure getClientConnection throws an exception incase the connection pool is closed

General

  • Issue #4775: Add snapshot stage for all release branches

To see all issues resolved in this release, refer to the attached commits.txt file.

Assets 33
Pre-release

@ravisharda ravisharda released this Dec 8, 2020 · 2 commits to r0.8 since this release

This release contains several bug fixes and performance improvements over 0.8.0:

Segment Store

  • Issue #5294: Bookkeeper Client may cache old DNS entries for too long
  • Issue #5269: Fixed a deadlock in StreamSegmentReadIndex

Controller

  • Issue #5341: Fix thread leak in controller

Client

  • Issue #5137: Remove dependency on netty-transport-native-epoll

General

  • Issue #4775: Add snapshot stage for all release branches

To see all issues resolved in this release, refer to the attached commits.txt file.

Assets 33
  • v0.7.3
  • 5d7e038
  • Compare
    Choose a tag to compare
    Search for a tag
  • v0.7.3
  • 5d7e038
  • Compare
    Choose a tag to compare
    Search for a tag

@andreipaduroiu andreipaduroiu released this Dec 9, 2020

This release contains several bug fixes and performance improvements over 0.7.2:

Segment Store

  • Issue #5294: Bookkeeper Client may cache old DNS entries for too long.
  • Issue #5269: (SegmentStore) Fixed a deadlock in StreamSegmentReadIndex.
  • Issue #5119: (SegmentStore) Copy-on-Read for Table Segment Compaction.
  • Issue #5155: (SegmentStore) Enabling copy-on-read for all Segment Reads.
  • Issue #5154: (SegmentStore) Fixed a stats concurrency bug in the Read Index.
  • Issue #5207: (SegmentStore) Read Index Bug Fixes.

Controller

  • Issue #4757: Remove stream reference from watermark bucket upon stream deletion.
  • Issue #5341: Fix thread leak in controller.
  • Issue #4915: Add timeout to all calls to segment store from controller.
Assets 32
Pre-release

@andreipaduroiu andreipaduroiu released this Dec 7, 2020

This release contains several bug fix and performance improvements over 0.7.2:

Segment Store

  • Issue #5294: Bookkeeper Client may cache old DNS entries for too long.
  • Issue #5269: (SegmentStore) Fixed a deadlock in StreamSegmentReadIndex.
  • Issue #5119: (SegmentStore) Copy-on-Read for Table Segment Compaction
  • Issue #5155: (SegmentStore) Enabling copy-on-read for all Segment Reads
  • Issue #5154: (SegmentStore) Fixed a stats concurrency bug in the Read Index (#5170)
  • Issue #5207: (SegmentStore) Read Index Bug Fixes

Controller

  • Issue #4757: Remove stream reference from watermark bucket upon stream deletion.
  • Issue #5341: Fix thread leak in controller.
  • Issue #4915: Add timeout to all calls to segment store from controller.

commits.txt

Assets 33
  • v0.8.0
  • e4c436b
  • Compare
    Choose a tag to compare
    Search for a tag
  • v0.8.0
  • e4c436b
  • Compare
    Choose a tag to compare
    Search for a tag

@ravisharda ravisharda released this Sep 29, 2020 · 115 commits to master since this release

This release contains exciting new features and significant enhancements over the previous major release line (0.7.x). This page provides an overview of the major ones.

Key-Value Tables

In 0.8, we have introduced an entirely new primitive that enables efficient insertions, updates and lookups of arbitrary key-value pairs. Using the same underlying infrastructure as Streams, Key-Value Tables provide the same features one would expect of a typical key-value store, plus some more.

Here are a few highlights:

  • Distributed Key-Value Store is built on top of append-only Pravega Segments.
  • Keys are sharded uniformly across underlying Segments or can be grouped by means of Key Families.
  • Conditional or unconditional updates and removals.
  • Single or multiple entry batch (atomic) updates or removals.
  • Single or multiple entry retrieval.
  • Listing of keys/entries in bitwise lexicographical order (by key).
  • Java Map wrapper provided for easy integration with existing applications.

Key-Value Tables are built on top of Table Segments - the same underlying primitive that Pravega has used to store its own Stream, Segment and Transaction metadata since release 0.5. The Pravega Controller and Segment Store rely on the consistency, durability and atomicity guarantees of Table Segments for all of their metadata needs - the same guarantees are also extended to Key-Value Tables as well.

For more details, please refer to PDP-39 - Key Value Tables.

As of 0.8, Key-Value Tables are released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

Schema Registry

With Pravega 0.8, we are also releasing Pravega Schema Registry 0.1. The Schema Registry service is designed to store and manage schemas for unstructured data stored in Pravega streams.

Here are a few highlights:

  • The service provides a RESTful interface to store and manage schemas under schema groups.
  • Users can choose from different compatibility policies for safely evolving their schema.
  • The service has built-in support for popular serialization formats in Avro, Profobuf and JSON schemas. However, users can store schemas of other formats too.
  • In v0.1 we have compatibility policies enabled only for Avro. Subsequent releases will support compatibility checks for Protobuf and JSON.
  • The service stores and manages encoding information in form of codec information. Codecs could correspond to different compression or encryption used while encoding the serialized data at rest. The service generates unique identifiers for schemas and codec information pairs that users may use to tag their data.

For more details, visit the Schema Registry repository.

As of 0.1, Schema Registry is released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

Performance Improvements and Optimizations

Pravega 0.8 contains significant improvements over the previous release. These are the main improvements included in 0.8.

  • Issue #4774: (Client) Saves significant compute cycles for a reader by minimizing and optimizing costly calls for managing its internal state.
  • Issue #4653: (Segment Store) Control the number of container starts/recoveries performed in parallel.
  • Issue #4294: Remove Netty from the client.

Configuration Names

Pravega 0.8 changes the names of almost all configuration items, based on a new uniform configuration naming scheme.

For instance the property controller.minRebalanceIntervalSeconds is replaced by controller.rebalance.interval.seconds.min and pravegaservice.storageImplementation is replaced by pravegaservice.storage.impl.name. See this Wiki page for new names of old properties.

Old properties are deprecated but will continue to work for now. So, upon upgrade from the previous version, your Pravega deployment will continue to work. You can use a combination of new and old properties to Configure Pravega components, although it is highly discouraged. If you specify a property using both new and old names, the old property will take precedence.

Fresh installations must use new properties (even though old properties will work too for now). For upgrade from previous versions, we highly encourage migrating to new properties at the earliest opportunity.

See issue #4712 for more details.

Other New Features and Updates

  • Issue #4893: (Controller, Metrics) Adds metrics for created and deleted scopes.
  • Issue #4910: (Segment Store, Metrics) Adds two new metrics to report the size distribution of appends and reads.
  • Issue #4680: (Client) Add a new truncation api to ByteStreamWriter.
  • Issue #4791: (Client, Controller) Add a new controller API to fetch segments for a given epoch
  • Issue #4845: (Client, Usability) StreamInfo obtained via io.pravega.client.admin.impl.StreamManager indicates if the Stream is sealed.
  • Issue #4846: (Library upgrades) Upgrade Netty to 4.1.50Final.
  • Issue #4883: (Library upgrades) Upgrade gRPC to 1.24.2.
  • Issue #4531: (Library upgrades) Upgrade Bookeeper client to 4.9.2.

Bug Fixes

  • Issue #4631: (Controller, Metrics) Report failed transaction commit metric only in case of non-retryable failures.
  • Issue #4847: (Client) Improve sealed stream handling of EventStreamWriter and ByteStreamWriter.

For a comprehensive listing of issues resolved in this release, refer to the attached commits.txt file.

Ecosystem Updates/Links

Assets 33
Pre-release

@ravisharda ravisharda released this Sep 25, 2020 · 115 commits to master since this release

This release contains exciting new features and significant enhancements over the previous major release line (0.7.x). This page provides an overview of the major ones.

Key-Value Tables

In 0.8, we have introduced an entirely new primitive that enables efficient insertions, updates and lookups of arbitrary key-value pairs. Using the same underlying infrastructure as Streams, Key-Value Tables provide the same features one would expect of a typical key-value store, plus some more.

Here are a few highlights:

  • Distributed Key-Value Store is built on top of append-only Pravega Segments.
  • Keys are sharded uniformly across underlying Segments or can be grouped by means of Key Families.
  • Conditional or unconditional updates and removals.
  • Single or multiple entry batch (atomic) updates or removals.
  • Single or multiple entry retrieval.
  • Listing of keys/entries in bitwise lexicographical order (by key).
  • Java Map wrapper provided for easy integration with existing applications.

Key-Value Tables are built on top of Table Segments - the same underlying primitive that Pravega has used to store its own Stream, Segment and Transaction metadata since release 0.5. The Pravega Controller and Segment Store rely on the consistency, durability and atomicity guarantees of Table Segments for all of their metadata needs - the same guarantees are also extended to Key-Value Tables as well.

For more details, please refer to PDP-39 - Key Value Tables.

As of 0.8, Key-Value Tables are released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

Schema Registry

With Pravega 0.8, we are also releasing Pravega Schema Registry 0.1. The Schema Registry service is designed to store and manage schemas for unstructured data stored in Pravega streams.

Here are a few highlights:

  • The service provides a RESTful interface to store and manage schemas under schema groups.
  • Users can choose from different compatibility policies for safely evolving their schema.
  • The service has built-in support for popular serialization formats in Avro, Profobuf and JSON schemas. However, users can store schemas of other formats too.
  • In v0.1 we have compatibility policies enabled only for Avro. Subsequent releases will support compatibility checks for Protobuf and JSON.
  • The service stores and manages encoding information in form of codec information. Codecs could correspond to different compression or encryption used while encoding the serialized data at rest. The service generates unique identifiers for schemas and codec information pairs that users may use to tag their data.

For more details, visit the Schema Registry repository.

As of 0.1, Schema Registry is released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

Performance Improvements and Optimizations

Pravega 0.8 contains significant improvements over the previous release. These are the main improvements included in 0.8.

  • Issue #4774: (Client) Saves significant compute cycles for a reader by minimizing and optimizing costly calls for managing its internal state.
  • Issue #4653: (Segment Store) Control the number of container starts/recoveries performed in parallel.
  • Issue #4294: Remove Netty from the client.

Configuration Names

Pravega 0.8 changes the names of almost all configuration items, based on a new uniform configuration naming scheme.

For instance the property controller.minRebalanceIntervalSeconds is replaced by controller.rebalance.interval.seconds.min and pravegaservice.storageImplementation is replaced by pravegaservice.storage.impl.name. See this Wiki page for new names of old properties.

Old properties are deprecated but will continue to work for now. So, upon upgrade from previous version your Pravega deployment will continue to work. You can use a combination of new and old properties to Configure Pravega components, although it is highly discouraged. If you specify a property using both new and old names, the old property will take precedence.

Fresh installations must use new properties (even though old properties will work too for now). For upgrade from previous versions, we highly encourage migrating to new properties at the earliest opportunity.

See issue #4712 for more details.

Other New Features and Updates

  • Issue #4893: (Controller, Metrics) Adds metrics for created and deleted scopes.
  • Issue #4910: (Segment Store, Metrics) Adds two new metrics to report the size distribution of appends and reads.
  • Issue #4680: (Client) Add a new truncation api to ByteStreamWriter.
  • Issue #4791: (Client, Controller) Add a new controller API to fetch segments for a given epoch
  • Issue #4845: (Client, Usability) StreamInfo obtained via io.pravega.client.admin.impl.StreamManager indicates if the Stream is sealed.
  • Issue #4846: (Library upgrades) Upgrade Netty to 4.1.50Final.
  • Issue #4883: (Library upgrades) Upgrade gRPC to 1.24.2.
  • Issue #4531: (Library upgrades) Upgrade Bookeeper client to 4.9.2.

Bug Fixes

  • Issue #4631: (Controller, Metrics) Report failed transaction commit metric only in case of non-retryable failures.
  • Issue #4847: (Client) Improve sealed stream handling of EventStreamWriter and ByteStreamWriter.

For a comprehensive listing of issues resolved in this release, refer to the attached commits.txt file.

Ecosystem Updates/Links

Assets 33
Pre-release

@ravisharda ravisharda released this Sep 18, 2020 · 115 commits to master since this release

This release contains exciting new features and significant enhancements over the previous major release line (0.7.x). This page provides an overview of the major ones.

Key-Value Tables

In 0.8, we have introduced an entirely new primitive that enables efficient insertions, updates and lookups of arbitrary key-value pairs. Using the same underlying infrastructure as Streams, Key-Value Tables provide the same features one would expect of a typical key-value store, plus some more.

Here are a few highlights:

  • Distributed Key-Value Store is built on top of append-only Pravega Segments.
  • Keys are sharded uniformly across underlying Segments or can be grouped by means of Key Families.
  • Conditional or unconditional updates and removals.
  • Single or multiple entry batch (atomic) updates or removals.
  • Single or multiple entry retrieval.
  • Listing of keys/entries in bitwise lexicographical order (by key).
  • Java Map wrapper provided for easy integration with existing applications.

Key-Value Tables are built on top of Table Segments - the same underlying primitive that Pravega has used to store its own Stream, Segment and Transaction metadata since release 0.5. The Pravega Controller and Segment Store rely on the consistency, durability and atomicity guarantees of Table Segments for all of their metadata needs - the same guarantees are also extended to Key-Value Tables as well.

For more details, please refer to PDP-39 - Key Value Tables.

As of 0.8, Key-Value Tables are released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

Schema Registry

With Pravega 0.8, we are also releasing Pravega Schema Registry 0.1. The Schema Registry service is designed to store and manage schemas for unstructured data stored in Pravega streams.

Here are a few highlights:

  • The service provides a RESTful interface to store and manage schemas under schema groups.
  • Users can choose from different compatibility policies for safely evolving their schema.
  • The service has built-in support for popular serialization formats in Avro, Profobuf and JSON schemas. However, users can store schemas of other formats too.
  • In v0.1 we have compatibility policies enabled only for Avro. Subsequent releases will support compatibility checks for Protobuf and JSON.
  • The service stores and manages encoding information in form of codec information. Codecs could correspond to different compression or encryption used while encoding the serialized data at rest. The service generates unique identifiers for schemas and codec information pairs that users may use to tag their data.

For more details, visit the Schema Registry repository.

As of 0.1, Schema Registry is released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

Performance Improvements and Optimizations

Pravega 0.8 contains significant improvements over the previous release. These are the main improvements included in 0.8.

  • Issue #4774: (Client) Saves significant compute cycles for a reader by minimizing and optimizing costly calls for managing its internal state.
  • Issue #4653: (Segment Store) Control the number of container starts/recoveries performed in parallel.
  • Issue #4294: Remove Netty from the client.

Configuration Names

Pravega 0.8 changes the names of almost all configuration items, based on a new uniform configuration naming scheme.

For instance the property controller.minRebalanceIntervalSeconds is replaced by controller.rebalance.interval.seconds.min and pravegaservice.storageImplementation is replaced by pravegaservice.storage.impl.name. See this Wiki page for new names of old properties.

Old properties are deprecated but will continue to work for now. So, upon upgrade from previous version your Pravega deployment will continue to work. You can use a combination of new and old properties to Configure Pravega components, although it is highly discouraged. If you specify a property using both new and old names, the old property will take precedence.

Fresh installations must use new properties (even though old properties will work too for now). For upgrade from previous versions, we highly encourage migrating to new properties at the earliest opportunity.

See issue #4712 for more details.

Other New Features and Updates

  • Issue #4893: (Controller, Metrics) Adds metrics for created and deleted scopes.
  • Issue #4910: (Segment Store, Metrics) Adds two new metrics to report the size distribution of appends and reads.
  • Issue #4680: (Client) Add a new truncation api to ByteStreamWriter.
  • Issue #4791: (Client, Controller) Add a new controller API to fetch segments for a given epoch
  • Issue #4845: (Client, Usability) StreamInfo obtained via io.pravega.client.admin.impl.StreamManager indicates if the Stream is sealed.
  • Issue #4846: (Library upgrades) Upgrade Netty to 4.1.50Final.
  • Issue #4883: (Library upgrades) Upgrade gRPC to 1.24.2.
  • Issue #4531: (Library upgrades) Upgrade Bookeeper client to 4.9.2.

Bug Fixes

  • Issue #4631: (Controller, Metrics) Report failed transaction commit metric only in case of non-retryable failures.
  • Issue #4847: (Client) Improve sealed stream handling of EventStreamWriter and ByteStreamWriter.

For a comprehensive listing of issues resolved in this release, refer to the attached commits.txt file.

Ecosystem Updates/Links

Assets 33
Pre-release

@ravisharda ravisharda released this Sep 11, 2020 · 15 commits to r0.8 since this release

This release contains exciting new features and significant enhancements over the previous major release line (0.7.x). This page provides an overview of the major ones.

Key-Value Tables

In 0.8, we have introduced an entirely new primitive that enables efficient insertions, updates and lookups of arbitrary key-value pairs. Using the same underlying infrastructure as Streams, Key-Value Tables provide the same features one would expect of a typical key-value store, plus some more.

Here are a few highlights:

  • Distributed Key-Value Store is built on top of append-only Pravega Segments.
  • Keys are sharded uniformly across underlying Segments or can be grouped by means of Key Families.
  • Conditional or unconditional updates and removals.
  • Single or multiple entry batch (atomic) updates or removals.
  • Single or multiple entry retrieval.
  • Listing of keys/entries in bitwise lexicographical order (by key).
  • Java Map wrapper provided for easy integration with existing applications.

Key-Value Tables are built on top of Table Segments - the same underlying primitive that Pravega has used to store its own Stream, Segment and Transaction metadata since release 0.5. The Pravega Controller and Segment Store rely on the consistency, durability and atomicity guarantees of Table Segments for all of their metadata needs - the same guarantees are also extended to Key-Value Tables as well.

For more details, please refer to PDP-39 - Key Value Tables.

As of 0.8, Key-Value Tables are released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

Schema Registry

With Pravega 0.8, we are also releasing Pravega Schema Registry 0.1. The Schema Registry service is designed to store and manage schemas for unstructured data stored in Pravega streams.

Here are a few highlights:

  • The service provides a RESTful interface to store and manage schemas under schema groups.
  • Users can choose from different compatibility policies for safely evolving their schema.
  • The service has built-in support for popular serialization formats in Avro, Profobuf and JSON schemas. However, users can store schemas of other formats too.
  • In v0.1 we have compatibility policies enabled only for Avro. Subsequent releases will support compatibility checks for Protobuf and JSON.
  • The service stores and manages encoding information in form of codec information. Codecs could correspond to different compression or encryption used while encoding the serialized data at rest. The service generates unique identifiers for schemas and codec information pairs that users may use to tag their data.

For more details, visit the Schema Registry repository.

As of 0.1, Schema Registry is released as Experimental. While fully supported, we will continue to make improvements over the upcoming releases and it is possible that we refine some of the APIs too.

Performance Improvements and Optimizations

Pravega 0.8 contains significant improvements over the previous release. These are the main improvements included in 0.8.

  • Issue #4774: (Client) Saves significant compute cycles for a reader by minimizing and optimizing costly calls for managing its internal state.
  • Issue #4653: (Segment Store) Control the number of container starts/recoveries performed in parallel.
  • Issue #4294: Remove Netty from the client.

Configuration Names

Pravega 0.8 changes the names of almost all configuration items, based on a new uniform configuration naming scheme.

For instance the property controller.minRebalanceIntervalSeconds is replaced by controller.rebalance.interval.seconds.min and pravegaservice.storageImplementation is replaced by pravegaservice.storage.impl.name. See this Wiki page for new names of old properties.

Old properties are deprecated but will continue to work for now. So, upon upgrade from previous version your Pravega deployment will continue to work. You can use a combination of new and old properties to Configure Pravega components, although it is highly discouraged. If you specify a property using both new and old names, the old property will take precedence.

Fresh installations must use new properties (even though old properties will work too for now). For upgrade from previous versions, we highly encourage migrating to new properties at the earliest opportunity.

See issue #4712 for more details.

Other New Features and Updates

  • Issue #4893: (Controller, Metrics) Adds metrics for created and deleted scopes.
  • Issue #4910: (Segment Store, Metrics) Adds two new metrics to report the size distribution of appends and reads.
  • Issue #4680: (Client) Add a new truncation api to ByteStreamWriter.
  • Issue #4791: (Client, Controller) Add a new controller API to fetch segments for a given epoch
  • Issue #4845: (Client, Usability) StreamInfo obtained via io.pravega.client.admin.impl.StreamManager indicates if the Stream is sealed.
  • Issue #4846: (Library upgrades) Upgrade Netty to 4.1.50Final.
  • Issue #4883: (Library upgrades) Upgrade gRPC to 1.24.2.
  • Issue #4531: (Library upgrades) Upgrade Bookeeper client to 4.9.2.

Bug Fixes

  • Issue #4631: (Controller, Metrics) Report failed transaction commit metric only in case of non-retryable failures.
  • Issue #4847: (Client) Improve sealed stream handling of EventStreamWriter and ByteStreamWriter.

For a comprehensive listing of issues resolved in this release, refer to the attached commits.txt file.

Ecosystem Updates/Links

Assets 33
You can’t perform that action at this time.