Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement SPI for tablet balancers #1880

Closed
brianloss opened this issue Jan 25, 2021 · 1 comment · Fixed by #1891
Closed

Implement SPI for tablet balancers #1880

brianloss opened this issue Jan 25, 2021 · 1 comment · Fixed by #1891
Assignees
@brianloss
Labels
enhancement This issue describes a new feature, improvement, or optimization.
Milestone
2.1.0

Comments

@brianloss
Copy link
Member

We need an SPI for tablet balancers to isolate users from underlying thrift types and have a stable, public API for balancers. This change is driven by #1642 which will impact all existing tablet balancers in some form (either through deprecation or immediate removal). This SPI is intended to be the long-term replacement.

The methods provided on TabletBalancer do not need to change. Only the thrift/internal types included in the TabletBalancer method signatures (and existing balancer implementations) need to be replaced with public types exposing equivalent functionality.
@brianloss brianloss added the enhancement This issue describes a new feature, improvement, or optimization. label Jan 25, 2021
@brianloss brianloss added this to the 2.1 milestone Jan 25, 2021
@brianloss brianloss self-assigned this Jan 25, 2021
@milleruntime
Copy link
Contributor

This is a sub-task of #1659

This was referenced Jan 25, 2021
Closed
Merged
@brianloss brianloss mentioned this issue Feb 1, 2021
Merged
brianloss added a commit that referenced this issue Feb 4, 2021
@brianloss
Add TabletBalancer to SPI package (#1891)
95a8df1 
Add TabletBalancer to SPI package

This change deprecates existing balancers and introduces copies of each
under the SPI package. The main balancer interface, TabletBalancer, has
been converted from an abstract class to an interface and its method
signatures use only public API types and some other data types in SPI
sub-packages. The new types used expose the same functionality as the
previous types, and the implementations generally simply wrap the old
thrift objects and delegate, but will allow for easier evolution of the
API in the future.

* Deprecated TabletId.getTableId(), which created a new Text object
  every time, in favor of new TabletId.getTable() which returns a
  TableId. All existing usages of the getTableId() method would
  immediately call toString() on the returned Text object, or construct
  a TableId from the Text object. Therefore, the new method avoids
  object creation and usages read cleaner in many cases.
* Add static factory method on KeyExtent to create from a TabletId and
  return the wrapped KeyExtent if the TabletId is a TabletIdImpl and
  otherwise constructs with the property values retrieved from the
  TabletId.
* Add functionality to PluginEnvironment.Configuration to expose the
  derived property calculation (AccumuloConfiguration.Deriver) as a
  Supplier.
* Deprecated abstract TabletBalancer class now implements the new SPI
  TabletBalancer interface and provides implementations of the SPI
  methods on that interface that convert between the new and old types
  and delegate to the old balance/getAssignments methods.
* Deprecated all balancers in org.apache.accumulo.server.master.balancer
  except for ChaoticLoadBalancer, which was moved to accumulo-test since
  its purpose is only for testing.
* Introduce types in org.apache.accumulo.core.spi.balancer.data for
  wrapping thrift types needed by the balancers.
* Add a BalancerEnvironment to the SPI to support balancer-specific
  system operations (table online status check, listing online tablets,
  etc) in an API-stable manner.
* Convert TabletBalancer from abstract class to interface with init,
  getAssignments, and balance methods. The methods take parameters
  objects which will make it easier in the future if the balancer API
  needs to evolve.
* Implement existing balancers (DefaultLoadBalancer, TableLoadBalancer,
  GroupBalancer, RegexGroupBalancer, HostRegexTableLoadBalancer) in the
  SPI package by copying existing code and converting usages of thrift
  types to new stable types, and adjusting method signatures such that
  the new TabletBalancer API is implemented. The existing class
  DefaultLoadBalancer was renamed to SimpleLoadBalancer.
* Changed default balancer and table load balancer properties to use the
  new SPI-packaged versions.
* Update Master to construct the new TabletBalancer class, consolidate
  all access to the balancer internally (getAssignments moved from 
  )

fixes #1880
DomGarguilo pushed a commit to DomGarguilo/accumulo that referenced this issue Feb 10, 2021
@brianloss @DomGarguilo
Add TabletBalancer to SPI package (apache#1891)
84bcfc4 
Add TabletBalancer to SPI package

This change deprecates existing balancers and introduces copies of each
under the SPI package. The main balancer interface, TabletBalancer, has
been converted from an abstract class to an interface and its method
signatures use only public API types and some other data types in SPI
sub-packages. The new types used expose the same functionality as the
previous types, and the implementations generally simply wrap the old
thrift objects and delegate, but will allow for easier evolution of the
API in the future.

* Deprecated TabletId.getTableId(), which created a new Text object
  every time, in favor of new TabletId.getTable() which returns a
  TableId. All existing usages of the getTableId() method would
  immediately call toString() on the returned Text object, or construct
  a TableId from the Text object. Therefore, the new method avoids
  object creation and usages read cleaner in many cases.
* Add static factory method on KeyExtent to create from a TabletId and
  return the wrapped KeyExtent if the TabletId is a TabletIdImpl and
  otherwise constructs with the property values retrieved from the
  TabletId.
* Add functionality to PluginEnvironment.Configuration to expose the
  derived property calculation (AccumuloConfiguration.Deriver) as a
  Supplier.
* Deprecated abstract TabletBalancer class now implements the new SPI
  TabletBalancer interface and provides implementations of the SPI
  methods on that interface that convert between the new and old types
  and delegate to the old balance/getAssignments methods.
* Deprecated all balancers in org.apache.accumulo.server.master.balancer
  except for ChaoticLoadBalancer, which was moved to accumulo-test since
  its purpose is only for testing.
* Introduce types in org.apache.accumulo.core.spi.balancer.data for
  wrapping thrift types needed by the balancers.
* Add a BalancerEnvironment to the SPI to support balancer-specific
  system operations (table online status check, listing online tablets,
  etc) in an API-stable manner.
* Convert TabletBalancer from abstract class to interface with init,
  getAssignments, and balance methods. The methods take parameters
  objects which will make it easier in the future if the balancer API
  needs to evolve.
* Implement existing balancers (DefaultLoadBalancer, TableLoadBalancer,
  GroupBalancer, RegexGroupBalancer, HostRegexTableLoadBalancer) in the
  SPI package by copying existing code and converting usages of thrift
  types to new stable types, and adjusting method signatures such that
  the new TabletBalancer API is implemented. The existing class
  DefaultLoadBalancer was renamed to SimpleLoadBalancer.
* Changed default balancer and table load balancer properties to use the
  new SPI-packaged versions.
* Update Master to construct the new TabletBalancer class, consolidate
  all access to the balancer internally (getAssignments moved from 
  )

fixes apache#1880
DomGarguilo pushed a commit to DomGarguilo/accumulo that referenced this issue Feb 10, 2021
@brianloss @DomGarguilo
Warn in constructor when deprecated balancers are used re apache#1880
6330d0d
@ctubbsii ctubbsii added this to the 2.1.0 milestone Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@brianloss brianloss

Labels
enhancement This issue describes a new feature, improvement, or optimization.
Projects
None yet
Milestone
2.1.0
Development

Successfully merging a pull request may close this issue.

Add TabletBalancer to SPI package
3 participants
@ctubbsii @brianloss @milleruntime