Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][ResourceTag] Support Resource Tag #6203

Merged
merged 5 commits into from
Sep 4, 2021

Conversation

morningman
Copy link
Contributor

@morningman morningman commented Jul 12, 2021

Proposed changes

Step 1 and 2 of #5902
This CL mainly changes:

  1. Support setting tags for BE nodes:

    alter system add backend "1272:9050, 1212:9050" properties("tag.location": "zoneA");
    alter system modify backend "1272:9050, 1212:9050" set ("tag.location": "zoneB");
    

    And for compatibility, all BE nodes will be set a "default" tag when upgrading: "tag.location": "default".

  2. Create a new class ReplicaAllocation to replace the previous replication_num.

    ReplicaAllocation represents the allocation of the replicas of a tablet. It contains a map from
    Tag to number of replicas.
    For example, if user set a table's replication num to 3, it will be converted to a ReplicaAllocation
    like: "tag.location.default" : "3", which means the tablet will have 3 replicas and all of them will be
    allocated in BE nodes with tag "default";

  3. Support create table with replication allocation:

    CREATE TABLE example_db.table_hash
    (
    k1 TINYINT
    )
    DISTRIBUTED BY HASH(k1) BUCKETS 32
    PROPERTIES (
        "replication_allocation"="tag.location.zone1:1, tag.location.zone2:2"
    );
    

    Also support set replica allocation for dynamic tables, and modify replica allocation at runtime.

    For compatibility, user can still set "replication_num" = "3", and it will be automatically converted to:
    "replication_allocation"="tag.location.default:3"

  4. Support tablet repair and balance based on Tag

    1. For tablets of non-colocate table, most of the logic is the same as before,
      but when selecting the destination node for clone, the tag of the node will be considered.
      If the required tag does not exist, it cannot be repaired.
      Similarly, under the condition of ensuring that the replicas are complete, the tablet will be
      reallocated according to the tag or the replicas will be balanced.

      Balancing is performed separately within each resource group.

    2. For tablets of colocate table, the backends sequence of buckets will be splitted by tag.
      For example, if replica allocation is "tag.location.zone1:1, tag.location.zone2:2",

      And zone1 has 2 BE: A, B; zone2 has 3 BE: C, D, F

      there will be 2 backend sequences: one is for zone1, and the other is for zone2.
      And one posible seqeunces will be:

      zone1: [A] [B] [A] [B]
      zone2: [C, D][D, F][F, C][C, D]

  5. Support setting tags for user and restrict execution node with tags:

    set property for 'cmy' 'resource_tags.location' : 'zone1, zone2';
    

    After setting, the user 'cmy' can only query data stored on backends with tag zone1 and zone2,
    And query can only be executed on backends with tag zone1 and zone2

    For compatibility, after upgrading, the property resource_tags.location will be empty,
    so that user can still query data stored on any backends.

  6. Modify the Unit test frame of FE so that we can created multi backends with different mocked IP in unit test.

    This help us to easily test some distributed cases like query, tablet repair and balance

The document will be added in another PR.

Also fix a bug described in #6194

Types of changes

What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)
  • Code refactor (Modify the code structure, format the code, etc...)
  • Optimization. Including functional usability improvements and performance improvements.
  • Dependency. Such as changes related to third-party components.
  • Other.

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • I have created an issue on (Fix [Proposal] Support resource division of BE nodes through Tags #5902) and described the bug/feature there in detail
  • Compiling and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works
  • If these changes need document changes, I have updated the document
  • Any dependent changes have been merged

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@morningman morningman added kind/feature Categorizes issue or PR as related to a new feature. kind/meta-version-change Categorizes issue or PR as related to changing meta version area/resource-tag labels Jul 12, 2021
@morningman morningman force-pushed the multi-tag-tablet-scheduler branch 2 times, most recently from 13d8ab0 to d61275b Compare July 19, 2021 03:01
@morningman morningman force-pushed the multi-tag-tablet-scheduler branch 2 times, most recently from b7b3fb1 to a767b76 Compare August 9, 2021 04:08
yangzhg
yangzhg previously approved these changes Aug 31, 2021
Copy link
Member

@yangzhg yangzhg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 31, 2021
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Sep 3, 2021
Copy link
Member

@yangzhg yangzhg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Sep 4, 2021

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Sep 4, 2021
@morningman morningman merged commit 7db8841 into apache:master Sep 4, 2021
@morningman morningman mentioned this pull request Oct 10, 2021
@tinkerrrr
Copy link
Contributor

Is there a way to check a BE's tags? @morningman

@morningman
Copy link
Contributor Author

Is there a way to check a BE's tags? @morningman

show backends

@tinkerrrr
Copy link
Contributor

Is there a way to check a BE's tags? @morningman

show backends

Oh I see. Another small question, if user1 is restricted to only use nodes in the group_a resource group for data query, what's the behavior when he create table without setting replication_allocation, is there any replica stored in BEs whose tag is not group_a?

@morningman
Copy link
Contributor Author

Is there a way to check a BE's tags? @morningman

show backends

Oh I see. Another small question, if user1 is restricted to only use nodes in the group_a resource group for data query, what's the behavior when he create table without setting replication_allocation, is there any replica stored in BEs whose tag is not group_a?

Yes. It will. Currently, the user property 'resource_tags.location will only be checked when querying.
So when creating table, replica will be allocated on BE which user specified.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. area/resource-tag kind/feature Categorizes issue or PR as related to a new feature. kind/meta-version-change Categorizes issue or PR as related to changing meta version reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Proposal] Support resource division of BE nodes through Tags
4 participants