Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DocDB] Cleanup CatalogManager #19715

Open
3 of 9 tasks
hari90 opened this issue Oct 29, 2023 · 1 comment
Open
3 of 9 tasks

[DocDB] Cleanup CatalogManager #19715

hari90 opened this issue Oct 29, 2023 · 1 comment
Assignees
@hari90 @SrivastavaAnubhav
Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue

Comments

@hari90
Copy link
Contributor

hari90 commented Oct 29, 2023 •
edited by SrivastavaAnubhav
Loading

Jira Link: DB-8520

Description

CatalogManager has become the only master manager and it handles all master logic. The class has grown too big and has created high coupling between all yb components. There is no single ownership of this class making it hard to manager. The plan is to break up CatalogManager into multiple managers that are loosely coupled to each other.

Issue Type

kind/enhancement

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@hari90 hari90 added area/docdb YugabyteDB core features status/awaiting-triage Issue awaiting triage labels Oct 29, 2023
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue and removed status/awaiting-triage Issue awaiting triage labels Oct 29, 2023
@hari90
Copy link
Contributor Author

hari90 commented Nov 4, 2023

https://docs.google.com/document/d/15vWa9Br0cviPsWydsOarVLHTO3yfHjyVAS1ObhOv3h4/edit

SrivastavaAnubhav pushed a commit that referenced this issue Apr 29, 2024
[#21899][#19715] docdb: Move some heartbeat code out of catalog_manag…
584f3e3 
…er.cc

Summary:
This diff moves some heartbeat code out of catalog_manager.cc into `master_heartbeat_service.cc`, which is the entry point for the `TsHeartbeat` RPC. This makes it easier to follow the main heartbeat code and makes CatalogManager smaller.
Jira: DB-10798, DB-8520

Test Plan: Existing tests

Reviewers: zdrudi, hsunder

Reviewed By: zdrudi

Subscribers: ybase, bogdan

Differential Revision: https://phorge.dev.yugabyte.com/D33966
SrivastavaAnubhav pushed a commit that referenced this issue Apr 29, 2024
[#21899][#19715] Move more heartbeat code out of catalog_manager.cc
df28756 
Summary:
This diff is a follow-up to D33966 which moves more functions which are only used on the
heartbeat path out of `catalog_manager.cc` and into `master_heartbeat_service.cc`. There are no
functional changes introduced in this diff.

Test Plan: Existing tests

Reviewers: zdrudi, hsunder

Reviewed By: zdrudi

Subscribers: ybase

Differential Revision: https://phorge.dev.yugabyte.com/D34568
SrivastavaAnubhav pushed a commit that referenced this issue May 3, 2024
[#21899][#19715] Remove GetReportedAndOrphanedTablets from catalog_ma…
ea5b325 
…nager.cc

Summary:
D33966 moved much of the heartbeat path code to `master_heartbeat_service.cc`. The code for
processing updated tablets to the list of orphaned tablets was not moved because it required the
catalog manager mutex to be held for its duration. This diff refactors that function
(`GetReportedAndOrphanedTabletsFromReport`) to use batch
APIs to get the table / tablet info objects, and then do the remaining processing not under the
catalog manager mutex.

Test Plan: existing tests

Reviewers: zdrudi, hsunder

Reviewed By: zdrudi

Subscribers: ybase

Differential Revision: https://phorge.dev.yugabyte.com/D34569
druzac added a commit that referenced this issue May 9, 2024
@druzac
[#19715] docdb: Move some MasterCluster RPC endpoints out of the cata…
87aa2e5 
…log manager.

Summary:
This diff introduces a new class, MasterClusterHandler, to contain some RPC endpoint methods formerly in the CatalogManager.
Jira: DB-8520

Test Plan: existing tests

Reviewers: asrivastava

Reviewed By: asrivastava

Subscribers: ybase, slingam

Differential Revision: https://phorge.dev.yugabyte.com/D34749
svarnau pushed a commit that referenced this issue May 25, 2024
[#21899][#19715] docdb: Move some heartbeat code out of catalog_manag…
7fa54b7 
…er.cc

Summary:
This diff moves some heartbeat code out of catalog_manager.cc into `master_heartbeat_service.cc`, which is the entry point for the `TsHeartbeat` RPC. This makes it easier to follow the main heartbeat code and makes CatalogManager smaller.
Jira: DB-10798, DB-8520

Test Plan: Existing tests

Reviewers: zdrudi, hsunder

Reviewed By: zdrudi

Subscribers: ybase, bogdan

Differential Revision: https://phorge.dev.yugabyte.com/D33966
svarnau pushed a commit that referenced this issue May 25, 2024
[#21899][#19715] Move more heartbeat code out of catalog_manager.cc
46c4f9f 
Summary:
This diff is a follow-up to D33966 which moves more functions which are only used on the
heartbeat path out of `catalog_manager.cc` and into `master_heartbeat_service.cc`. There are no
functional changes introduced in this diff.

Test Plan: Existing tests

Reviewers: zdrudi, hsunder

Reviewed By: zdrudi

Subscribers: ybase

Differential Revision: https://phorge.dev.yugabyte.com/D34568
svarnau pushed a commit that referenced this issue May 25, 2024
[#21899][#19715] Remove GetReportedAndOrphanedTablets from catalog_ma…
4e60c93 
…nager.cc

Summary:
D33966 moved much of the heartbeat path code to `master_heartbeat_service.cc`. The code for
processing updated tablets to the list of orphaned tablets was not moved because it required the
catalog manager mutex to be held for its duration. This diff refactors that function
(`GetReportedAndOrphanedTabletsFromReport`) to use batch
APIs to get the table / tablet info objects, and then do the remaining processing not under the
catalog manager mutex.

Test Plan: existing tests

Reviewers: zdrudi, hsunder

Reviewed By: zdrudi

Subscribers: ybase

Differential Revision: https://phorge.dev.yugabyte.com/D34569
svarnau pushed a commit that referenced this issue May 25, 2024
@druzac
[#19715] docdb: Move some MasterCluster RPC endpoints out of the cata…
4bbbf3a 
…log manager.

Summary:
This diff introduces a new class, MasterClusterHandler, to contain some RPC endpoint methods formerly in the CatalogManager.
Jira: DB-8520

Test Plan: existing tests

Reviewers: asrivastava

Reviewed By: asrivastava

Subscribers: ybase, slingam

Differential Revision: https://phorge.dev.yugabyte.com/D34749
SrivastavaAnubhav pushed a commit that referenced this issue Jun 2, 2024
@SrivastavaAnubhav
[#22603][#19715] docdb: Move some tablet splitting code out of catalo…
246cd73 
…g manager

Summary:
This diff moves some RPCs and functions which are specific to tablet splitting out of `catalog_manager.cc` and into `tablet_split_manager.cc`.

It also removes the 3 interfaces we have which are all satisfied by the CatalogManager class right now.
Jira: DB-11513, DB-8520

Test Plan: existing tests

Reviewers: hsunder

Reviewed By: hsunder

Subscribers: ybase, asrivastava

Differential Revision: https://phorge.dev.yugabyte.com/D34764
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hari90 hari90

@SrivastavaAnubhav SrivastavaAnubhav

Labels
area/docdb YugabyteDB core features kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@hari90 @yugabyte-ci @SrivastavaAnubhav