HBASE-28149 move_servers_namespaces_rsgroup is not changing the new r… #5661
Conversation
…s group in namespace description
|
🎊 +1 overall
This message was automatically generated. |
|
Actually hbase:rsgroup table should be single source of truth for all the rsgroup information. Currently namespace specific rsgroup information stored in namespace descriptor which can be also can be stored in hbase:rsgroup table. |
|
🎊 +1 overall
This message was automatically generated. |
|
💔 -1 overall
This message was automatically generated. |
IIRC we intentionally changed the rs group information for a table or a namespace to its descriptor in the past. Now the hbase:rsgroup table only stores the rsgroup <-> region server relationships. |
|
Got it @Apache9 from HBASE-22514. Moving the rsgroup to table configs which will reopen regions to move to the RS group servers automatically is good idea. |
|
There is a test case timeout which is not related to this. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
I'm not very familiar with the shell command here, so what does move_servers_namespaces_rsgroup actually do? |
@Apache9 These commands utilizing the rsadmin APIs provided in ruby to move existing tables in the namespaces to the dest RS group but not setting the new rsgroup name in the namenode descriptor.Without which any new tables created after running this command might go to old rsgroup servers which leads to inconsistent state. Since the core APIs not supporting the setting the rsgroup name in the namespace descriptor utilising the alter_namespace command in the above two commands. I would like to suggest simple admin API called moveNamespaceToRSGroup also along with existing RS group APIs so that setting configuration value for RS Group namespace in namespace descriptor can be taken care within the API level. |
This is exactly what I concern here... It is a bit strange that why we need to an extra step after calling an admin method, is this because we are calling an old API in RSGroupAdmin, not the method in Admin interface? I'm still a bit confused by 'moves tables in the namespaces and servers to rsgroup.'. Why we need to care about tables here? Just change the namespace descriptor and also move the servers is enough? |
After the update HBASE-23807, the old APIs in RSGroupAdmin are no longer being utilized in shell commands. The commands related to moving namespaces only move tables belonging to the namespaces to a new RS group without modifying the namespace descriptor itself. This behavior has remained same since the initial development of commands like move_namespaces_rsgroup and move_servers_namespaces_rsgroup (HBASE-19336). Having Admin API would be beneficial to control moving the namespace to new RS group programmatically.
If we just change the namespace descriptor and don't move the tables in the namespace then the existing tables belong to old RS group and what ever the tables created after that will be with new RS group so with in the the same namespace tables belongs to multiple rs groups in that case. If we are still ok to have such behavior then just changing the namespace descriptor is fine. |
|
This code in RSGroupUtil helps to get the rsgroup of the tables belongs to namespace but since first priority is given to checking at table descriptor and rsgroup info which gives old rsgroup first so if we don't move tables while moving namespace there could be possible tables within the namespace belongs to multiple groups.
|
OK, finally I understand the problem here. We are simulating the old behavior of move_namespaces_rsgroup and move_servers_namespaces_rsgroup, where there was no rs group information in namespace descriptor yet, so we just change all the rs group information for the tables in the namespace. I think in the old time, newly created tables will still be in the default rs group if you do not set it manually right? So in the HBase OP's view, usually if they want to change the rs group of a namespace, they will
Since now, we have namespace descriptor support, seems the simplest way is to just set the rs group information in namespace descriptor, and then reopen all the tables under the namespace. But there is another problem that, if we have rs group information in the table descriptor, it we just change the rs group information in namespace descriptor, it is not enough to move the table to the new rs group. So the problem is we add a new dimension here, there are new combinations of configurations which makes old shell commands not enough to cover all the possibilities. There are basically two ways, first is to change the old shell commands's behavior, like what we have done here, but not enough ,for me I think at least we should provide an option that whether we should override the rs group information in table descriptor. Another way is to keep the old shell commands as is, and introduce new shell commands. WDYT? I do not use rs group feature much, so I'm not sure which one is better. Thanks. |
|
@Apache9 Thanks for response.
With this options we can give control to user choose proper option based on the requirements. Will start working on it. We wanted to use RS Groups for managing multiple tenants data in same cluster currently a namespace belongs to a tenant so first option is best fit for us. |
@Apache9 wdyt? Will create separate JIRA for this and work on it if it's fine. |
created HBASE-28406 to support this. Can we merge this to support existing behaviour till then. |
…s group in namespace description