-
Notifications
You must be signed in to change notification settings - Fork 3.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HBASE-28313 StorefileRefresherChore should not refresh readonly table #5641
Conversation
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
I think the refresher is only for refreshing secondary replicas? So we could just change the test to check whether the region is a primary region? If so we just skip. WDYT? |
Related to this issue, there are two independent options that can be combined for configuration: whether readonly mode is enabled and whether replicas are enabled. There are four possible combinations:
There are two implementation approaches:
The key difference between these approaches is when the table is in readonly mode and replicas are enabled, whether the Chore needs to be triggered. If the Chore needs to be triggered, even after the table has been set to readonly, it will continue checking replica regions. If the Chore interval is set too short, it may result in a significant refresh of HDFS audit logs. |
The key point for whether we need to refresh the store files is that, the store file can be changed outside our control. This could only happen when we are a secondary replica, as all writes will go to the primary replica and the primary replica will do flush and compact, which change the store files. And I got your point, that if the table itself is readonly, we can make sure that there will be no writes to the primary replica, so we do not need to refresh secondary replicas too. So I prefer we change the condition to check primary region first, and then check whether the table is in read only mode. In both cases, we skip the refreshing. And better also add more comments to describe the logic here. WDYT? Thanks. |
Thanks for the guidance, I agree with your point. |
…ch can accept writes and the readonly table
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
💔 -1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
🎊 +1 overall
This message was automatically generated. |
According to semantics, when StoreFile Refresher is turned on, only the area with replicas should be refreshed. Tables with read-only attributes will not be refreshed.
There are the following scenarios: Assume that some tables in the database have replica enabled, while others do not. If a table that does not have replica enabled needs to perform migration tasks - first enable readonly and then export the snapshot, StorefileRefresherChore will always refresh the table that does not have replica enabled but performs migration tasks. This is obviously illogical.
Therefore, StorefileRefresherChore needs to determine whether the region has a read-only copy, and refresh it if it has a read-only copy. And tables with read-only properties are not refreshed.