Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix an issue that can’t enable automatic incremental update for a single Hive catalog. #46797

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[BugFix] Fix an issue that can’t enable automatic incremental update for a single Hive catalog. #46797

wants to merge 1 commit into from

Conversation

zhangheihei
Copy link
Contributor

@zhangheihei zhangheihei commented Jun 10, 2024 •
edited
Loading

Why I'm doing:

Fix an issue that can’t enable automatic incremental update for a single Hive catalog.

What I'm doing:

Fixes #46848

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.3
    • 3.2
    • 3.1
    • 3.0
    • 2.5

@zhangheihei zhangheihei requested a review from a team as a code owner June 10, 2024 04:05
@zhangheihei
Copy link
Contributor Author

hi @stephen-shelby
plz review the code

stephen-shelby
stephen-shelby reviewed Jun 11, 2024
View reviewed changes
fe/fe-core/src/main/java/com/starrocks/server/GlobalStateMgr.java
if (Config.enable_hms_events_incremental_sync) {
metastoreEventsProcessor.start();
}
metastoreEventsProcessor.start();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why you do this ? Please provide detailed reasons

Copy link
Contributor Author

@zhangheihei zhangheihei Jun 11, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stephen-shelby tks review.

1 set CATALOG PROPERTIES enable_hms_events_incremental_sync=true for one catalog
2 but fe.conf param enable_hms_events_incremental_sync=false
3 then can’t enable automatic incremental update for a single Hive catalog.
image

Copy link
Contributor

@stephen-shelby stephen-shelby Jun 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please put the issue link in pr body

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for your contribution. the incremental refreshing based on the hms event is deprecated. I recommand that you use https://docs.starrocks.io/docs/data_source/catalog/hive_catalog/#periodically-refresh-metadata-cache instread of incremental refreshing.

Copy link
Contributor Author

@zhangheihei zhangheihei Jun 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tks review @stephen-shelby
Should the Automatic incremental update doc removed?
The issue had added.

@zhangheihei
[BugFix] Fix an issue that can’t enable automatic incremental update …
31c78a2 
…for a single Hive catalog.

Signed-off-by: edwinhzhang <edwinhzhang@tencent.com>
@zhangheihei zhangheihei force-pushed the fix-automatic-incremental-behavior branch from 63f6792 to 31c78a2 Compare June 12, 2024 02:05
@sonarcloud SonarCloud
Copy link

sonarcloud bot commented Jun 12, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

@zhangheihei zhangheihei mentioned this pull request Jun 12, 2024
Open
@github-actions GitHub Actions
Copy link

[FE Incremental Coverage Report]

✅ pass : 3 / 3 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 com/starrocks/connector/hive/events/MetastoreEventsProcessor.java 2 2 100.00% []
🔵 com/starrocks/server/GlobalStateMgr.java 1 1 100.00% []

@github-actions GitHub Actions
Copy link

[BE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dirtysalt dirtysalt dirtysalt approved these changes

@stephen-shelby stephen-shelby Awaiting requested review from stephen-shelby

At least 2 approving reviews are required to merge this pull request.

Assignees

@zhangheihei zhangheihei

Labels
2.5 3.0 3.1 3.2 3.3 behavior_changed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Fix an issue that can’t enable automatic incremental update for a single Hive catalog
3 participants
@zhangheihei @dirtysalt @stephen-shelby