Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DocDB] Fix infinite log spew if the master loses leadership when it has a backfill rpc pending and the rpc happens to fail. #13595

Closed
amitanandaiyer opened this issue Aug 11, 2022 · 0 comments
Closed

[DocDB] Fix infinite log spew if the master loses leadership when it has a backfill rpc pending and the rpc happens to fail. #13595

amitanandaiyer opened this issue Aug 11, 2022 · 0 comments
Assignees
@amitanandaiyer
Labels
area/docdb YugabyteDB core features kind/bug This issue is a bug priority/medium Medium priority issue

Comments

@amitanandaiyer
Copy link
Contributor

amitanandaiyer commented Aug 11, 2022 •
edited by yugabyte-ci
Loading

Jira Link: DB-3170

Description

If a leader has loses it’s leadership and can’t write to the syscatalog, it fails to mark the index as failed.

In the failed-rpc code path,
https://github.com/yugabyte/yugabyte-db/blob/master/src/yb/master/backfill_index.cc#L1225

We should exit out of the first if {} if we can’t persist information, instead of proceeding further and trying to LaunchNextChunkOrDone() based off of outdated information.
@amitanandaiyer amitanandaiyer added area/docdb YugabyteDB core features status/awaiting-triage Issue awaiting triage labels Aug 11, 2022
@amitanandaiyer amitanandaiyer self-assigned this Aug 11, 2022
@yugabyte-ci yugabyte-ci added kind/bug This issue is a bug priority/medium Medium priority issue and removed status/awaiting-triage Issue awaiting triage labels Aug 11, 2022
amitanandaiyer added a commit that referenced this issue Aug 18, 2022
@amitanandaiyer
[#13595] Docdb: Plumb through any status failures in case the leader …
186f1b7 
…cannot persist information to syscatalog.

Summary:
If a leader has loses it’s leadership and can’t write to the syscatalog, it fails to mark the index as failed.

In the failed-rpc code path,
we should exit out if we can’t persist information, instead of proceeding further and trying to LaunchNextChunkOrDone() based off of outdated information.

Test Plan: ybd --cxx-test cassandra_cpp_driver-test

Reviewers: jason, asrivastava

Reviewed By: asrivastava

Subscribers: ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D18895
@bmatican bmatican closed this as completed Nov 4, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@amitanandaiyer amitanandaiyer

Labels
area/docdb YugabyteDB core features kind/bug This issue is a bug priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bmatican @yugabyte-ci @amitanandaiyer