Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid creating command topic when backup file exists #7753

Closed
stevenpyzhang opened this issue Jun 30, 2021 · 0 comments · Fixed by #8257
Closed

Avoid creating command topic when backup file exists #7753

stevenpyzhang opened this issue Jun 30, 2021 · 0 comments · Fixed by #8257
Assignees
Labels
blocker core-infra Issues owned by the ksqlDB Core Infra team enhancement
Milestone

Comments

@stevenpyzhang
Copy link
Member

stevenpyzhang commented Jun 30, 2021

Is your feature request related to a problem? Please describe.
If the command topic is deleted when backups are enabled, and then the server is restarted, the command topic gets recreated even though a backup file exists. The server will most likely immediately enter a DEGRADED state due to the backup not matching the newly created command topic

Describe the solution you'd like
As part of startup, we could avoid creating the command topic if the backup file is enabled and the server detects a backup file exists.

Additional context
In KsqlRestApplication

  private void registerCommandTopic() {

    final String commandTopic = commandStore.getCommandTopicName();

    KsqlInternalTopicUtils.ensureTopic(
        commandTopic,
        ksqlConfigNoPort,
        serviceContext.getTopicClient()
    );
...

Ensure topic doesn't check to see if backups are enabled or if a backup file exists already, so it'll just always create a command topic when the server starts up and the broker doesn't have a command topic.

@stevenpyzhang stevenpyzhang added enhancement needs-triage core-infra Issues owned by the ksqlDB Core Infra team labels Jun 30, 2021
@rodesai rodesai added P0 Denotes must-have for a given milestone blocker and removed P0 Denotes must-have for a given milestone needs-triage labels Jul 7, 2021
@rodesai rodesai added this to the 0.22.0 milestone Jul 7, 2021
@rodesai rodesai modified the milestones: 0.22.0, 0.23, 0.23.0 Aug 23, 2021
@mkandaswamy mkandaswamy self-assigned this Oct 7, 2021
mkandaswamy added a commit that referenced this issue Oct 20, 2021
… a valid backup exists #7753 (#8257)

* fix: Avoid recreating command topic when it has been deleted and a valid backup exists.

* fix Integration test avoidRecreatingCommandTopicWithActiveBackup

* Added a new unit test in CommandTopicBackupImplTest and fixed checkstyle errors.

* Fixed unit test failure due to a NullPointerException. Also, improved error messages.

* Fix integration test failure. Also, refactor integration test code changes into its own test file: QuickDegradeAndRestoreCommandTopicIntegrationTest

* Addressed Steven's review comment. Also, got rid of an unnecessary import in RestoreCommandTopicIntegrationTest

* Addressed Steven's review comment by swapping check for degraded state and topic exists in integration test.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocker core-infra Issues owned by the ksqlDB Core Infra team enhancement
Projects
None yet
3 participants