Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete associated ledgers before deleting cluster metadata #8244

Merged

Conversation

BewareMyPower
Copy link
Contributor

Motivation

#8169 introduced a command tool to delete a cluster's metadata from ZK. This PR intends to delete the cluster's ledgers from BK.

Modifications

  • Retrieve ledger ids from related ZK nodes
  • Add an optional argument to specify BK metadata service URI, then delete these ledgers if it's specified

Verifying this change

  • Make sure that the change passes the CI checks.

@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

Copy link
Member

@sijie sijie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BewareMyPower Can you delete the managed ledger and ledgers using an iterator rather than collecting all the ledgers into a list? If a cluster has a lot of ledgers, it will fill up the memory.

@BewareMyPower
Copy link
Contributor Author

@sijie OK

@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

1 similar comment
@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

@BewareMyPower
Copy link
Contributor Author

The change is done, PTAL, @sijie .

In addition, some schema ledgers are not recorded in ZK, so it's not possible to locate them now. In #8528 I try to solve the issue. If one of these two PRs was merged first, I would complete the rest work in the other PR.

@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

@BewareMyPower
Copy link
Contributor Author

Now all checks have passed, PTAL again, @sijie @eolivelli

Copy link
Member

@sijie sijie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BewareMyPower Can you please add unit tests and integration tests for this tool?

@sijie sijie added this to the 2.7.0 milestone Oct 21, 2020
@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

2 similar comments
@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

@BewareMyPower
Copy link
Contributor Author

/pulsarbot run-failure-checks

@BewareMyPower
Copy link
Contributor Author

I've added an integration test, PTAL, @sijie

@sijie sijie added the area/cli label Oct 22, 2020
@BewareMyPower BewareMyPower changed the title [broker] Delete associated ledgers before deleting cluster metadata Delete associated ledgers before deleting cluster metadata Oct 26, 2020
Copy link
Member

@sijie sijie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BewareMyPower Great job!

@codelipenghui codelipenghui merged commit 9922249 into apache:master Oct 27, 2020
wolfstudy pushed a commit that referenced this pull request Oct 30, 2020
### Motivation

#8169 introduced a command tool to delete a cluster's metadata from ZK. This PR intends to delete the cluster's ledgers from BK.

### Modifications

- Retrieve ledger ids from related ZK nodes
- Add an optional argument to specify BK metadata service URI, then delete these ledgers if it's specified

(cherry picked from commit 9922249)
huangdx0726 pushed a commit to huangdx0726/pulsar that referenced this pull request Nov 13, 2020
### Motivation

apache#8169 introduced a command tool to delete a cluster's metadata from ZK. This PR intends to delete the cluster's ledgers from BK.

### Modifications

- Retrieve ledger ids from related ZK nodes
- Add an optional argument to specify BK metadata service URI, then delete these ledgers if it's specified
flowchartsman pushed a commit to flowchartsman/pulsar that referenced this pull request Nov 17, 2020
### Motivation

apache#8169 introduced a command tool to delete a cluster's metadata from ZK. This PR intends to delete the cluster's ledgers from BK.

### Modifications

- Retrieve ledger ids from related ZK nodes
- Add an optional argument to specify BK metadata service URI, then delete these ledgers if it's specified
@BewareMyPower BewareMyPower deleted the bewaremypower/delete-ledgers-dev branch December 2, 2020 07:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants