Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

First-sync of nodes in DR configuration fails silently and fills disk if sync timeout is too short #1892

Closed
nblumhardt opened this issue Jul 12, 2023 · 1 comment
Closed

First-sync of nodes in DR configuration fails silently and fills disk if sync timeout is too short #1892

nblumhardt opened this issue Jul 12, 2023 · 1 comment
Labels
bug
Milestone
2023.3.9558

Comments

@nblumhardt
Copy link
Member

When adding a new follower to a DR leader, if the initial sync cannot be completed in the leader's sync time limit, synchronization fails and is re-started.

Temporary files from the initial attempt are not cleaned up properly, causing the follower machine to gradually run out of disk space as the sync is retried.

Because the leader drops follower connections on timeout, no useful diagnostic information is shown in Settings > Cluster.
@nblumhardt nblumhardt added the bug label Jul 12, 2023
@nblumhardt nblumhardt added this to the 2023.3 milestone Jul 12, 2023
@nblumhardt
Copy link
Member Author

2023.3 now allows more configurable timeouts and cleans up after failures.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
2023.3.9558
Development

No branches or pull requests
1 participant
@nblumhardt