Skip to content

Skip WalPeriods that are empty #26237

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hiltontj opened this issue Apr 8, 2025 · 0 comments
Open

Skip WalPeriods that are empty #26237

hiltontj opened this issue Apr 8, 2025 · 0 comments
Labels

Comments

@hiltontj
Copy link
Contributor

hiltontj commented Apr 8, 2025

Problem

#26223 resolved an issue that was leading to empty WriteBatchs being created, which therefore had min and max timestamps of i64::MAX and i64::MIN, respectively, and could produce corrupted WAL files (see #25650).

There may still be corrupted WAL files created on users' object stores, however, so we need to handle those without panic'ing and preventing server start where completely clearing the WAL from object store is not an option.

Proposed solution

Add a check here:

// add this to the snapshot tracker, so we know what to clear out later if the replay
// was a wal file that had a snapshot
self.flush_buffer
.lock()
.await
.replay_wal_period(WalPeriod::new(
wal_contents.wal_file_number,
Timestamp::new(wal_contents.min_timestamp_ns),
Timestamp::new(wal_contents.max_timestamp_ns),
));

So that if the min and max timestamps are equal to i64::MAX and i64::MIN, respectively, we skip adding the WalPeriod to the buffer (but still handle snapshot if necessary). This will avoid the panic in #25650, but we also need to verify that skipping WalPeriods does not lead to other downstream issues.

This might require some manual testing to get right since we resolved the bug that allowed for the generation of these corrupt WAL files in the first place.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant