mito: Potential issue to recover truncated data

[evenyag]

What type of bug is this?

Incorrect result

What subsystems are affected?

Standalone mode, Datanode

Minimal reproduce step

Truncate a region and restart the process before obsoleting the truncated entry.

What did you expect to see?

Region is empty

What did you see instead?

Region contains truncated data

What operating system did you use?

All

Relevant log output and stack trace

NA

[evenyag]

Now the region doesn't consider truncated entries during replay.

greptimedb/src/mito2/src/region/opener.rs

Lines 256 to 270 in c2db970

	let flushed_entry_id = version.flushed_entry_id;
	let version_control = Arc::new(VersionControl::new(version));
	if !self.skip_wal_replay {
	info!(
	"Start replaying memtable at flushed_entry_id + 1 {} for region {}",
	flushed_entry_id + 1,
	region_id
	);
	replay_memtable(
	wal,
	&wal_options,
	region_id,
	flushed_entry_id,
	&version_control,
	)

It should skip entries <= truncated_entry_id. We might do this in replay_memtable().

[evenyag]

After checking how we recover the truncate action, I found that apply_truncate() updates the flushed entry id by the truncated entry id. Thus the region won't replay entries before the truncated entry id.

greptimedb/src/mito2/src/manifest/action.rs

Lines 147 to 153 in c2db970

	pub fn apply_truncate(&mut self, manifest_version: ManifestVersion, truncate: RegionTruncate) {
	self.manifest_version = manifest_version;
	self.flushed_entry_id = truncate.truncated_entry_id;
	self.flushed_sequence = truncate.truncated_sequence;
	self.truncated_entry_id = Some(truncate.truncated_entry_id);
	self.files.clear();
	}