[Feature]: Testing Disk and GC Impact of --data_append_mode

[holiday-404]

Check if --data_append_mode=true causes more disk space usage and if GC makes performance unstable. Also compare it with append_mode=false in terms of disk usage and GC behavior

[holiday-404]

Because our system uses a COW (copy-on-write) design, each write operation creates a new 4KB copy instead of updating in place. The original data is then marked as invalid, but it’s not immediately reclaimed. A file is only reclaimed when all pages in it are marked invalid.

The system uses a circular layout, and when data_append_mode = true, it always writes data by appending, without reusing old invalid pages (unless an entire file is collected by GC).

When the disk usage goes full circle and returns to the beginning, it may find some files with small remaining valid pages (“bubbles”). At this point, the system needs to "squeeze the bubbles" — move the valid pages to other files with free space, so that the current file can be fully reclaimed. In reality, GC is often triggered this way, rather than simply waiting for a file to become fully invalid on its own.

This leads to disk space amplification, and GC can use a lot of resources, possibly causing performance drops.

On the other hand, if data_append_mode = false, the system reuses old invalid pages directly, meaning it performs random writes instead of sequential writes. This may cause higher CPU usage, but disk space usage stays low, and GC is usually not needed, since space is reused immediately.