Rolling file appender overgrows x.log.0 on restart when max_file_size is set

[Mota-Link]

Hello, and thanks for the great project.

I believe I’ve found an issue in the rolling file appender when max_file_size is configured.

Problem description

• When rotation happens during a run, files are created as x.log.0, x.log.1, ...
• After the program exits and restarts, the appender starts writing to x.log.0 again in append mode.
• It continues appending to x.log.0 until the amount of data written in this new run reaches max_file_size, and only then rotates to the next file.
• As a result, x.log.0 can grow to roughly 2 × max_file_size after a single restart. If the program is restarted multiple times, x.log.0 keeps growing cumulatively beyond the configured limit.

Expected behavior

• If max_file_size is set, the appender should discover the latest existing file on startup:
  • Scan x.log.0, x.log.1, ... to find the highest index n that exists.
  • If x.log.n is not yet full (size < max_file_size), resume appending to x.log.n.
  • If x.log.n is already full (size >= max_file_size), roll to x.log.(n+1) and start a new file.
• In short, resume from the latest index and honor the size limit across restarts.

I have attached a minimal reproducible example, adapted from the example code of rolling_file. You can reproduce the issue simply by running the program multiple times.

use logforth::append::rolling_file::RollingFileBuilder;
use logforth::layout::JsonLayout;

fn main() {
    let (rolling_writer, _guard) = RollingFileBuilder::new("logs")
        .layout(JsonLayout::default())
        .max_file_size(500)
        .filename_prefix("app_log")
        .build()
        .unwrap();

    logforth::builder()
        .dispatch(|d| d.filter(log::LevelFilter::Trace).append(rolling_writer))
        .apply();

    let repeat = 1;

    for i in 0..repeat {
        log::error!("Hello error!");
        log::warn!("Hello warn!");
        log::info!("Hello info!");
        log::debug!("Hello debug!");
        log::trace!("Hello trace!");

        if i + 1 < repeat {
            std::thread::sleep(std::time::Duration::from_secs(10));
        }
    }
}

Could you please confirm whether this is the intended behavior or if it might be a bug?

Thank you!

[tisonkun]

Looks like a bug - the current_filesize is set to 0 each time the state gets initialized, while we'd use the real current size if the file already exists.

• feat: allow rolling file at size #32

logforth/src/append/rolling_file/rolling.rs

Line 189 in ad597df

let current_filesize = 0;

[tisonkun]

Or we may resolve the initial cnt to always use a new file on started.

[Mota-Link]

I’ve noticed that handling user manipulation of the log directory can increase the complexity of log rotation significantly. From reviewing the delete_oldest_logs function, it seems that file rotation currently depends on the files’ created time, not strictly on the numeric file index.

For example, suppose at program termination the log directory contains files like x.log.9, x.log.10, and x.log.11 (with max_size and max_files configured). If a user manually adds a file such as x.log.2 during downtime, the rotation logic—which relies on the latest creation time to determine the file counter—might start incrementing from x.log.2 upon restart. This still causes issues when rolling over past existing files like x.log.9.

Alternatively, trying to determine the latest log file by filename isn’t trivial, since simple lexicographical sorting would treat "2" as greater than "11". To get correct behavior, every file’s suffix has to be parsed as a number. If a user then creates x.log.13 during runtime, this introduces more corner cases that are hard to handle robustly.

I think issue #143 Log file name style is a great suggestion. Adopting a scheme with a fixed set of sequence number files (e.g., only x.log.0 to x.log.2) would prevent users from manually creating files with smaller sequence numbers, and we wouldn’t need to track or consider extraneous higher-index files. This would greatly simplify the rotation logic and make it less error-prone.

[tisonkun]

Implementing at #159

@Mota-Link welcome to give a review