Debounced Create/Rename/Remove is out of order for tempfiles

[xobs]

When updating files on Windows, the Create event comes before the Rename and Remove events. This is particularly noticeable when using vi, which creates swap files first and does a rename().

Using the example program, changing the path to my own path D:/Code/rust/test-notify and running 4.0.1:

extern crate notify;

use notify::{Watcher, RecursiveMode, watcher};
use std::sync::mpsc::channel;
use std::time::Duration;

fn main() {
    // Create a channel to receive the events.
    let (tx, rx) = channel();

    // Create a watcher object, delivering debounced events.
    // The notification back-end is selected based on the platform.
    let mut watcher = watcher(tx, Duration::from_secs(10)).unwrap();

    // Add a path to be watched. All files and directories at that path and
    // below will be monitored for changes.
    watcher.watch("D:/Code/rust/test-notify/notify", RecursiveMode::Recursive).unwrap();

    loop {
        match rx.recv() {
           Ok(event) => println!("{:?}", event),
           Err(e) => println!("watch error: {:?}", e),
        }
    }
}

For this test, I create the empty directory "notify", then run the program. As the program is run, I use vim to edit order-test.txt, save it, modify it, and save it again:

[6:54:19 PM] D:/Code/rust/test-notify> cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `target\debug\test-notify.exe`
Create("D:/Code/rust/test-notify/notify\\.order-test.txt.swp")
Create("D:/Code/rust/test-notify/notify\\order-test.txt")
NoticeRemove("D:/Code/rust/test-notify/notify\\order-test.txt")
NoticeWrite("D:/Code/rust/test-notify/notify\\.order-test.txt.swp")
Write("D:/Code/rust/test-notify/notify\\.order-test.txt.swp")
Create("D:/Code/rust/test-notify/notify\\order-test.txt")
Rename("D:/Code/rust/test-notify/notify\\order-test.txt", "D:/Code/rust/test-notify/notify\\order-test.txt~")
Remove("D:/Code/rust/test-notify/notify\\order-test.txt~")

The first Create() is when I open vim, the second Create() is when I save the file. The final NoticeWrite()/Write()/Create()/Rename()/Remove() comes when I save the modified file.

The Rename() should come before the Create().

[xobs]

This looks like an issue with DebouncedEvents. It works as expected if I set the debounce timer to 0.

The following testcase demonstrates this:

#[test]
fn create_rename_remove_temp_file() {
    let tdir = TempDir::new("temp_dir").expect("failed to create temporary directory");

    tdir.create("file1");
    sleep_macos(10);

    let (tx, rx) = mpsc::channel();
    let mut watcher: RecommendedWatcher = Watcher::new(tx, Duration::from_millis(DELAY_MS)).expect("failed to create debounced watcher");
    watcher.watch(tdir.mkpath("."), RecursiveMode::Recursive).expect("failed to watch directory");

    sleep_macos(10);
    tdir.rename("file1", "file2");
    sleep_macos(10);
    tdir.create("file1");
    tdir.remove("file2");

    assert_eq!(recv_events_debounced(&rx), vec![
        DebouncedEvent::NoticeRemove(tdir.mkpath("file1")),
        DebouncedEvent::Rename(tdir.mkpath("file1"), tdir.mkpath("file2")),
        DebouncedEvent::Create(tdir.mkpath("file1")),
        DebouncedEvent::Remove(tdir.mkpath("file2"))
     ]);
}

[passcod]

This only happens on Windows, or is that just where you tested?

[xobs]

I initially tested it only on Windows, but I have now also confirmed that the issue happens on a Linux machine.

[passcod]

I'm not entirely certain this is a bug. Order of events is not something that's very guaranteed, even at the platform level. Is this just something you noticed, or does it cause an issue with your usecase?

[xobs]

It's causing an issue with my usecase, however I've worked around it by creating my own debounce queue and disabling debounce in notify. The behavior is somewhat surprising and took me a while to track down, which is why I opened the issue.

My application is monitoring a directory for unit files, much like systemd. When a unit file is added, it is activated. When a unit file is removed, it is deactivated. If you edit a unit file in vim and save it, because of the out-of-order messages, the "remove" message will come last which will cause the unit file to be removed from the unit table.

[dfaust]

Thanks for reporting. This was a tricky bug. I changed the behavior of the debounce module to ignore all events for files that have been moved and deleted while a new file at the original location has been created. From the user's point of view this is as if the original file was modified. Just instead of a write event notify emits a create event, which shouldn't be a problem, right?

[xobs]

That shouldn't be a problem. Thanks for fixing it so quickly!