create_directory_watch_subdirectories fails on Fedora

[ignatenkobrain]

---- create_directory_watch_subdirectories stdout ----
thread 'create_directory_watch_subdirectories' panicked at 'assertion failed: `(left == right)`
  left: `[("/tmp/temp_dir.ARQ1ErCLe1Od/dir1", CREATE, None), ("/tmp/temp_dir.ARQ1ErCLe1Od/dir1/dir2", CREATE, None), ("/tmp/temp_dir.ARQ1ErCLe1Od/dir1/dir2/file1", CREATE, None), ("/tmp/temp_dir.ARQ1ErCLe1Od/dir1/dir2/file1", CLOSE_WRITE, None)]`,
 right: `[("/tmp/temp_dir.ARQ1ErCLe1Od/dir1", CREATE, None), ("/tmp/temp_dir.ARQ1ErCLe1Od/dir1/dir2/file1", CREATE, None), ("/tmp/temp_dir.ARQ1ErCLe1Od/dir1/dir2/file1", CLOSE_WRITE, None)]`', tests/notify.rs:507:9
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

[passcod]

Consistently? On what filesystem, Rust version, kernel version? What does the backtrace say?

[ignatenkobrain]

• 4 out of 6 :)
• rustc 1.33.0
• Linux version 5.1.0-0.rc0.git4.2.fc31.x86_64 (mockbuild@buildvm-18.phx2.fedoraproject.org) (gcc version 9.0.1 20190227 (Red Hat 9.0.1-0.8) (GCC)) #1 SMP Fri Mar 8 18:58:06 UTC 2019

---- create_directory_watch_subdirectories stdout ----
thread 'create_directory_watch_subdirectories' panicked at 'assertion failed: `(left == right)`
  left: `[("/tmp/temp_dir.o617Am4z9gWW/dir1", CREATE, None), ("/tmp/temp_dir.o617Am4z9gWW/dir1/dir2", CREATE, None), ("/tmp/temp_dir.o617Am4z9gWW/dir1/dir2/file1", CREATE, None), ("/tmp/temp_dir.o617Am4z9gWW/dir1/dir2/file1", CLOSE_WRITE, None)]`,
 right: `[("/tmp/temp_dir.o617Am4z9gWW/dir1", CREATE, None), ("/tmp/temp_dir.o617Am4z9gWW/dir1/dir2/file1", CREATE, None), ("/tmp/temp_dir.o617Am4z9gWW/dir1/dir2/file1", CLOSE_WRITE, None)]`', tests/notify.rs:507:9
stack backtrace:
   0:     0x560d281eb33f - std::sys::unix::backtrace::tracing::imp::unwind_backtrace::h8bd79c2bf8ecd6f3
   1:     0x560d281e33b7 - std::sys_common::backtrace::print::h67807d3cdb7b7237
   2:     0x560d281ee00f - std::panicking::default_hook::{{closure}}::he2bb229789800be8
   3:     0x560d281edd20 - std::panicking::default_hook::h91e5274148186d07
   4:     0x560d281ee700 - std::panicking::rust_panic_with_hook::h2ff3a9bb34d4e1a9
   5:     0x560d281ee291 - std::panicking::continue_panic_fmt::h31f93fb3d2584798
   6:     0x560d281ee1de - std::panicking::begin_panic_fmt::h1f6d3d4150997da4
   7:     0x560d28156d19 - core::ops::function::FnOnce::call_once::h5df92395fa7852b5
   8:     0x560d2816d6ee - <F as alloc::boxed::FnBox<A>>::call_box::hfc6c1f9e02699fd9
   9:     0x560d281f12a9 - __rust_maybe_catch_panic
  10:     0x560d2818d82d - std::sys_common::backtrace::__rust_begin_short_backtrace::hd82c1e31508f0064
  11:     0x560d2818ba64 - std::panicking::try::do_call::h9c648196c22223e6
  12:     0x560d281f12a9 - __rust_maybe_catch_panic
  13:     0x560d2818ef0c - <F as alloc::boxed::FnBox<A>>::call_box::h407023d842c792fe
  14:     0x560d281ec80d - std::sys_common::thread::start_thread::hc63b4752b13400d0
  15:     0x560d281df3d5 - std::sys::unix::thread::Thread::new::thread_start::hda130ff6710e06d8
  16:     0x7fd47be6a5a1 - start_thread
  17:     0x7fd47bd7efa2 - __GI___clone
  18:                0x0 - <unknown>

[ignatenkobrain]

Oh, and filesystem...

tmpfs on /tmp type tmpfs (rw,nosuid,nodev,seclabel)

[passcod]

OH and is this the debounce or the notify test run?

[ignatenkobrain]

@passcod it is actually both of them. But from few tens of runs, notify one seems to fail more often :)

[passcod]

Also I notice this is an pre-release kernel? Which is currently my only lead because I can't repro on kernel 5.0.0

[passcod]

woo! we get to read kernel diffs 😓

[passcod]

I would ask that you try on a stable kernel, if possible, at least to isolate to that if that is indeed the cause.

[passcod]

From a cursory look, I think they're correcting an oversight in inotify and that's leading to a new event in this situation. This is great for correctness but obviously will make the test suite differ depending on the kernel version. Unsure how that will play out, and in any case I'll want to wait until 5.1 is out with those changes before doing anything concrete.

[ignatenkobrain]

I'll try it later today on 5.0.0 and write you back. Thanks for your help, much appreciated :)

[ignatenkobrain]

Seems it is not reproducible with Linux version 5.1.0-0.rc0.git8.2.fc31.x86_64 (mockbuild@buildhw-04.phx2.fedoraproject.org) (gcc version 9.0.1 20190312 (Red Hat 9.0.1-0.10) (GCC)) #1 SMP Thu Mar 14 21:53:37 UTC 2019

[ignatenkobrain]

Apparently it is reproducible on Linux buildvm-16.phx2.fedoraproject.org 5.0.6-200.fc29.x86_64 #1 SMP Wed Apr 3 15:09:51 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux...

But I suppose not much can be done.

[passcod]

I believe the new kernel behaviour is correct and the 4.0 test suite is overly strict... testing exactly to the platform instead of testing what notify should expect, which has also been a problem elsewhere. I'll try to fix that general issue when I write the tests back during one of the 5.0 prereleases.