(Assumed) Race-condition leads to par-each/rm error

[NotTheDr01ds]

Describe the bug

I came across this when answering Question #12560, but I believe there are two issues at play. This is the first, and I believe it's probably an upstream issue in the uutils/coreutils rm implementation.

Given a directory that looks like:

╭───┬─────────┬──────┬─────────┬──────────╮
│ # │  name   │ type │  size   │ modified │
├───┼─────────┼──────┼─────────┼──────────┤
│ 0 │ foo     │ dir  │ 4.0 KiB │ now      │
│ 1 │ foo/fru │ file │     0 B │ now      │
╰───┴─────────┴──────┴─────────┴──────────╯

An intermittent error (perhaps 1 in 10 tries on my system) will occur under the following conditions:

• ls **/* with a
• par-each with a
• rm -rf $in.name

This does not occur with ^rm rf $in.name

How to reproduce

Given it's intermittent, it's easiest to see by running multiple times:

for _ in 1..10 {
  let dir = mktemp -d
  cd $dir
  mkdir foo
  touch foo/fru
  print (ls **/*)
  ls **/*
  | par-each {
      let filename = $in.name
      # print $'Deleting ($filename)'
      rm -rf $filename
    }
  cd -
  rm -rf $dir
}

Notes:

• The above creates test, temporary directories and removes them when done.
• If you don't get the error on the first run, give it another go and/or increase the number of loops.

Eventually you'll see, amongst the results:

Error: nu::shell::remove_not_possible

  × Remove not possible
    ╭─[entry #1:11:14]
 10 │       # print $'Deleting ($filename)'
 11 │       rm -rf $filename
    ·              ────┬────
    ·                  ╰── Could not delete /tmp/tmp.F6oJbVGMXz/foo/fru: No such file or directory (os error 2)
 12 │     }
    ╰────

Note that:

• This is different from the normal message you'd receive with rm thisFileDoesntExist:

  rm thisFileDoesntExist
Error: nu::shell::directory_not_found

  × Directory not found
   ╭─[entry #1:1:4]
 1 │ rm thisFileDoesntExist
   ·    ─────────┬─────────
   ·             ╰── directory not found
   ╰────
  help: /tmp/thisFileDoesntExist does not exist

• rm -rf thisFileDoesntExist should never fail under normal conditions, either in the uutils/coreutils version or Gnu.

Expected behavior

No error. As mentioned, the problem doesn't occur with the external rm:

for _ in 1..1_000 {
  let dir = mktemp -d
  cd $dir
  mkdir foo
  touch foo/fru
  print (ls **/*)
  ls **/*
  | par-each {
      let filename = $in.name
      # print $'Deleting ($filename)'
      ^rm -rf $filename
    }
  cd -
  rm -rf $dir
}

Screenshots

No response

Configuration

key	value
version	0.92.2
branch
commit_hash
build_os	linux-x86_64
build_target	x86_64-unknown-linux-gnu
rust_version	rustc 1.77.2 (25ef9e3d8 2024-04-09)
rust_channel	1.77.2-x86_64-unknown-linux-gnu
cargo_version	cargo 1.77.2 (e52e36006 2024-03-26)
build_time	2024-04-12 20:02:27 -04:00
build_rust_channel	release
allocator	mimalloc
features	dataframe, default, sqlite, trash, which
installed_plugins

Additional context

I assume that this is a race condition while rm -rf foo is being processed:

1. It obtains the list of files to walk under the foo directory.
2. In a parallel operation, the separate rm -rf foo/fru completes between steps 1 and 3.
3. The rm -rf foo fails when it attempts to remove foo/fru.

Again, if this is the case, it's probably an upstream issue.