launching retry into the background disrupts bash's while read -r

[adrelanos]

test script:

#!/bin/bash

set -x
set -e

command -v retry >/dev/null

testfile=./testfile
echo "1
2
3" > "$testfile"

while read -r testvar ; do
  true "testvar: $testvar"

  retry --times=3 --delay=1 -- /bin/true &
  pid=$!

  wait "$pid"
  true "WAIT OK"
done < "$testfile"

true "END"

chmod +x ./test

./test

actual output:

+ set -e
+ command -v retry
+ testfile=./testfile
+ echo '1
2
3'
+ read -r testvar
+ true 'testvar: 1'
+ pid=395200
+ wait 395200
+ retry --times=3 --delay=1 -- /bin/true
+ true 'WAIT OK'
+ read -r testvar
+ true END

The loop runs actually only 1 time instead of the expected 3 times.

[adrelanos]

With retry removed from the test scripts...

#!/bin/bash

set -x
set -e

command -v retry >/dev/null

testfile=./testfile
echo "1
2
3" > "$testfile"

while read -r testvar ; do
  true "testvar: $testvar"

  /bin/true &
  pid=$!

  wait "$pid"
  true "WAIT OK"
done < "$testfile"

true "END"

The script runs es expected.

+ set -e
+ command -v retry
+ testfile=./testfile
+ echo '1
2
3'
+ read -r testvar
+ true 'testvar: 1'
+ pid=395211
+ wait 395211
+ /bin/true
+ true 'WAIT OK'
+ read -r testvar
+ true 'testvar: 2'
+ pid=395212
+ wait 395212
+ /bin/true
+ true 'WAIT OK'
+ read -r testvar
+ true 'testvar: 3'
+ pid=395213
+ wait 395213
+ /bin/true
+ true 'WAIT OK'
+ read -r testvar
+ true END

[adrelanos]

chatgpt had an why this is happening and a workaround:

The behavior is likely due to the retry command in the background consuming the rest of the input from the file, which is being fed into the while read -r loop. When retry is launched in the background, it probably inherits the file descriptor for stdin, reads from it, and hence the while read -r loop doesn't get a chance to process the remaining lines from the input file.

To fix this issue, you can redirect the stdin for the retry command to /dev/null like so: retry --times=3 --delay=1 -- /bin/true < /dev/null &. This way, retry won't consume the input intended for the while read -r loop, allowing the loop to process all lines from the input file.

And indeed. Command:

retry --times=3 --delay=1 -- /bin/true < /dev/null &

was functional.

Is there something you could do to fix this in retry?

[minfrin]

This behaviour is by design:

Retry captures stdin into memory as the data is passed to the repeated command, and this captured stdin is then replayed should the command be repeated.

Passing /dev/null to stdin is a useful way to stop retry reading stdin when you don't want it to. It is possible to add a switch to turn off the stdin/stdout pump where needed.