Pattern: Reading and writing same file in the pipeline
Issue: -
Each step in a pipeline runs in parallel.
In this case, grep foo file.txt
will immediately try to read file.txt
while sed .. > file.txt
will immediately try to truncate it.
This is a race condition, and results in the file being partially or (far more likely) entirely truncated.
Example of incorrect code:
grep foo file.txt | sed -e 's/foo/bar/g' > file.txt
Example of correct code:
grep foo file.txt | sed -e 's/foo/bar/g' > tmpfile && mv tmpfile file.txt
You can ignore this error if:
- The file is a device or named pipe. These files don't truncate in the same way.
- The command mentions the filename but doesn't read/write it, such as
echo log.txt > log.txt
.