Command-line tool clears the file if the output file path is identical to the input file path

[Socialdarwinist]

Usage example:
ftfy -e windows-1251 -o weirdfile.txt weirdfile.txt

Result: An empty file.

[rspeer]

Thanks, I'd say it sounds important to catch that case so we aren't accidentally clearing files. (Although, my data cleaning pipeline usually involves writing to a different file, not the same file -- what if ftfy changes, or you want to run it with different options?)

[sonOfRa]

I also just ran into this error. GNU sed has a similar problem and solves it by adding the "-i" flag, which does an in-place replacement. So an example invocation would be ftfy -e windows1251 -i weirdfile.txt.

That would also make it harder to do by accident (maybe error out if output and input are the same, and buffer the output in a temporary file for the -i solution).

A workaround I've used is sponge(1) from moreutils and use output redirection instead of ftfy's -o option:

ftfy -e windows-1251 weirdfile.txt | sponge weirdfile.txt. Basically sponge takes everything it gets from stdin, buffers it to a temporary file, and then writes the contents to the output file.