UTF-8 support in dos2unix

[a-iv]

Git version 1.9.4-preview20140929 ships with old version of dos2unix without utf-8 support.

$ dos2unix --version
dos2unix 3.1 (Thu Nov 19 1998)

Old dos2unix returns 0x00 for non 7-bit ASCII symbols:

$ echo "42¢" | dos2unix
42

Here is non-printable symbols:

$ echo "42¢" | cat --show-all
42M-BM-"$

$ echo "42¢" | dos2unix | cat --show-all
42^@^@^M$

You can update dos2unix to avoid this issue. E.g:

$ /c/Program\ Files\ \(x86\)/dos2unix/bin/dos2unix.exe --version
dos2unix 7.1 (2014-10-06)

$ echo "42¢" | /c/Program\ Files\ \(x86\)/dos2unix/bin/dos2unix.exe | cat --show-all
42M-BM-"$

Also please note that old dos2unix is in \bin\ and will be used by default.

N.B: I am using Microsoft Windows [Version 6.3.9600].

[dscho]

Great analysis!

Please note, though, that we are in the middle of switching to a different development environment to develop Git for Windows, away from msysGit to the Git for Windows SDK. With that development environment, we no longer have to compile everything from scratch ourselves but can make use of the excellent mingw-wget packaging system. The current dos2unix version as per mingw-wget might already have the feature you desire.

How about hopping over and give it a try?

[a-iv]

@dscho thank you for feedback!

You are right. Git for Windows SDK contains dos2unix with utf-8 support.

But for now I can't hop over.

[a-iv]

Thank you!

Git for Windows SDK works like a charm.

[dscho]

Good! I will close this here ticket, then, because we will fix it by switching to the SDK.