Switch Unix tools from Git internal tools to MSYS2 tools on Windows images

[AlenaSviridenko]

Breaking changes

"Git For Windows" Unix tools will be replaced by "MSYS2" Unix tools on Windows images.
Currently, folders C:\Program Files\Git\bin and C:\Program Files\Git\usr\bin and C:\Program Files\Git\mingw64\bin are added to PATH and it expose a ton of Unix tools (Git For Windows Internal tools) to PATH. We would like to remove those folders from PATH because one day they can be removed by Git maintainers.
A lot of people already depend on those tools so we would like to replace Git Unix tools by MSYS2 Unix tools to reduce impact.

Target date

Deployment will start on November, 2 and changes propagation will take 2-3 days.
This change was rolled back and should be reevaluated

The motivation for the changes

Per communication with Git For Windows maintainers, we should not add Git Internal tools to PATH and rely on it. They could be removed in future and break a lot of people.

More details can be found in this PR by @dscho #1435.

All discussions can be found in this issue, also in MSYS2 issue #1572 and in the final PR #1648

Possible impact

The most of the tools are identical between "Git For Windows" Internal and MSYS2 but some minor versions could be different and it can affect some builds.

• Windows 2016
• Windows 2019

Mitigation ways

You will be able to return "Git For Windows" Unix tools to PATH using the following powershell step:

• GitHub Actions:

echo "C:\Program Files\Git\mingw64\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "C:\Program Files\Git\usr\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append

• Azure Pipelines

echo "##vso[task.prependpath]C:\Program Files\Git\mingw64\bin"
echo "##vso[task.prependpath]C:\Program Files\Git\usr\bin"

[maxim-lobanov]

These changes will remove some Unix tools (that come with Git) from PATH on Windows machines. See the full list of tools in post above. This change won't affect such tools like bash and tar.

@bryanmacfarlane @konradpabjan , @damccorm do you know if any of these tools can be used on Windows by actions/toolkit or azure-pipelines-task-lib or and it can affect you somehow?
After quick look, I see that you mostly use fs Node.JS module and default Windows commands. The single unix command that you use is tar but it won't be affected since it is located in Windows/System32.

[damccorm]

As far as I know we should be good with this change from the Azure Pipelines side

[joshmgross]

The single unix command that you use is tar but it won't be affected since it is located in Windows/System32.

@maxim-lobanov I believe Windows 2016 relies on the Git Unix tar since only Windows 2019 has the Windows tar (see https://docs.microsoft.com/en-us/virtualization/community/team-blog/2017/20171219-tar-and-curl-come-to-windows).

tar is necessary for actions/cache

[thboop]

Besides tar on earlier versions of windows, to the best of my knowledge I think we should be good for the toolkit. We will definitely want to manually install that before we remove this from path.

Individual actions or users may be using these utilities though, there isn't a great way to audit that. We should plan on rolling out slowly and rolling back in case we start seeing issues.

We do use stuff like git-lfs but there appears to be a non internal git copy that takes precedent (the /cmd one)

Application git-lfs.exe                                    2.28.0.1          C:\Program Files\Git\cmd\git-lfs.exe
Application git-lfs.exe                                    0.0.0.0           C:\Program Files\Git\mingw64\bin\git-lfs.…

[maxim-lobanov]

@joshmgross , that is good point.
I think we can put to the separate folder and put this folder to the PATH, right?
actions/cache should find it: https://github.com/actions/toolkit/blob/c9819f79d2a27c1f26ef7ea85ff44346e7d60741/packages/cache/src/internal/tar.ts#L25

The reason for this change is that we shouldn't rely on Unix tools that come from Git, they can be removed in future.
Initially, all these tools were added to PATH unexpectedly. You can find more details in #1435 (comment)

[shivammathur]

@maxim-lobanov similar to tar, would it be possible to keep printf in PATH.
It is useful to output text with color as Write-Host parameters -ForegroundColor and -BackgroundColor have no effect.

[dscho]

We do use stuff like git-lfs but there appears to be a non internal git copy that takes precedent (the /cmd one)

As long as you do not try to call it via git-lfs.exe, but via git lfs (or even better: implicitly via the common smudge/clean filters), the internal copy will be picked up by Git. My PR won't change that.

[dscho]

Re: tar and printf: I really hate to spoil the party, but it is really, like really not guaranteed that Git for Windows will ship with these!

If you want to make sure that those are shipped, and officially supported, it will need to be handled via C:\Program Files\Git\cmd, and I will have to be convinced that this does not increase my maintenance burden.

One plan, for example, is to use BusyBox at some stage in the future, which does have optional support for tar (and it also supports printf, but it might not come with a printf.exe). Your reliance on these Unix tools in Git for Windows would be broken by that change. That's why I highly recommended against relying on them: you are using unsupported features as if they were supported ones.