Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch Unix tools from Git internal tools to MSYS2 tools on Windows images #1525

Closed
2 tasks done
AlenaSviridenko opened this issue Sep 1, 2020 · 72 comments
Closed
2 tasks done

Comments

@AlenaSviridenko
Copy link
Contributor

AlenaSviridenko commented Sep 1, 2020

Breaking changes

"Git For Windows" Unix tools will be replaced by "MSYS2" Unix tools on Windows images.
Currently, folders C:\Program Files\Git\bin and C:\Program Files\Git\usr\bin and C:\Program Files\Git\mingw64\bin are added to PATH and it expose a ton of Unix tools (Git For Windows Internal tools) to PATH. We would like to remove those folders from PATH because one day they can be removed by Git maintainers.
A lot of people already depend on those tools so we would like to replace Git Unix tools by MSYS2 Unix tools to reduce impact.

Target date

Deployment will start on November, 2 and changes propagation will take 2-3 days.
This change was rolled back and should be reevaluated

The motivation for the changes

Per communication with Git For Windows maintainers, we should not add Git Internal tools to PATH and rely on it. They could be removed in future and break a lot of people.

More details can be found in this PR by @dscho #1435.

All discussions can be found in this issue, also in MSYS2 issue #1572 and in the final PR #1648

Possible impact

The most of the tools are identical between "Git For Windows" Internal and MSYS2 but some minor versions could be different and it can affect some builds.

  • Windows 2016
  • Windows 2019

Mitigation ways

You will be able to return "Git For Windows" Unix tools to PATH using the following powershell step:

  • GitHub Actions:
echo "C:\Program Files\Git\mingw64\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
echo "C:\Program Files\Git\usr\bin" | Out-File -FilePath $env:GITHUB_PATH -Encoding utf8 -Append
  • Azure Pipelines
echo "##vso[task.prependpath]C:\Program Files\Git\mingw64\bin"
echo "##vso[task.prependpath]C:\Program Files\Git\usr\bin"
@AlenaSviridenko AlenaSviridenko pinned this issue Sep 1, 2020
@AlenaSviridenko AlenaSviridenko changed the title [Announcement] Git internal tools will be removed from PATH on September, 15 for Windows 2016. Git internal tools will be removed from PATH on September, 15 for Windows 2016. Sep 2, 2020
@maxim-lobanov
Copy link
Contributor

maxim-lobanov commented Sep 4, 2020

These changes will remove some Unix tools (that come with Git) from PATH on Windows machines. See the full list of tools in post above. This change won't affect such tools like bash and tar.

@bryanmacfarlane @konradpabjan , @damccorm do you know if any of these tools can be used on Windows by actions/toolkit or azure-pipelines-task-lib or and it can affect you somehow?
After quick look, I see that you mostly use fs Node.JS module and default Windows commands. The single unix command that you use is tar but it won't be affected since it is located in Windows/System32.

@damccorm
Copy link

damccorm commented Sep 4, 2020

As far as I know we should be good with this change from the Azure Pipelines side

@joshmgross
Copy link
Member

joshmgross commented Sep 4, 2020

The single unix command that you use is tar but it won't be affected since it is located in Windows/System32.

@maxim-lobanov I believe Windows 2016 relies on the Git Unix tar since only Windows 2019 has the Windows tar (see https://docs.microsoft.com/en-us/virtualization/community/team-blog/2017/20171219-tar-and-curl-come-to-windows).

tar is necessary for actions/cache

@thboop
Copy link

thboop commented Sep 4, 2020

Besides tar on earlier versions of windows, to the best of my knowledge I think we should be good for the toolkit. We will definitely want to manually install that before we remove this from path.

Individual actions or users may be using these utilities though, there isn't a great way to audit that. We should plan on rolling out slowly and rolling back in case we start seeing issues.

We do use stuff like git-lfs but there appears to be a non internal git copy that takes precedent (the /cmd one)

Application git-lfs.exe                                    2.28.0.1          C:\Program Files\Git\cmd\git-lfs.exe
Application git-lfs.exe                                    0.0.0.0           C:\Program Files\Git\mingw64\bin\git-lfs.…

@maxim-lobanov
Copy link
Contributor

maxim-lobanov commented Sep 4, 2020

@joshmgross , that is good point.
I think we can put to the separate folder and put this folder to the PATH, right?
actions/cache should find it: https://github.com/actions/toolkit/blob/c9819f79d2a27c1f26ef7ea85ff44346e7d60741/packages/cache/src/internal/tar.ts#L25

The reason for this change is that we shouldn't rely on Unix tools that come from Git, they can be removed in future.
Initially, all these tools were added to PATH unexpectedly. You can find more details in #1435 (comment)

@shivammathur
Copy link
Contributor

shivammathur commented Sep 6, 2020

@maxim-lobanov similar to tar, would it be possible to keep printf in PATH.
It is useful to output text with color as Write-Host parameters -ForegroundColor and -BackgroundColor have no effect.

@dscho
Copy link
Contributor

dscho commented Sep 7, 2020

We do use stuff like git-lfs but there appears to be a non internal git copy that takes precedent (the /cmd one)

As long as you do not try to call it via git-lfs.exe, but via git lfs (or even better: implicitly via the common smudge/clean filters), the internal copy will be picked up by Git. My PR won't change that.

@dscho
Copy link
Contributor

dscho commented Sep 7, 2020

Re: tar and printf: I really hate to spoil the party, but it is really, like really not guaranteed that Git for Windows will ship with these!

If you want to make sure that those are shipped, and officially supported, it will need to be handled via C:\Program Files\Git\cmd, and I will have to be convinced that this does not increase my maintenance burden.

One plan, for example, is to use BusyBox at some stage in the future, which does have optional support for tar (and it also supports printf, but it might not come with a printf.exe). Your reliance on these Unix tools in Git for Windows would be broken by that change. That's why I highly recommended against relying on them: you are using unsupported features as if they were supported ones.

@maxim-lobanov
Copy link
Contributor

maxim-lobanov commented Sep 7, 2020

Hello everyone,
We are a bit concerned that removing Git Unix tools from Windows images could cause unpredictable impact, also it doesn't look like a good idea to remove Git Unix tools from PATH but support hardcoded list of tools to have in PATH (like tar and printf as was asked above) so we would like to continue some discussion here.

Summary & Details:
Currently, PATH contains the following folders with Unix tools (in order of PATH priority): C:\Program Files\Git\bin, C:\windows\system32, C:\Program Files\Git\cmd, C:\Program Files\Git\mingw64\bin, C:\Program Files\Git\usr\bin.

  • C:\Program Files\Git\bin contains only bash.exe, sh.exe, git.exe.
  • C:\windows\system32 contains some Unix tools only on Windows2019.
  • C:\Program Files\Git\cmd contains only git.exe, git-gui.exe, gitk.exe, git-lfs.exe.
  • C:\Program Files\Git\usr\bin contains a lot of Unix tools (see the full list in the first message on this issue).

Per discussion with Git for Windows maintainers, we shouldn't add Git Unix tools to PATH because those tools can be unexpectedly removed in future. It means that we should remove the following folders from PATH: C:\Program Files\Git\mingw64\bin and C:\Program Files\Git\usr\bin. Since these folders are located at the end of PATH, binaries which are resolved from them, will become unavailable.
This change can cause issues and impact since some customers / tools depend on those binaries. At least, we must not remove tar and printf from PATH and potentially some other tools that we are not aware about right now.

Possible options:

  1. Accept this change in current view (remove all Unix tools from PATH except tar, printf, ???).
  2. Replace those two Git folders (C:\Program Files\Git\usr\bin and C:\Program Files\Git\mingw64\bin) by MSYS Unix binaries. We can add C:\msys64\usr\bin folder to PATH. This folder contains pretty the same list of binaries like C:\Program Files\Git\usr\bin with minimal differences and mostly the same versions. If we add C:\msys64\usr\bin at the end of PATH, removing Git tools from PATH should be pretty smoothly since the most tools will continue to work as previously. (Unix tools that are resolved from System32 folder right now, will continue to work as expected since system32 is located before in PATH)

Tools with changes versions (3 tools)

awk (5.0.0 -> 5.1.1)
gawk (5.0.0 -> 5.1.1)
grep (3.1 -> 3.0)

Tools that will be removed (54 tools)

ahost
antiword
blocked-file-util
brotli
certtool
connect
create-shortcut
edit_test
edit_test_dll
ex
fido2-assert
fido2-cred
fido2-token
git-askyesno
git-credential-helper-selector
git-receive-pack
git-upload-archive
git-upload-pack
lzmadec
lzmainfo
nano
odt2txt
pdftotext
pluginviewer
proxy-lookup
psktool
rnano
rview
rvim
sasldblistusers2
saslpasswd2
srptool
ssh
ssh-pageant
tig
unxz
view
vim
vimdiff
WhoUses
winpty
winpty-agent
winpty-debugserver
wintoast
wish
wish86
xmlwf
xxd
xz
xzcat
xzdec
zipcmp
zipmerg
ziptool

Tools without changes (223 tools)

[
arch
b2sum
base32
base64
basename
basenc
facto
test
bunzip2
bzcat
bzip2
bzip2recover
captoinfo
cat
chattr
chcon
chgrp
chmod
chown
chroot
cksum
cmp
column
comm
cp
csplit
cut
cygcheck
cygpath
cygwin-console-helper
d2u
date
dd
df
diff
diff3
dir
dircolors
dirmngr
dirmngr-client
dirname
dos2unix
du
dumpsexp
env
envsubst
expr
false
file
fmt
fold
getconf
getfacl
getopt
gettext
gkill
glib-compile-schemas
gpg
gpg-agent
gpgconf
gpg-connect-agent
gpg-error
gpgparsemail
gpgscm
gpgsm
gpgtar
gpgv
gpg-wks-server
groups
gsettings
gzip
head
hmac256
hostid
id
install
join
kbxutil
kill
ldd
ldh
less
lessecho
lesskey
link
ln
locale
locate
logname
ls
lsattr
mac2unix
md5sum
minidumper
mintty
mkdir
mkfifo
mkgroup
mknod
mkpasswd
mktemp
mount
mpicalc
msgattrib
msgcat
msgcmp
msgcomm
msgconv
msgen
msgfilter
msgfmt
msggrep
msginit
msgmerge
msgunfmt
msguniq
mv
nettle-hash
nettle-lfib-stream
nettle-pbkdf2
ngettext
nice
nl
nohup
nproc
numfmt
od
p11-kit
p11tool
passwd
paste
pathchk
pinentry
pinentry-w32
pinky
pkcs1-conv
pldd
printenv
printf
ps
pr
psl
ptx
pwd
readlink
realpath
rebase
recode-sr-latin
rm
rmdir
runcon
sdiff
sed
seq
setfacl
setmetamode
sexp-conv
sha1sum
sha224sum
sha256sum
sha384sum
sha512sum
shred
shuf
sleep
split
ssp
stat
strace
stty
sum
sync
tac
tail
tar
toe
touch
tput
tr
true
truncate
trust
tset
tsort
tty
tzset
u2d
umount
uname
unexpand
uniq
unix2dos
unix2mac
unlink
unzip
unzipsfx
users
vdir
watchgnupg
wc
which
who
xargs
xgettext
yat2m
yes
zipinfo
clear
dash
gapplication
gdbus
gencat
gio-querymodules
gobject-query
infocmp
infotocap
regtool
tabs
tclsh
tclsh86
tic
tee

  1. Please share any other suggestions

We don't have much experience with Unix tools on Windows but the second option looks like a good way to keep the most of the tools to work as previously and stop relying on Git Unix binaries

cc: @dscho as Git for Windows maintainer
cc: @MSP-Greg and @eine as MSYS experts
cc: @alepauly @AlenaSviridenko

@MSP-Greg
Copy link
Contributor

MSP-Greg commented Sep 7, 2020

I don't see any problems with this.

ruby/setup-ruby and MSP-Greg/setup-ruby-pkgs are enabling MSYS2 early in their code, and have switched to using the MSYS2 bash shell.

Re option 2 above, you may need to also add C:\msys64\mingw64\bin. It's been a while since I've compared the tools in each, as I've always used both MSYS2 paths in CI and locally. Some tools exist in both.

Note that there are two issues, first, what exe is run from a command/ps1 prompt, and the second, does this affect the use of bash shell with Windows runners.

@dscho
Copy link
Contributor

dscho commented Sep 7, 2020

I am in favor of replacing C:\Program Files\Git\usr\bin (and ...\ming64\bin) in PATH by their MSYS2 equivalents: in contrast to Git for Windows, which only so happens to ship those tools at the moment, MSYS2 does support the indicated use cases explicitly.

@MSP-Greg
Copy link
Contributor

MSP-Greg commented Sep 7, 2020

@dscho

Thanks. What is the suggested way to add Git to path, C:\Program Files\Git\bin or C:\Program Files\Git\cmd? I assume cmd, as it doesn't have bash in the folder?

EDIT: I believe the current path contains:

C:\Program Files\Git\bin

then, later in the path:

C:\Program Files\Git\cmd
C:\Program Files\Git\mingw64\bin
C:\Program Files\Git\usr\bin

@dscho
Copy link
Contributor

dscho commented Sep 8, 2020

If you want to be able to use Git for Windows' Bash, you need the Git\bin directory. Otherwise, the Git\cmd directory would be the appropriate one.

@al-cheb al-cheb self-assigned this Sep 9, 2020
@eine
Copy link

eine commented Sep 9, 2020

Initially, all these tools were added to PATH unexpectedly.

@maxim-lobanov, IMHO, this is not correct. Instead, I'd say it is because of lack of expertise, and specially lack of attention. I've been following this repository since it was created, and there were discussions about this same issue before (related to MSYS2 and/or WSL). This has been going around for about a year already (before Actions were "officially" out of beta). Overall, many different employees seem to be doing their best effort, but I feel that none of them is being scheduled the time for properly understanding the issue, providing a stable solution and maintaining it. Until managers understand the severe impact of all these arbitrary "fixes", the product is going to suffer (even more), regardless of all the effort that engineers and contributors are devoting.

As the maintainer of Git for Windows said, none of the mingw/mingw64 tools shipped with it should be used at all, under any circumstance. Should anyone be doing it, they should learn about the development tools they should be using instead, and fix it ASAP. Should their workflows be broken suddenly, that's a problem on their side, because they followed an ugly and hacky approach (see #1435 (comment)). GitHub Actions might be blamed for making it easier to unexperienced users doing mistakes and learning inadequate practices. However, I believe that's something GitHub is not worried about, given that Actions are written in JavaScript only. So, this should be fixed ASAP, regardless of the impact: "the longer you wait, the bigger the impact will be" (#1435 (comment)).

However, given that the mechanism for communicating these breaking changes seems to be "good luck finding an specific changelog", it might be better to do nothing until that is addressed.


From a wider point of view, most if not all the users that are relying on Git for Windows' internal GNU/Linux tools, do likely want to use MSYS2 instead. In MSYS2, all those utilities are explicitly expected to be used (#1525 (comment)). Plus, there is a plethora of packages available through pacman, which typically install much faster than using chocolatey (another tool which has not been discussed). Hence, the natural and shameless migration would have been to drop MSYS2 in the PATH as a replacement of Git for Windows, when MSYS2 was added some months ago (not now) (see #342 (comment)). Instead, there was no feedback about the proposal, the implementation was done by a third-party contributor, attending his own needs only, and no maintainer (employee or contributor) took care of it.

Very fortunately, some months ago actions/toolkit#318 was partially addressed, and it is now possible to preset a custom shell, even before the file/script exists. See, for instance:

  defaults:
    run:
      # We define a custom shell script here, although `msys2.cmd` does neither exist nor is it available in the PATH yet
      shell: msys2 {0}
  steps:

  # We use a JS Action, which calls the system terminal or other custom terminals directly, if required
  - uses: msys2/setup-msys2@v2
    with:
      update: true
      install: base-devel git

  # We want to change the configuration of the git command that actions/checkout will be using (since it is not possible to set autocrlf through the action yet, see actions/checkout#226).
  - run: git config --global core.autocrlf input
    shell: bash

  # This uses the default git in the environment, thus Git for Windows
  - uses: actions/checkout@v2

  # This uses the default shell, which is msys2 (defined above, and "installed" by msys2/setup-msys2)
  - run: git describe --dirty

At the same time, maintainers of MSYS2 were really pro-active and put lots of love into the Action. Updates and fixes are addressed in few hours, and all their knowledge about the environment is seen in the Action. From user's perspective, even if 0.5-1 GB of ad-hoc packages need to be installed, startup takes no longer than 90s.

Summarizing, I believe that both Git for Windows and MSYS2 can and should be dropped from the environment. There should be an Action for installing Git for Windows. That Action should also be available as a library, in case other Actions rely on it. Nevertheless, I believe that Actions such as actions/checkout should be compatible with any git (i.e. with the one available through msys/setup-msys2 and/or in WSL too).

Alternatively, Git for Windows, MSYS2 and WSL might coexist, along with handy built-in setup scripts and/or custom entry shells. That would be ideal. Unfortunately, that needs a thoughful proposal and discussion, instead of fixing partial issues as they come by and looking at limited scopes.


Disclaimer: I am not an MSYS2 expert by any means. I am an average user who maintains a couple of unofficial PKGBUILD scripts. My involvement with GitHub Actions and MSYS2 was the following so far:

During that period, I maintained my fork of setup-msys2 alive, in the hope that the "official" solution would get to be good enough for ditching the Action. That didn't happen. At the end of May, msys2/msys2-installer#5 was opened. It is worth mentioning that @MSP-Greg is apparently not aware of the most basic netiquette. He/she tries to sneak by splitting the discussions in multiple unnecessary threads and NOT pinging users which he/she knows would provide arguments discussing his/her proposals. I tried to explain what taking care of a codebase implies: #916 (comment). He/she responded with, msys2/msys2-installer#5, another effort to mess the discussion. So, at some point, I was fed up with trying to have an (at least) decent official MSYS2 solution on GitHub's side.

Fortunately, the reaction of MSYS2 maintainers was lovely. In a couple of weeks, there was an official Action and automatic caching was enabled. In July, lots of enhancements were contributed. Now, I help maintain msys2/setup-msys2, and I understand the codebase, but I'm not an expert on the technical details.

Note that the enhancements in setup-msys2 are compatible with the built-in MSYS2 install provided in this repo (see https://github.com/msys2/setup-msys2#release), but I am not aware of anyone using it since the startup time is typically worse than just ignoring it (see msys2/setup-msys2#23 (comment)).


Please, please, someone (employee, contributor, maintainer...) step back and think about the complexity of "having bash and git in Windows". If you are not familiar with some or any of the tools, that's ok. I believe that maintainers of Git for Windows, MSYS2, Chocolatey, etc. all will be delighted to have some public and clarifying discussion about how should all of them coexist in GitHub Actions (both, in the environments and in the ecosystem). But, please, stop doing fast and not properly thought modifications (after this issue, which is NEEDED). Users can solve their own typically complex ad-hoc problems faster than you (or anyone else) can even understand them. IMHO, you should focus on providing an stable environment that makes sense (any sense, you define it).

@MSP-Greg
Copy link
Contributor

MSP-Greg commented Sep 9, 2020

@eine

@MSP-Greg is apparently not aware of the most basic netiquette

I could say the same.

Re removing MSYS2:

  1. MSYS2 was/is installed on AppVeyor, and it was used for several years. Why is it now not appropriate for Actions to do the same? AppVeyor did not update their images often, which did cause issues. Given that Actions images are updated at least a few times a month, issues are almost nonexistent.

  2. You mention 90 seconds. For many repos CI, that is a significant amount of time. Additionally, the size of the MSYS2 download is far greater than all the other code needed for the CI.

  3. In some respects, the idea of removing MSYS2 is a partial application of the idea that CI shouldn't have any software installed. That idea doesn't seem to be considered practical.

  4. People are certainly able to use the MSYS2 action if it meets their needs.

  5. Since all these issues are related to Path, users or the Actions team can adjust it to met their needs. As long as software doesn't add files to the default Windows locations, and it isn't placed in Path, it's invisible to the system.

  6. I think many Actions users are using MSYS2 for the compile tools. Having them installed is beneficial, and updating them is simple and fast.

Hence, I feel that MSYS2 should remain installed.

My name is Greg, and I identify myself as male.

@dscho
Copy link
Contributor

dscho commented Sep 9, 2020

@eine @MSP-Greg I really value seeing both of your perspectives spelled out clearly. Maybe we could meet real quick on https://meet.jit.si/GHA-MSYS2 to hash some things out "in person"?

@dscho
Copy link
Contributor

dscho commented Sep 9, 2020

Maybe we could meet real quick on https://meet.jit.si/GHA-MSYS2 to hash some things out "in person"?

I'm going to log off for the day, but maybe we can agree on a time to meet face to face, to resolve any conflicts and work on a mutually satisfactory solution?

@MSP-Greg
Copy link
Contributor

MSP-Greg commented Sep 9, 2020

@dscho

Re meet.jit.si, never used it, just installed on my phone.

Obviously, I feel strongly that MSYS2 should be installed on the Actions Windows image(s).

Past that, I'm not sure. I've worked with MSYS2 for a while, build packages locally (most recently OpenSSL 3.0.0), but I'm not a strong c type. I've also used it to build Ruby master/main, and the build has been widely used in Ruby related CI for a few years. It was originally built and stored on AppVeyor, now it's also built on Actions.

I think you, @eine, and myself have dealt with MSYS2 enough to always 'make it work'. With custom actions, we can match the needs of other users and preset the environment for them. @eine and I are involved with separate actions to accomplish that.

Assuming that having MSYS2 installed but not enabled is ok, the issue here is what makes sense for the majority of Actions users. I'm not sure I know the answer to that.

To me, two main issues. How to configure the bash shell, and conflicts.

JFYI, I'm -0500 (central US)

@eine
Copy link

eine commented Sep 9, 2020

@MSP-Greg is apparently not aware of the most basic netiquette

I could say the same.

@MSP-Greg, sure, you are entitled. However, I explained why I thought it and what is the implication for other users. This is not about you and me, but about how our actions affect other users. I'd be so glad if you could warn them properly. Right now it's a childish "you too", again.

  1. MSYS2 was/is installed on AppVeyor, and it was used for several years. Why is it now not appropriate for Actions to do the same? AppVeyor did not update their images often, which did cause issues. Given that Actions images are updated at least a few times a month, issues are almost nonexistent.

This is a false premise that has been discussed and replied before. Please, bring the relevent references instead of trying to tendentiously make a null point, again. Basic netiquette.

  1. You mention 90 seconds. For many repos CI, that is a significant amount of time. Additionally, the size of the MSYS2 download is far greater than all the other code needed for the CI.

Again, false assertions without referencing relevant discussion which you are aware of. Bring them. Basic netiquette.

I did not mention, it is tested by tens, hundreds, thousands of users. Because of your decisions, the built-it solution needs up to 7-12min. It was not my decision/suggestion to ditch the built-in installation. A maintainer of MSYS2 suggested it after being surprised by all the packages installed by default: msys2/setup-msys2#23 (comment). Nonetheless, I agree with the decision, of course.

Note to readers, "for many repos CI" and "all the other code" do mean "his many repos" and "all his other code". There has been no analysis, at least he has not done any public, about the truthfulness of those mantras.

  1. In some respects, the idea of removing MSYS2 is a partial application of the idea that CI shouldn't have any software installed. That idea doesn't seem to be considered practical.

Ideally, MSYS2 would be available as a built-in resource. I said that. I did never propose that CI shouldn't have any software. That's something that you seem to have proposed out of nowhere.

What I said, and I still propose, is that no one seems to be taking care of the elephant in the room. Then, better open the garage door and let it run free.

  1. People are certainly able to use the MSYS2 action if it meets their needs.

Some people NEED to use the MSYS2 Action for avoiding your opinionated and arbitrary decisions. In order to reduce those 90s for your specific use case, you are forcing a +300s penalty in the workflows of all the users which want to use an updated solution. Your reply is: msys2/setup-msys2#23 (comment). And you were explicitly told that partial updates are NOT supported. This one is not about netiquette, but about basic etiquette, since it's other people's money that you are wasting.

Fortunately, thanks to MSYS2 maintainers, the Action can partially alleviate this severe design flaw.

  1. Since all these issues are related to Path, users or the Actions team can adjust it to met their needs. As long as software doesn't add files to the default Windows locations, and it isn't placed in Path, it's invisible to the system.

This is the premise of this issue. Here, "users" and "the Actions team" are discussing how to adjust the PATH for meeting their needs. We agree that removing everything from the PATH (and from the environment) is a solution for avoiding conflicts. However, I believe that's not the solution (integration) we are aiming for.

  1. I think many Actions users are using MSYS2 for the compile tools. Having them installed is beneficial, and updating them is simple and fast.

As explained several times, what you think is irrelevant, given your purposely limited point of view. Let me just place an counterexample:

- uses: msys2/setup-msys2@v2
  with:
    msystem: MINGW32
    update: true
- uses: ghdl/setup-ghdl-ci@nightly
  with:
    backend: mcode
- shell: msys2 {0}
  run: |
    ghdl --version

Currently, GHDL is the only open source solution for testing and synthesis of hardware designs written in VHDL.

Hence, I feel that MSYS2 should remain installed.

I disagree. MSYS2 should only remain installed if properly taken care of.

My name is Greg, and I identify myself as male.

Thanks.


@dscho, I prefer not to use either video or voice. However, I am open to engaging the discussion in any chat, pad, wiki... Nonetheless, as said, I believe I am not a relevant agent to take decisions. The discussion needs to be public and open to maintainers of other projects. Otherwise, we will be having this scope limited conversation again and again.

Two months ago, I proposed Greg to translate part of setup-msys2 to Powershell, so that it could be integrated in this environment (see msys2/msys2-installer#5 (comment)) and other environments. He passed, again.


@eine and I are involved with separate actions to accomplish that.

@MSP-Greg, as explained in #1525 (comment), I am involved with providing a usable environment in GitHub Actions that allows using MSYS2 without favouring a very specific subset of users and severily penalizing others. This is specially relevant because GitHub Actions are paid per minute, not per run.

The fact that I am involved in a general purpose Action is just a result of the poor built-in solution. I wish setup-msys2 did not exist. I am a maintainer of other projects, and I'd prefer to spend time on those. setup-msys2 is just a tool that I (and others) need for testing software on Windows.

It is quite tendencious on your side to put on the same page my involvement in setup-msys2 and your involvement in any of the actions you maintain. Neither the scope nor the motivation have any relation. Moreover, you were invited multiple times to help extending setup-msys2 for suiting your specific needs, instead of creating one or multiple custom Actions. You ignored those.

@dscho
Copy link
Contributor

dscho commented Sep 9, 2020

I prefer not to use either video or voice.

@eine fair enough. I only proposed it because I see that emotions run unnecessarily high, and in the past I found that a quick face-to-face conversation helped alleviate similar situations. There is just so much lost when using text-only communication, and it is very easy to make much more aggressive arguments than intended.

I am convinced that we can resolve this issue in a way that meets the needs of all (currently) involved parties.

@eine
Copy link

eine commented Sep 9, 2020

@dscho, just to clarify: I am not a native speaker, and my comments might sound harder than they should. I want them to be direct, and clear, not rude. I am neither angry, nor do I have any personal issue with Greg. I would love to collaborate and find a technical solution that fits all the use cases with as little conflicts as possible. That includes discussing technical details and objective decisions with Greg.

My frustration is motivated by how little attention is GitHub as a company paying to their GitHub Actions product. It came out of beta almost a year ago and the UX is still awful. They had (and still have) lots of references to learn from and to copy from. Yet, there is no captain sailing this boat. That is, Greg pushed his PRs and the environment is as it is now because of GitHub, not because of Greg. In any regular open source project, the review process would have arised all of this problems, and those would have been prevented.

EDIT

Of course, Greg (and any other) is absolutely entitled to replying "I don't want to do that", "I don't want to spend more time on this" or "I believe that employees should do it instead of me". Those are all valid points/replies in a discussion.

@eine
Copy link

eine commented Nov 16, 2020

I followed the references in #1525 (comment) and I didn't identify those as self-hosted agents. Could you please clarify how you reached that conclusion? My understanding is that self-hosted agents don't pull the virtual environment provided in this repo.

@dscho
Copy link
Contributor

dscho commented Nov 17, 2020

Sorry, I meant to say that they're trying to run their workflows in their own laptops and VMs as if they were self-hosted agents.

My understanding is that self-hosted agents don't pull the virtual environment provided in this repo.

That's exactly it! If your workflows don't require MSYS2, you shouldn't need it. But that's the exact opposite of what you suggested.

It's still a bad idea, in my opinion, to try to convince users that they have to use MSYS2 when they don't actually want to.

@eine
Copy link

eine commented Nov 21, 2020

I think there is a misunderstanding. I did not say that they have to use MSYS2. I suggested that their workflow might be significantly simplified if they used MSYS2 instead of Git for Windows. At least 30 lines can be removed straightaway. That is because Git for Windows does not suffice and they need to install an external package manager as well as setting up custom paths for the dependencies installed through that manager. Since the missing dependency is openssl-mingw, and because they said they just want to ensure the execution on Windows, I honestly think it is easier to maintain. Nonetheless, I said that they may also keep using Git for Windows after this issue is done (#1525 (comment)). Users did not say whether they want to use MSYS2 or not; that's up to them.

@igagis
Copy link

igagis commented Nov 23, 2020

The target date in 0th post is set to Nov 2, is the bash shell in windows-latest image already a msys2?

@MSP-Greg
Copy link
Contributor

MSP-Greg commented Nov 23, 2020

Using:

  Environment: windows-2019
  Version: 20201116.1

bash shell is listed in the log as C:\Program Files\Git\bin\bash.EXE

@igagis
Copy link

igagis commented Nov 23, 2020

I.e. target date from 0-th post is incorrect...

@umarcor
Copy link

umarcor commented Nov 24, 2020

The target date in 0th post is set to Nov 2, is the bash shell in windows-latest image already a msys2?

@igagis, I think it was set on date, but then reverted: #1525 (comment).

@maxim-lobanov
Copy link
Contributor

maxim-lobanov commented Nov 24, 2020

I have updated the first post to be clear:
This change was reverted for now because it broke some customers. We need to investigate it deeper and fix some issues before proceed with these changes

@igagis
Copy link

igagis commented Nov 28, 2020

Just for information for those who want to use msys2 as default bash shell for windows images already now.

To switch bash to be msys2 shell add the following step as a very first step to your job:

- name: switch git-bash shell to msys2 shell by adding msys2 path to PATH front
  run: echo "C:\msys64\usr\bin" >> $GITHUB_PATH
  shell: bash

After this step bash will refer to msys2.

@eine
Copy link

eine commented Nov 28, 2020

@igagis, @ericsampson, @pepijndevos, that was the reason for the change to be reverted, and what is delaying this issue being fixed.

Please, see options msystem and path-type in msys2/setup-msys2. You don't need to use that Action if you don't want to, but you should try to replicate the MSYS2/MINGW64/MINGW32 entrypoint: https://github.com/msys2/setup-msys2/blob/master/main.js#L164-L168 (note CHERE_INVOKING).

The easiest and most robust solution for using the built-in MSYS2 installation is the following:

  defaults:
    run:
      shell: msys2 {0}
  steps:

  - uses: msys2/setup-msys2@v2
    with:
      update: false
      release: false

  - run: |
      Your commands, already on msys2

  - run: |
      Your commands, NOT in msys2
    shell: bash

  - run: Some other command, also on msys2 (by defaults above)

See Usage for guidelines about how to execute either one command or a sequence of commands in MSYS2.

EDIT

An example using numpy, matplotlib and openssl inside MSYS2: https://github.com/ghdl/ghdl-cosim/blob/master/.github/workflows/test.yml#L67-L87

@eine
Copy link

eine commented Nov 28, 2020

As a further note, the reference workflow above should remain almost unaffected, regardless of the modifications to the virtual environments in this repo. That's because all MSYS2/MINGW32/MINGW64 shells are isolated from the system PATH by default. Users need to explicitly set an option for making it visible. That avoids most of the reported conflicts, per se. In fact, maintainers of this repo nicely avoided introducing aliases which would conflict with the msys2 entrypoint.

The second run command (shell: bash) is the only one that might see some effect (because it is Git for Windows ATM, and it will change to MSYS2).

My hope is that, at some point, this repository will borrow the entrypoint only from setup-msys2, or it will present it as the canonical action to be used for running MSYS2 shells.

dscho added a commit to git-for-windows/build-extra that referenced this issue Dec 1, 2020
The main purpose of the `minimal-sdk` artifact is to serve in automated
Azure Pipelines and GitHub workflows. Let's make sure that we are not
picking up any unintended executables along the way.

One such unintended executable would be Git for Windows' gpg.exe: Git
for Windows is installed in the build agents of Azure Pipelines and
GitHub Actions, and its `/usr/bin/` is sadly in the `PATH` and will most
likely remain there indefinitely, for details, see
actions/runner-images#1525.

However, this can lead to `gpg.exe` being picked up by Git's test suite,
and unfortunately the GPG agent tends to hang in the VMs in which the
tests are run.

So why does this not happen all the time? Most likely due to mismatches
of the MSYS2 runtime, which let `gpg.exe` fail to load (and which Git's
test suite takes as sign that "we looked all around but no GPG could be
found"). This is a fragile situation, of course: at any point, outside
of Git's (or Git for Windows') control, `gpg.exe` can start working
again.

By specifically overriding the `PATH` to only include the `bin/`
directories of the minimal SDK, plus Windows' system directories, we can
avoid this from ever happening again.

Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
@dscho
Copy link
Contributor

dscho commented Dec 1, 2020

Per communication with Git For Windows maintainers, we should not add Git Internal tools to PATH and rely on it. They could be removed in future and break a lot of people.

Okay. Well, we tried. And it seems that we simply have to add Git's internal tools to the PATH for eternity.

I guess I will have to keep the build agents in mind going forward, which binds my hands when it comes to e.g. removing Bash in favor of Dash or trying to use BusyBox' ash instead. An unideal situation for me as Git for Windows maintainer! But it apparently cannot be helped, that ship has now sailed.

@planetmarshall
Copy link

planetmarshall commented Dec 21, 2020

Is there a solution for using the MSYS2 tools in Azure Pipelines? I'm having some issues compiling a package on CI because the git tools do not correctly handle windows paths -

/bin/sh ../libtool --mode=compile --tag=RC windres --target=pe-x86-64 `C:/Program Files/Git/usr/bin/sh.exe ./../windows/windres-options --escape 1.16` -i ./../windows/libiconv.rc -o libiconv.res.lo --output-format=coff

/usr/bin/sh: C:/Program: No such file or directory

@dscho
Copy link
Contributor

dscho commented Dec 22, 2020

Is there a solution for using the MSYS2 tools in Azure Pipelines? I'm having some issues compiling a package on CI because the git tools do not correctly handle windows paths -

/bin/sh ../libtool --mode=compile --tag=RC windres --target=pe-x86-64 `C:/Program Files/Git/usr/bin/sh.exe ./../windows/windres-options --escape 1.16` -i ./../windows/libiconv.rc -o libiconv.res.lo --output-format=coff

/usr/bin/sh: C:/Program: No such file or directory

AFAICT you simply have to add C:\msys64\mingw64\bin and C:\msys64\usr\bin to the PATH (or some variation thereof)

@eine
Copy link

eine commented Dec 22, 2020

@planetmarshall I'd say it can handle Windows paths, but it cannot handle paths with spaces (without quoting them). So:

/bin/sh ../libtool --mode=compile --tag=RC windres --target=pe-x86-64  `'C:/Program Files/Git/usr/bin/sh.exe' ./../windows/windres-options --escape 1.16` -i ./../windows/libiconv.rc -o libiconv.res.lo --output-format=coff

@igagis
Copy link

igagis commented Dec 22, 2020

AFAICT you simply have to add C:\msys64\mingw64\bin and C:\msys64\usr\bin to the PATH (or some variation thereof)

It is better to use special action to switch to msys shell, as described in this comment: #1525 (comment)

@planetmarshall
Copy link

planetmarshall commented Dec 22, 2020

It is better to use special action to switch to msys shell, as described in this comment: #1525 (comment)

Is that applicable to Azure Pipelines?

@igagis
Copy link

igagis commented Dec 22, 2020

@planetmarshall I suppose, it is applicable as long as we are talking about github actions.

@eine
Copy link

eine commented Dec 22, 2020

@igagis I think that they can use an entrypoint such as https://github.com/msys2/setup-msys2/blob/master/main.js#L164-L168 in Azure Pipelines, but not the msys2/setup-msys2 Action. Nevertheless, the error seems to be related to an unquoted path containing an space.

@maxim-lobanov
Copy link
Contributor

maxim-lobanov commented Dec 28, 2020

Hello everyone!
We have decided to keep Git For Windows Bash in PATH for existing Windows images according to #1525 (comment)
We will consider switching default Bash to WSL or MSYS as a replacement of Git For Windows bash for future Windows images.

Unfortunately, there is no easy way to switch default Bash for existing images and don't affect customers.
Anyway, we are going to work on the following items:

  • Add entry points for different shells (winbash.exe, gitbash.exe, msysbash.exe), it is very good proposal and convenient way to switch shells in GitHub Actions
  • Improve delivery of MSYS2 #1572

@dscho , JFYI some other CI systems use Git for Windows as default Bash too and depend on pre-installed tools, for example Travis CI.

@eine
Copy link

eine commented Jan 10, 2021

FTR, how to setup shell and env in matrix when using setup-msys2:

/cc @eyal0 @ferdnyc

@ferdnyc
Copy link

ferdnyc commented Jan 11, 2021

I wonder if there shouldn't just be an msys2-latest image, separate from windows-latest. (Which would imply PowerShell, and MSVC, and all the other greatest hits. Whereas msys2-latest would be a GNU-for-Windows environment, complete with bash or other traditionally Unix shell, mingw-w64-x86_64-gcc, the whole works.)

Pretending they're NOT completely different environments hasn't really been to anyone's benefit so far, seems to me... And if the Windows people really want a bash shell in their windows-latest, then it sounds like someone may want to think about volunteering to take it off @dscho 's hands and maintain it as a separate project all on its own.

@eine
Copy link

eine commented Jan 11, 2021

And if the Windows people really want a bash shell in their windows-latest, then it sounds like someone may want to think about volunteering to take it off @dscho 's hands and maintain it as a separate project all on its own.

I'm no expert, but I believe that a "Windows bash" exists in windows-latest, apart from MSYS2, Git for Windows, or WSL. The motivation for using Git for Windows as the default bash on Windows environments was because windows-2016 does not have the "built-in" bash. It was done without much knowledge about the ecosystem and the implications. Several months later, the conflict arises from willing to fix the problem without requiring any effort from users relying on unsupported features, instead of documenting a forced break for technical reasons.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests