msys2-runtime-devel: Automatic upgrade through update-core fails #373

maehne · 2015-10-28T08:21:31Z

The update-core script for upgrading the MSYS2 core packages gets systematically stuck reporting fork errors when msys2-runtime-devel is installed and should get upgraded together with msys2-runtime:

$ update-core
==> Update package databases...
:: Synchronizing package databases...
 mingw32                  212.7 KiB   496K/s 00:00 [#####################] 100%
 mingw32.sig               96.0   B  0.00B/s 00:00 [#####################] 100%
 mingw64                  211.3 KiB   741K/s 00:00 [#####################] 100%
 mingw64.sig               96.0   B  93.8K/s 00:00 [#####################] 100%
 msys                     125.8 KiB   880K/s 00:00 [#####################] 100%
 msys.sig                  96.0   B  0.00B/s 00:00 [#####################] 100%
==> Checking if there are critical packages to upgrade.
pacman 4.2.1.6198.439d376-1 -> 4.2.1.6230.6ff894e-1
msys2-runtime 2.2.1.16655.6ecd782-2 -> 2.2.1.16667.d31de1b-1
==> WARNING: You MUST restart MSYS2 to finish core packages update!!!
==> Update core packages...
warning: bash-4.3.042-1 is up to date -- skipping
resolving dependencies...
looking for conflicting packages...

Packages (3) msys2-runtime-2.2.1.16667.d31de1b-1
             msys2-runtime-devel-2.2.1.16667.d31de1b-1
             pacman-4.2.1.6230.6ff894e-1

Total Download Size:    9.56 MiB
Total Installed Size:  54.23 MiB
Net Upgrade Size:      20.59 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages ...
 msys2-runtime-2.2.1...     2.3 MiB  1073K/s 00:02 [#####################] 100%
 pacman-4.2.1.6230.6...     3.8 MiB  1440K/s 00:03 [#####################] 100%
 msys2-runtime-devel...     3.5 MiB  1454K/s 00:02 [#####################] 100%
(3/3) checking keys in keyring                     [#####################] 100%
(3/3) checking package integrity                   [#####################] 100%
(3/3) loading package files                        [#####################] 100%
(3/3) checking for file conflicts                  [#####################] 100%
(3/3) checking available disk space                [#####################] 100%
(1/3) upgrading msys2-runtime                      [#####################] 100%
(2/3) upgrading pacman                             [#####################] 100%
(3/3) installing msys2-runtime-devel               [#####################] 100%
      0 [main] bash 2884 fork: child -1 - forked process 5124 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
1023699 [main] bash 2884 fork: child -1 - forked process 160 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
3051865 [main] bash 2884 fork: child -1 - forked process 4760 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
7079357 [main] bash 2884 fork: child -1 - forked process 6444 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
15096628 [main] bash 2884 fork: child -1 - forked process 6740 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: Resource temporarily unavailable
15115379 [main] bash 2884 fork: child -1 - forked process 6468 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
16130379 [main] bash 2884 fork: child -1 - forked process 7068 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
18156061 [main] bash 2884 fork: child -1 - forked process 5500 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
[...]

Once this error occured, the MSYS2 shell needs to be closed and the MSYS2 installation remains in an inconsistent state. Running update-core again won't finish the msys2-runtime-devel installation part of the core package update process.

I already reported this issue a month ago to the MSYS2 bug tracker on SourceForge, but there hasn't been any reaction. It has struck me several times since I started using MSYS in February. It also happened with the old manual upgrade instructions for the core packages from the MSYS2 wiki.

How could this problem be worked around to make update-core more robust?

The text was updated successfully, but these errors were encountered:

jpsdr · 2015-10-28T09:12:31Z

Try running the "autorebase.bat" file in your msys2 directory install after a core system update. I personnaly run it after each .

maehne · 2015-10-28T09:31:16Z

Thank you for your feedback! However, I don't see how this will resolve the issue with msys2-runtime-devel, as this package is upgraded automatically by update-core. Thus, the actions of autorebase.bat come too late. The endless loop of fork errors has already happened.

Alexpux · 2015-10-28T09:33:11Z

Maybe you have running some MSYS2 applications during update?

maehne · 2015-10-28T09:45:15Z

Thanks for the hint! My .bashrc is launching the process ssh-pageant.exe, which stays in the background. Next time I will run update-core, I will try to kill the ssh-pageant.exe process first. Hopefully, this will fix the issue for me. It would be nice if update-core could somehow warn about background processes, which may disturb its operation. If a general solution is not possible, a blacklist may help. For example, the ssh-pageant.exe process is seen by ps ax:

 $ ps ax | grep ssh-pageant
     2100       1    2100       2100  ?        1151398 08:20:38 /usr/bin/ssh-pageant

elieux · 2015-10-29T07:30:59Z

The autorebase script does a similar check, IIRC; patches are welcome.

FWIW, I don't have any daemons running, but I do get the same error with update-core (it has happened consistently last two or three times I ran it). Maybe there's something else at play, but I'll keep an open eye for the background process issue, as well.

maehne · 2015-11-02T09:54:21Z

This morning update-core wanted to upgrade msys2-runtime again. So, I tried whether killing ssh-pageant prior the update helps:

$ update-core
==> Update package databases...
:: Synchronizing package databases...
 mingw32                                      213.9 KiB   502K/s 00:00 [#######################################] 100%
 mingw32.sig                                   96.0   B  0.00B/s 00:00 [#######################################] 100%
 mingw64                                      212.5 KiB   748K/s 00:00 [#######################################] 100%
 mingw64.sig                                   96.0   B  0.00B/s 00:00 [#######################################] 100%
 msys                                         125.8 KiB   886K/s 00:00 [#######################################] 100%
 msys.sig                                      96.0   B  0.00B/s 00:00 [#######################################] 100%
==> Checking if there are critical packages to upgrade.
pacman 4.2.1.6230.6ff894e-1 -> 4.2.1.6254.98e6f9a-1
msys2-runtime 2.2.1.16667.d31de1b-1 -> 2.2.1.16681.8ba11f6-1
==> Core packages require updating.
==> Please close all other MSYS2 derived windows (e.g. terminal
==> windows, Bash sessions, etc) before proceeding.
==> WARNING: When the update has completed, you MUST close this MSYS2 window
==> WARNING: (use Alt-F4 or red [ X ], etc.), rather than 'exit'!!!
Press [Enter] key when ready to start update...
$ ps ax | grep ssh
     6488       1    6488       6488  ?        1151398 08:27:29 /usr/bin/ssh-pageant
$ kill 6488
$ ps ax | grep ssh
$ update-core
==> Update package databases...
:: Synchronizing package databases...
 mingw32 is up to date
 mingw64                                      212.8 KiB   500K/s 00:00 [#######################################] 100%
 mingw64.sig                                   96.0   B  0.00B/s 00:00 [#######################################] 100%
 msys is up to date
==> Checking if there are critical packages to upgrade.
pacman 4.2.1.6230.6ff894e-1 -> 4.2.1.6254.98e6f9a-1
msys2-runtime 2.2.1.16667.d31de1b-1 -> 2.2.1.16681.8ba11f6-1
==> Core packages require updating.
==> Please close all other MSYS2 derived windows (e.g. terminal
==> windows, Bash sessions, etc) before proceeding.
==> WARNING: When the update has completed, you MUST close this MSYS2 window
==> WARNING: (use Alt-F4 or red [ X ], etc.), rather than 'exit'!!!
Press [Enter] key when ready to start update...
==> Updating core packages...
warning: bash-4.3.042-1 is up to date -- skipping
resolving dependencies...
looking for conflicting packages...

Packages (3) msys2-runtime-2.2.1.16681.8ba11f6-1  msys2-runtime-devel-2.2.1.16681.8ba11f6-1
             pacman-4.2.1.6254.98e6f9a-1

Total Download Size:    9.54 MiB
Total Installed Size:  54.25 MiB
Net Upgrade Size:      20.54 MiB

:: Proceed with installation? [Y/n]
:: Retrieving packages ...
 msys2-runtime-2.2.1.16681.8ba11f6-1-x86_64     2.3 MiB  1083K/s 00:02 [#######################################] 100%
 pacman-4.2.1.6254.98e6f9a-1-x86_64             3.8 MiB  1500K/s 00:03 [#######################################] 100%
 msys2-runtime-devel-2.2.1.16681.8ba11f6...     3.5 MiB  1462K/s 00:02 [#######################################] 100%
(3/3) checking keys in keyring                                         [#######################################] 100%
(3/3) checking package integrity                                       [#######################################] 100%
(3/3) loading package files                                            [#######################################] 100%
(3/3) checking for file conflicts                                      [#######################################] 100%
(3/3) checking available disk space                                    [#######################################] 100%
(1/3) upgrading msys2-runtime                                          [#######################################] 100%
(2/3) upgrading pacman                                                 [#######################################] 100%
(3/3) installing msys2-runtime-devel                                   [#######################################] 100%
      0 [main] bash 3708 fork: child -1 - forked process 4312 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
1007456 [main] bash 3708 fork: child -1 - forked process 1232 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
3040337 [main] bash 3708 fork: child -1 - forked process 7256 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
7071038 [main] bash 3708 fork: child -1 - forked process 7060 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: retry: No child processes
15088608 [main] bash 3708 fork: child -1 - forked process 8168 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
bash: fork: Resource temporarily unavailable
      0 [main] mc 6832 fork: child -1 - forked process 8080 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
                                                                                                                     Cannot spawn the subshell process: Resource temporarily unavailable (11)
      0 [main] bash 3176 fork: child -1 - forked process 8084 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
  bash: fork: retry: No child processes
                                       1030679 [main] bash 3176 fork: child -1 - forked process 7784 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
                                         bash: fork: retry: No child processes
                                                                              3046944 [main] bash 3176 fork: child -1 - forked process 2064 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
                                                                                bash: fork: retry: No child processes
                                                                                                                     7195958 [main] bash 3176 fork: child -1 - forked process 8008 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
 bash: fork: retry: No child processes
                                      15213330 [main] bash 3176 fork: child -1 - forked process 7748 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
                                         bash: fork: Resource temporarily unavailable
                                                                                     15232709 [main] bash 3176 fork: child -1 - forked process 6412 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
                                                                                        bash: fork: retry: No child processes
       16253217 [main] bash 3176 fork: child -1 - forked process 4228 died unexpectedly, retry 0, exit code 0xC0000005, errno 11
[...]

As you can see, the issue remains even if no daemon is running in the background: Upgrading of msys2-runtime and pacman succeeds, but installing msys2-runtime-devel fails with the fork error.

Alexpux · 2015-11-02T09:57:02Z

I can't reproduce this:

warlock+alexx@WARLOCK MSYS ~
$ ps -a
      PID    PPID    PGID     WINPID   TTY         UID    STIME COMMAND
     7800    5468    7800       9116  pty0      197608 12:55:56 /usr/bin/ps
     5468    1860    5468       3088  pty0      197608   Oct 20 /usr/bin/bash
     1860       1    1860       1860  ?         197608   Oct 20 /usr/bin/mintty

warlock+alexx@WARLOCK MSYS ~
$ update-core
==> Update package databases...
:: Синхронизируются базы данных пакетов...
 mingw32                  213,9 KiB   209M/s 00:00 [#####################] 100%
 mingw32.sig               96,0   B  0,00B/s 00:00 [#####################] 100%
 mingw64                  212,8 KiB   208M/s 00:00 [#####################] 100%
 mingw64.sig               96,0   B  0,00B/s 00:00 [#####################] 100%
 msys                     125,8 KiB   123M/s 00:00 [#####################] 100%
 msys.sig                  96,0   B  93,8K/s 00:00 [#####################] 100%
==> Checking if there are critical packages to upgrade.
pacman 4.2.1.6230.6ff894e-1 -> 4.2.1.6254.98e6f9a-1
msys2-runtime 2.2.1.16667.d31de1b-1 -> 2.2.1.16681.8ba11f6-1
==> Core packages require updating.
==> Please close all other MSYS2 derived windows (e.g. terminal
==> windows, Bash sessions, etc) before proceeding.
==> ВНИМАНИЕ: When the update has completed, you MUST close this MSYS2 window
==> ВНИМАНИЕ: (use Alt-F4 or red [ X ], etc.), rather than 'exit'!!!
Press [Enter] key when ready to start update...
==> Updating core packages...
предупреждение: bash-4.3.042-1 не устарел -- пропускается
разрешение зависимостей...
проверка конфликтов...

Пакеты (3) msys2-runtime-2.2.1.16681.8ba11f6-1
           msys2-runtime-devel-2.2.1.16681.8ba11f6-1
           pacman-4.2.1.6254.98e6f9a-1

Будет установлено:  54,25 MiB
Изменение размера:   0,01 MiB

:: Приступить к установке? [Y/n]
(3/3) проверка ключей                              [#####################] 100%
(3/3) проверяется целостность пакета               [#####################] 100%
(3/3) загрузка файлов пакетов                      [#####################] 100%
(3/3) проверка конфликтов файлов                   [#####################] 100%
(3/3) проверяется доступное место                  [#####################] 100%
(1/3) обновление msys2-runtime                     [#####################] 100%
(2/3) обновление pacman                            [#####################] 100%
(3/3) обновление msys2-runtime-devel               [#####################] 100%

warlock+alexx@WARLOCK MSYS ~
$

maehne · 2015-11-02T16:06:10Z

To advance on this issue, is there any easy way to rollback an MSYS2 installation to older versions of the core packages so that it could be more easily tested what other background processes could influence the execution of update-core?

DavidEGrayson · 2015-11-02T16:30:19Z

@maehne: What is the value of $PS1 and $PROMPT_COMMAND in your shell?

maehne · 2015-11-02T16:36:05Z

Here are the requested values for $PS1 and $PROMPT_COMMAND:

 $ echo $PS1
\[\033[00;32m\]\u\[\033[00;32m\]@\[\033[00;32m\]\h:\[\033[01;34m\]\W \[\033[31m\]$(scm_ps1)\[\033[00m\]$\[\033[00m\]
$ echo $PROMPT_COMMAND

$

scm_ps1 is defined in .bashrc as:

        . /usr/share/git/git-prompt.sh
        scm_ps1() {
            local s=
            if [[ -d ".svn" ]] ; then
                s=\(svn:$(svn info | sed -n -e '/^Revision: \([0-9]*\).*$/s//\1/p' )\)
            else
                s=$(__git_ps1 "(git:%s)")
            fi
            echo -n "$s"
        }
        case "$TERM" in
            xterm*|rxvt*)
                export PS1="\[\033[00;32m\]\u\[\033[00;32m\]@\[\033[00;32m\]\h:\[\033[01;34m\]\W \[\033[31m\]\$(scm_ps1)\[\033[00m\]$\[\033[00m\] "
                ;;
            *)
                export PS1="\u@\h:\W \$(scm_ps1)$ "
                ;;
        esac

DavidEGrayson · 2015-11-02T16:39:18Z

I think the fact that PS1 runs a subcommand (scm_ps1) is causng problems. I have a similar setup and I always run this before allowing update-core to update things:

export PS1='$ '

In my experience, that is necessary to avoid errors.

maehne · 2015-11-02T16:47:14Z

Thank you for this valuable hint! Next time update-core is about to upgrade core packages, I will set PS1 as you suggest it. If it works, this should probably be done by update-core as a pre-caution. Anyway, the terminal window will have to be closed after the upgrade.

DavidEGrayson · 2015-11-02T16:53:20Z

If you want to test it sooner, you could always make a fresh, temporary installation of MSYS2 in a new directory and run update-core to reproduce the problem, and then retry it and see if anything changes when you change PS1. That's basically how I narrowed down on the problem. I wanted to figure out how to install MSYS2 properly and I did that experiment until it worked.

PROMPT_COMMAND to harmless values to avoid this problem: msys2/MSYS2-packages#373 Note that this patch alone is not sufficient to fix the problem; we also have to make sure that update-core gets sourced so that it can modify the environment variables of the user's shell.

This prevents control from returning to the parent shell, which might in turn try to execute commands found inside PS1 and PROMPT_COMMAND. This commit should fix issues like this: msys2/MSYS2-packages#373

elieux · 2015-11-03T08:33:08Z

Cool. That may be the reason -- I, too, have a git-enhanced prompt.

DavidEGrayson · 2015-11-03T13:31:29Z

With the recent pull requests I made to MSYS2-pacman (see above), this issue should be fixed.

maehne · 2015-11-03T14:48:56Z

Dear David,

thank you very much for looking into this issue! I can confirm that the endless loop in update-core prevents fork error triggered by my git-enhanced shell prompt. While testing on my side your fix, I just had problems to close the mintty window via the X-button or ALT-F4. The dialog warning of the running process appeared, but clicking OK wouldn't close the window. I needed to force the ending of the mintty.exe process using the Windows Task Manager

I tested this with a new installation from msys2-base-x86_64-20150916.tar.xz, in which I installed Git, ssh-pageant and my .bashrc file.

DavidEGrayson · 2015-11-03T14:50:23Z

Hmm, that's too bad. I was able to close the Window with no issues besides having to click "OK" in the message box that pops up.

maehne · 2015-11-24T12:16:55Z

Today, I was first able to test the updated update-core script. with the endless loop to prevent the fork error in an installation, where I didn't have to patch in your modifications myself. The fork error is prevented by the loop. However, closing the mintty window with Alt-F4 or the X-button does not. The warning dialog about running processes appears and clicking "OK" would not close the window. The issue is not resolved by killing ssh-pageant.exe through the task manager. Killing bash.exe yields the fork error and then the window can be immediately closed through either Alt-F4 or the X-button. Maybe, there's a bug in the mintty.exe warning dialog?

DavidEGrayson · 2015-11-24T18:54:04Z

That's too bad. If you could make some minimal steps (that don't involve running pacman) to reproduce the problem then maybe someone will be able to see what is going on and fix it.

elieux · 2015-11-25T12:59:03Z

Could be the same underlying issue: git-for-windows/git#545

dscho · 2015-11-26T11:32:42Z

@elieux did you mean git-for-windows/git#537?

EDIT: Oops, sorry, I did not realize that a separate bug was reported as a comment to this otherwise unrelated ticket. Sorry for the noise.

maehne · 2015-12-02T12:48:01Z

I was able to work around the issue today by running update-core from within a MSYS2 bash shell in ConEmu. The latter accepts closing the terminal window even though the script is still running. Still, I would prefer that cause for this issue could be fixed in mintty ConEmu doesn't play as nicely together with more complex Unix-based console applications such as vim or mc.

DavidEGrayson mentioned this issue Nov 2, 2015

update-core: If we are updating core packages, set PS1 and Alexpux/MSYS2-pacman#20

Merged

DavidEGrayson mentioned this issue Nov 3, 2015

Added an infinite loop after updating packages. Alexpux/MSYS2-pacman#21

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

msys2-runtime-devel: Automatic upgrade through update-core fails #373

msys2-runtime-devel: Automatic upgrade through update-core fails #373

maehne commented Oct 28, 2015

jpsdr commented Oct 28, 2015

maehne commented Oct 28, 2015

Alexpux commented Oct 28, 2015

maehne commented Oct 28, 2015

elieux commented Oct 29, 2015

maehne commented Nov 2, 2015

Alexpux commented Nov 2, 2015

maehne commented Nov 2, 2015

DavidEGrayson commented Nov 2, 2015

maehne commented Nov 2, 2015

DavidEGrayson commented Nov 2, 2015

maehne commented Nov 2, 2015

DavidEGrayson commented Nov 2, 2015

elieux commented Nov 3, 2015

DavidEGrayson commented Nov 3, 2015

maehne commented Nov 3, 2015

DavidEGrayson commented Nov 3, 2015

maehne commented Nov 24, 2015

DavidEGrayson commented Nov 24, 2015

elieux commented Nov 25, 2015

dscho commented Nov 26, 2015

maehne commented Dec 2, 2015

msys2-runtime-devel: Automatic upgrade through update-core fails #373

msys2-runtime-devel: Automatic upgrade through update-core fails #373

Comments

maehne commented Oct 28, 2015

jpsdr commented Oct 28, 2015

maehne commented Oct 28, 2015

Alexpux commented Oct 28, 2015

maehne commented Oct 28, 2015

elieux commented Oct 29, 2015

maehne commented Nov 2, 2015

Alexpux commented Nov 2, 2015

maehne commented Nov 2, 2015

DavidEGrayson commented Nov 2, 2015

maehne commented Nov 2, 2015

DavidEGrayson commented Nov 2, 2015

maehne commented Nov 2, 2015

DavidEGrayson commented Nov 2, 2015

elieux commented Nov 3, 2015

DavidEGrayson commented Nov 3, 2015

maehne commented Nov 3, 2015

DavidEGrayson commented Nov 3, 2015

maehne commented Nov 24, 2015

DavidEGrayson commented Nov 24, 2015

elieux commented Nov 25, 2015

dscho commented Nov 26, 2015

maehne commented Dec 2, 2015